5 Commits

Author SHA1 Message Date
Spitap
4e2e9b6ab9 Revert "Make pip quiet"
This reverts commit 7ccc871da7.
2024-02-05 12:27:17 +01:00
Spitap
7ccc871da7 Make pip quiet 2024-02-05 12:23:06 +01:00
Spitap
5f4817736f Increased verbosity 2024-02-05 11:52:26 +01:00
Spitfireap
804c20a18d fixed output not decoded 2024-02-01 13:13:06 +01:00
Spitfireap
dd32b21ce9 Improved version checking 2024-02-01 13:07:11 +01:00
18 changed files with 99 additions and 313 deletions

View File

@@ -11,29 +11,29 @@ jobs:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: [3.8, 3.9, '3.10', '3.11']
python-version: [3.7, 3.8, 3.9]
fail-fast: false
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v2
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: |
pip install -r test-requirements.txt
- name: Run tests
if: ${{ matrix.python-version != '3.11' }}
if: ${{ matrix.python-version != '3.9' }}
run: |
python tests.py
- name: Run tests and coverage
if: ${{ matrix.python-version == '3.11' }}
if: ${{ matrix.python-version == '3.9' }}
run: |
coverage run tests.py
- name: Upload coverage result
if: ${{ matrix.python-version == '3.11' }}
uses: actions/upload-artifact@v4
if: ${{ matrix.python-version == '3.9' }}
uses: actions/upload-artifact@v2
with:
name: coverage-results
path: .coverage
@@ -42,16 +42,16 @@ jobs:
needs: test
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v2
- name: Set up Python
uses: actions/setup-python@v5
uses: actions/setup-python@v2
with:
python-version: '3.11'
python-version: '3.9'
- name: Install dependencies
run: |
pip install codecov
- name: Download coverage results
uses: actions/download-artifact@v4
uses: actions/download-artifact@v2
with:
name: coverage-results
- name: Report coverage

View File

@@ -1,32 +0,0 @@
name: Update version file
on:
workflow_run:
branches: [ master ]
workflows: [Modoboa installer]
types:
- completed
jobs:
update-version:
runs-on: ubuntu-latest
steps:
- name: Checkout repo
uses: actions/checkout@v4
with:
fetch-depth: 0 # otherwise, there would be errors pushing refs to the destination repository.
ref: ${{ github.head_ref }}
- name: Overwrite file
uses: "DamianReeves/write-file-action@master"
with:
path: version.txt
write-mode: overwrite
contents: ${{ github.sha }}
- name: Commit & Push
uses: Andro999b/push@v1.3
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
branch: ${{ github.ref_name }}
force: true
message: '[GitHub Action] Updated version file'

View File

@@ -1,5 +1,5 @@
**modoboa-installer**
=====================
modoboa-installer
=================
|workflow| |codecov|
@@ -9,8 +9,9 @@ An installer which deploy a complete mail server based on Modoboa.
This tool is still in beta stage, it has been tested on:
* Debian 10 and upper
* Debian Buster (10) / Bullseye (11)
* Ubuntu Bionic Beaver (18.04) and upper
* CentOS 7
.. warning::
@@ -76,7 +77,7 @@ If you want more information about the installation process, add the
``--debug`` option to your command line.
Upgrade mode
============
------------
An experimental upgrade mode is available.
@@ -92,7 +93,7 @@ You can activate it as follows::
It will automatically install latest versions of modoboa and its plugins.
Backup mode
===========
------------
An experimental backup mode is available.
@@ -107,19 +108,7 @@ You can start the process as follows::
Then follow the step on the console.
There is also a non-interactive mode::
$ sudo ./run.py --silent-backup <your domain>
You can also add a path, else it will be saved in ./modoboa_backup/Backup_M_Y_d_H_M::
$ sudo ./run.py --silent-backup --backup-path "/My_Backup_Path" <your domain>
if you want to disable mail backup::
$ sudo ./run.py --backup --no-mail <your domain>
This can be useful for larger instance
There is also a non-interactive mode:
1. Silent mode
@@ -141,7 +130,7 @@ configuration file (set enabled to False).
This can be useful for larger instance.
Restore mode
============
------------
An experimental restore mode is available.
@@ -152,7 +141,7 @@ You can start the process as follows::
Then wait for the process to finish.
Change the generated hostname
=============================
-----------------------------
By default, the installer will setup your email server using the
following hostname: ``mail.<your domain>``. If you want a different
@@ -171,24 +160,12 @@ modifications.
Finally, run the installer without the
``--stop-after-configfile-check`` option.
Certificate
===========
Self-signed
-----------
It is the default type of certificate the installer will generate, it
is however not recommended for production use.
Letsencrypt
-----------
Let's Encrypt certificate
-------------------------
.. warning::
Please note that by using this option, you agree to the `ToS
<https://community.letsencrypt.org/tos>`_ of
letsencrypt and that your IP will be logged (see ToS).
Please also note this option requires the hostname you're using to be
Please note this option requires the hostname you're using to be
valid (ie. it can be resolved with a DNS query) and to match the
server you're installing Modoboa on.
@@ -199,8 +176,6 @@ modify the following settings::
[certificate]
generate = true
type = letsencrypt
tls_cert_file_path =
tls_key_file_path =
[letsencrypt]
email = admin@example.com
@@ -208,27 +183,6 @@ modify the following settings::
Change the ``email`` setting to a valid value since it will be used
for account recovery.
Manual
------
.. warning::
It is not possible to configure manual certs interactively, so
you'll have to do it in 2 steps. Please run ``run.py`` with
`--stop-after-configfile-check` first, configure your file as
desired and apply the configuration as written bellow. Then run
``run.py`` again but without `--stop-after-configfile-check` or
`--interactive`.
If you want to use already generated certs, simply edit the
``installer.cfg`` file and modify the following settings::
[certificate]
generate = true
type = manual
tls_cert_file_path = *path to tls fullchain file*
tls_key_file_path = *path to tls key file*
.. |workflow| image:: https://github.com/modoboa/modoboa-installer/workflows/Modoboa%20installer/badge.svg
.. |codecov| image:: http://codecov.io/github/modoboa/modoboa-installer/coverage.svg?branch=master
:target: http://codecov.io/github/modoboa/modoboa-installer?branch=master

View File

@@ -1,37 +0,0 @@
"""Checks to be performed before any install or upgrade"""
import sys
from urllib.request import urlopen
from modoboa_installer import utils
def check_version():
local_version = ""
with open("version.txt", "r") as version:
local_version = version.readline()
remote_version = ""
with urlopen("https://raw.githubusercontent.com/modoboa/modoboa-installer/master/version.txt") as r_version:
remote_version = r_version.read().decode()
if local_version == "" or remote_version == "":
utils.printcolor(
"Could not check that your installer is up-to-date: "
f"local version: {local_version}, "
f"remote version: {remote_version}",
utils.YELLOW
)
if remote_version != local_version:
utils.error(
"Your installer seems outdated.\n"
"Check README file for instructions about how to update.\n"
"No support will be provided without an up-to-date installer!"
)
answer = utils.user_input("Continue anyway? (Y/n) ")
if not answer.lower().startswith("y"):
sys.exit(0)
else:
utils.success("Installer seems up to date!")
def handle():
check_version()

View File

@@ -21,14 +21,13 @@ COMPATIBILITY_MATRIX = {
"modoboa-sievefilters": ">=1.1.1",
"modoboa-webmail": ">=1.2.0",
},
"2.1.0": {
"modoboa-pdfcredentials": None,
"modoboa-dmarc": None,
"modoboa-imap-migration": None,
},
}
EXTENSIONS_AVAILABILITY = {
"modoboa-contacts": "1.7.4",
}
REMOVED_EXTENSIONS = {
"modoboa-pdfcredentials": "2.1.0",
"modoboa-dmarc": "2.1.0",
"modoboa-imap-migration": "2.1.0"
}

View File

@@ -30,21 +30,16 @@ ConfigDictTemplate = [
{
"name": "certificate",
"values": [
{
"option": "generate",
"default": "true",
},
{
"option": "type",
"default": "self-signed",
"customizable": True,
"question": "Please choose your certificate type",
"values": ["self-signed", "letsencrypt", "manual"],
"non_interactive_values": ["manual"],
},
{
"option": "tls_cert_file_path",
"default": ""
},
{
"option": "tls_key_file_path",
"default": ""
"values": ["self-signed", "letsencrypt"],
}
],
},

View File

@@ -57,12 +57,12 @@ class DEBPackage(Package):
def install(self, name):
"""Install a package."""
self.update()
utils.exec_cmd("apt-get -o Dpkg::Progress-Fancy=0 install --quiet --assume-yes -o DPkg::options::=--force-confold {}".format(name))
utils.exec_cmd("apt-get -o Dpkg::Progress-Fancy=0 install --quiet --assume-yes {}".format(name))
def install_many(self, names):
"""Install many packages."""
self.update()
return utils.exec_cmd("apt-get -o Dpkg::Progress-Fancy=0 install --quiet --assume-yes -o DPkg::options::=--force-confold {}".format(
return utils.exec_cmd("apt-get -o Dpkg::Progress-Fancy=0 install --quiet --assume-yes {}".format(
" ".join(names)))
def get_installed_version(self, name):
@@ -108,7 +108,7 @@ def get_backend():
"""Return the appropriate package backend."""
distname = utils.dist_name()
backend = None
if distname in ["debian", "debian gnu/linux", "ubuntu", "linuxmint"]:
if distname in ["debian", "debian gnu/linux", "ubuntu"]:
backend = DEBPackage
elif "centos" in distname:
backend = RPMPackage

View File

@@ -1,5 +1,6 @@
"""Python related tools."""
import json
import os
import sys
@@ -48,22 +49,21 @@ def install_packages(names, venv=None, upgrade=False, **kwargs):
def get_package_version(name, venv=None, **kwargs):
"""Returns the version of an installed package."""
cmd = "{} show {}".format(
get_pip_path(venv),
name
)
cmd = f"{get_pip_path(venv)} list --format json"
exit_code, output = utils.exec_cmd(cmd, **kwargs)
if exit_code != 0:
utils.error(f"Failed to get version of {name}. "
f"Output is: {output}")
sys.exit(1)
print(f"name: {name}, venv: {venv}, cmd: {cmd}, exit_code: {exit_code}, output: {output.decode()}")
list_dict = json.loads(output.decode())
version_list = []
for element in list_dict:
if element["name"] == name:
version_list = element["version"].split(".")
break
version_list_clean = []
for line in output.decode().split("\n"):
if not line.startswith("Version:"):
continue
version_item_list = line.split(":")
version_list = version_item_list[1].split(".")
for element in version_list:
try:
version_list_clean.append(int(element))
@@ -72,11 +72,6 @@ def get_package_version(name, venv=None, **kwargs):
f"Failed to decode some part of the version of {name}",
utils.YELLOW)
version_list_clean.append(element)
if len(version_list_clean) == 0:
utils.printcolor(
f"Failed to find the version of {name}",
utils.RED)
sys.exit(1)
return version_list_clean
@@ -99,10 +94,16 @@ def install_package_from_remote_requirements(url, venv=None, **kwargs):
utils.exec_cmd(cmd, **kwargs)
def setup_virtualenv(path, sudo_user=None):
def setup_virtualenv(path, sudo_user=None, python_version=2):
"""Install a virtualenv if needed."""
if os.path.exists(path):
return
if python_version == 2:
python_binary = "python"
packages = ["python-virtualenv"]
if utils.dist_name() == "debian":
packages.append("virtualenv")
else:
if utils.dist_name().startswith("centos"):
python_binary = "python3"
packages = ["python3"]
@@ -111,5 +112,8 @@ def setup_virtualenv(path, sudo_user=None):
packages = ["python3-venv"]
package.backend.install_many(packages)
with utils.settings(sudo_user=sudo_user):
if python_version == 2:
utils.exec_cmd("virtualenv {}".format(path))
else:
utils.exec_cmd("{} -m venv {}".format(python_binary, path))
install_packages(["pip", "setuptools"], venv=path, upgrade=True)
install_packages(["pip", "setuptools\<58.0.0"], venv=path, upgrade=True)

View File

@@ -56,7 +56,8 @@ class Automx(base.Installer):
def _setup_venv(self):
"""Prepare a python virtualenv."""
python.setup_virtualenv(self.venv_path, sudo_user=self.user)
python.setup_virtualenv(
self.venv_path, sudo_user=self.user, python_version=3)
packages = [
"future", "lxml", "ipaddress", "sqlalchemy < 2.0", "python-memcached",
"python-dateutil", "configparser"

View File

@@ -150,6 +150,7 @@ postscreen_dnsbl_sites =
zen.spamhaus.org=127.0.0.[2..11]*3
bl.spameatingmonkey.net=127.0.0.2*2
bl.spamcop.net=127.0.0.2
dnsbl.sorbs.net=127.0.0.[2..15]
postscreen_dnsbl_threshold = 3
postscreen_dnsbl_action = enforce

View File

@@ -149,4 +149,4 @@ autoreply unix - n n - - pipe
%{amavis_enabled} -o smtpd_client_connection_count_limit=0
%{amavis_enabled} -o smtpd_client_connection_rate_limit=0
%{amavis_enabled} -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks
%{amavis_enabled} -o local_header_rewrite_clients=permit_mynetworks,permit_sasl_authenticated
%{amavis_enabled} -o local_header_rewrite_clients=

View File

@@ -65,31 +65,21 @@ class Modoboa(base.Installer):
def is_extension_ok_for_version(self, extension, version):
"""Check if extension can be installed with this modo version."""
if extension not in compatibility_matrix.EXTENSIONS_AVAILABILITY:
return True
version = utils.convert_version_to_int(version)
if extension in compatibility_matrix.EXTENSIONS_AVAILABILITY:
min_version = compatibility_matrix.EXTENSIONS_AVAILABILITY[extension]
min_version = utils.convert_version_to_int(min_version)
return version >= min_version
if extension in compatibility_matrix.REMOVED_EXTENSIONS:
max_version = compatibility_matrix.REMOVED_EXTENSIONS[extension]
max_version = utils.convert_version_to_int(max_version)
return version < max_version
return True
def _setup_venv(self):
"""Prepare a dedicated virtualenv."""
python.setup_virtualenv(self.venv_path, sudo_user=self.user)
python.setup_virtualenv(
self.venv_path, sudo_user=self.user, python_version=3)
packages = ["rrdtool"]
version = self.config.get("modoboa", "version")
if version == "latest":
packages += ["modoboa"] + self.extensions
for extension in list(self.extensions):
if extension in compatibility_matrix.REMOVED_EXTENSIONS.keys():
self.extensions.remove(extension)
self.extensions = [
extension for extension in self.extensions
if extension not in compatibility_matrix.REMOVED_EXTENSIONS
]
else:
matrix = compatibility_matrix.COMPATIBILITY_MATRIX[version]
packages.append("modoboa=={}".format(version))

View File

@@ -21,7 +21,7 @@ class Nginx(base.Installer):
def get_template_context(self, app):
"""Additionnal variables."""
context = super().get_template_context()
context = super(Nginx, self).get_template_context()
context.update({
"app_instance_path": (
self.config.get(app, "instance_path")),

View File

@@ -31,7 +31,8 @@ class Radicale(base.Installer):
def _setup_venv(self):
"""Prepare a dedicated virtualenv."""
python.setup_virtualenv(self.venv_path, sudo_user=self.user)
python.setup_virtualenv(
self.venv_path, sudo_user=self.user, python_version=3)
packages = [
"Radicale", "radicale-dovecot-auth", "pytz"
]

View File

@@ -7,7 +7,7 @@ from . import package
from . import utils
class CertificateBackend:
class CertificateBackend(object):
"""Base class."""
def __init__(self, config):
@@ -24,44 +24,13 @@ class CertificateBackend:
return False
return True
def generate_cert(self):
"""Create a certificate."""
pass
class ManualCertificate(CertificateBackend):
"""Use certificate provided."""
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
path_correct = True
self.tls_cert_file_path = self.config.get("certificate",
"tls_cert_file_path")
self.tls_key_file_path = self.config.get("certificate",
"tls_key_file_path")
if not os.path.exists(self.tls_key_file_path):
utils.error("'tls_key_file_path' path is not accessible")
path_correct = False
if not os.path.exists(self.tls_cert_file_path):
utils.error("'tls_cert_file_path' path is not accessible")
path_correct = False
if not path_correct:
sys.exit(1)
self.config.set("general", "tls_key_file",
self.tls_key_file_path)
self.config.set("general", "tls_cert_file",
self.tls_cert_file_path)
class SelfSignedCertificate(CertificateBackend):
"""Create a self signed certificate."""
def __init__(self, *args, **kwargs):
"""Sanity checks."""
super().__init__(*args, **kwargs)
super(SelfSignedCertificate, self).__init__(*args, **kwargs)
if self.config.has_option("general", "tls_key_file"):
# Compatibility
return
@@ -96,7 +65,7 @@ class LetsEncryptCertificate(CertificateBackend):
def __init__(self, *args, **kwargs):
"""Update config."""
super().__init__(*args, **kwargs)
super(LetsEncryptCertificate, self).__init__(*args, **kwargs)
self.hostname = self.config.get("general", "hostname")
self.config.set("general", "tls_cert_file", (
"/etc/letsencrypt/live/{}/fullchain.pem".format(self.hostname)))
@@ -146,24 +115,12 @@ class LetsEncryptCertificate(CertificateBackend):
cfg_file = "/etc/letsencrypt/renewal/{}.conf".format(self.hostname)
pattern = "s/authenticator = standalone/authenticator = nginx/"
utils.exec_cmd("perl -pi -e '{}' {}".format(pattern, cfg_file))
with open("/etc/letsencrypt/renewal-hooks/deploy/reload-services.sh", "w") as fp:
fp.write(f"""#!/bin/bash
HOSTNAME=$(basename $RENEWED_LINEAGE)
if [ "$HOSTNAME" = "{self.hostname}" ]
then
systemctl reload dovecot
systemctl reload postfix
fi
""")
def get_backend(config):
"""Return the appropriate backend."""
cert_type = config.get("certificate", "type")
if cert_type == "letsencrypt":
if not config.getboolean("certificate", "generate"):
return None
if config.get("certificate", "type") == "letsencrypt":
return LetsEncryptCertificate(config)
if cert_type == "manual":
return ManualCertificate(config)
return SelfSignedCertificate(config)

View File

@@ -316,17 +316,6 @@ def get_entry_value(entry, interactive):
if entry.get("values") and user_value != "":
user_value = values[int(user_value)]
non_interactive_values = entry.get("non_interactive_values", [])
if user_value in non_interactive_values:
error(
f"{user_value} cannot be set interactively. "
"Please configure installer.cfg manually by running "
"'python3 run.py --stop-after-configfile-check domain'. "
"Check modoboa-installer README for more information."
)
sys.exit(1)
return user_value if user_value else default_value

45
run.py
View File

@@ -11,7 +11,6 @@ except ImportError:
import ConfigParser as configparser
import sys
import checks
from modoboa_installer import compatibility_matrix
from modoboa_installer import constants
from modoboa_installer import package
@@ -38,12 +37,6 @@ PRIMARY_APPS = [
def installation_disclaimer(args, config):
"""Display installation disclaimer."""
hostname = config.get("general", "hostname")
utils.printcolor(
"Notice:\n"
"It is recommanded to run this installer on a FRESHLY installed server.\n"
"(ie. with nothing special already installed on it)\n",
utils.CYAN
)
utils.printcolor(
"Warning:\n"
"Before you start the installation, please make sure the following "
@@ -54,7 +47,7 @@ def installation_disclaimer(args, config):
hostname.replace(".{}".format(args.domain), ""),
hostname
),
utils.YELLOW
utils.CYAN
)
utils.printcolor(
"Your mail server will be installed with the following components:",
@@ -120,9 +113,6 @@ def backup_system(config, args):
utils.copy_file(args.configfile, backup_path)
# Backup applications
for app in PRIMARY_APPS:
if app == "dovecot" and args.no_mail:
utils.printcolor("Skipping mail backup", utils.BLUE)
continue
scripts.backup(app, config, backup_path)
@@ -174,17 +164,11 @@ def main(input_args):
help="For script usage, do not require user interaction "
"backup will be saved at ./modoboa_backup/Backup_M_Y_d_H_M "
"if --backup-path is not provided")
parser.add_argument(
"--no-mail", action="store_true", default=False,
help="Disable mail backup (save space)")
parser.add_argument(
"--restore", type=str, metavar="path",
help="Restore a previously backup up modoboa instance on a NEW machine. "
"You MUST provide backup directory"
),
parser.add_argument(
"--skip-checks", action="store_true", default=False,
help="Skip the checks the installer performs initially")
)
parser.add_argument("domain", type=str,
help="The main domain of your future mail server")
args = parser.parse_args(input_args)
@@ -205,12 +189,6 @@ def main(input_args):
utils.success("Welcome to Modoboa installer!\n")
# Checks
if not args.skip_checks:
utils.printcolor("Checking the installer...", utils.BLUE)
checks.handle()
utils.success("Checks complete\n")
is_config_file_available, outdate_config = utils.check_config_file(
args.configfile, args.interactive, args.upgrade, args.backup, is_restoring)
@@ -223,11 +201,11 @@ def main(input_args):
if is_config_file_available and outdate_config:
answer = utils.user_input("It seems that your config file is outdated. "
"Would you like to update it? (Y/n) ")
if not answer or answer.lower().startswith("y"):
if answer.lower().startswith("y"):
config_file_update_complete(utils.update_config(args.configfile))
if not args.stop_after_configfile_check:
answer = utils.user_input("Would you like to stop to review the updated config? (Y/n)")
if not answer or answer.lower().startswith("y"):
if answer.lower().startswith("y"):
return
else:
utils.error("You might encounter unexpected errors ! "
@@ -263,7 +241,7 @@ def main(input_args):
components = []
for section in config.sections():
if section in ["general", "database", "mysql", "postgres",
"certificate", "letsencrypt", "backup"]:
"certificate", "letsencrypt"]:
continue
if (config.has_option(section, "enabled") and
not config.getboolean(section, "enabled")):
@@ -298,19 +276,6 @@ def main(input_args):
"Restore complete! You can enjoy Modoboa at https://{} (same credentials as before)"
.format(config.get("general", "hostname"))
)
utils.success(
"\n"
"Modoboa is a free software maintained by volunteers.\n"
"You like the project and want it to be sustainable?\n"
"Then don't wait anymore and go sponsor it here:\n"
)
utils.printcolor(
"https://github.com/sponsors/modoboa\n",
utils.YELLOW
)
utils.success(
"Thank you for your help :-)\n"
)
if __name__ == "__main__":

View File

@@ -1 +0,0 @@
c1abbe97925917d4ec62ff11a70b375d40be5147