evgeny/modoboa-installer
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Update from master

Browse Source 
commit 5c22600d98
Merge: bc12ca7 bcdbb4a
Author: Antoine Nguyen <tonio@ngyn.org>
Date:   Tue Nov 29 16:54:28 2022 +0100

    Merge pull request #462 from Spitfireap/randomize-api-call-time

    randomize api call time

commit bcdbb4a2ce
Author: Spitap <dev@asdrip.fr>
Date:   Tue Nov 29 14:53:05 2022 +0100

    fix typo

commit bd1ddcef21
Author: Spitap <dev@asdrip.fr>
Date:   Tue Nov 29 13:45:31 2022 +0100

    randomize api call time

commit bc12ca7327
Merge: d364239 bd0ecd0
Author: Antoine Nguyen <tonio@ngyn.org>
Date:   Mon Nov 14 15:49:41 2022 +0100

    Merge pull request #458 from Spitfireap/fix-include_try

    fix typo in dovecot configuration file

commit bd0ecd0949
Author: Spitap <dev@asdrip.fr>
Date:   Thu Nov 10 14:57:43 2022 +0100

    fix typo in dovecot configuration file

commit d364239348
Merge: 61838db 3763300
Author: Antoine Nguyen <tonio@ngyn.org>
Date:   Wed Nov 9 10:51:30 2022 +0100

    Merge pull request #456 from modoboa/feature/improved_backup_restore

    WIP: Improved backup/restore system.

commit 37633008cb
Author: Antoine Nguyen <tonio@ngyn.org>
Date:   Wed Nov 9 10:30:44 2022 +0100

    Fixed restore mode

commit d6f9a5b913
Author: Antoine Nguyen <tonio@ngyn.org>
Date:   Tue Nov 8 17:20:25 2022 +0100

    Few fixes.

commit 8b1d60ee59
Author: Antoine Nguyen <tonio@ngyn.org>
Date:   Tue Nov 8 17:19:23 2022 +0100

    Few fixes

commit 2b5edae5d5
Author: Antoine Nguyen <tonio@ngyn.org>
Date:   Sun Nov 6 10:30:24 2022 +0100

    WIP: Improved backup/restore system.

commit 61838dbe4d
Author: Antoine Nguyen <tonio@ngyn.org>
Date:   Sat Nov 5 09:30:50 2022 +0100

    Check if restore is defined before doing anything else.

    fix #453

commit 962cac3ad9
Merge: 1b192c5 ef2359a
Author: Antoine Nguyen <tonio@ngyn.org>
Date:   Fri Nov 4 09:41:20 2022 +0100

    Merge pull request #450 from Spitfireap/fixed-super-call

    fixed super call in modoboa's script

commit ef2359a2a8
Author: Spitap <dev@asdrip.fr>
Date:   Thu Nov 3 23:10:21 2022 +0100

    fixed super call

commit 1b192c5fd5
Merge: 754d652 b0b0146
Author: Antoine Nguyen <tonio@ngyn.org>
Date:   Thu Nov 3 15:34:48 2022 +0100

    Merge pull request #449 from Spitfireap/fixed-import-typo

    fixed constants import

commit b0b01465d9
Author: Spitap <dev@asdrip.fr>
Date:   Thu Nov 3 15:00:07 2022 +0100

    fixed constants import

commit 754d652fc2
Author: Antoine Nguyen <tonio@ngyn.org>
Date:   Thu Nov 3 12:27:04 2022 +0100

    Few fixes

commit cb5fa75693
Merge: 1afb8e6 e01265a
Author: Antoine Nguyen <tonio@ngyn.org>
Date:   Thu Nov 3 12:20:25 2022 +0100

    Merge pull request #444 from Spitfireap/tighter-config-file-perm

    tighter config file permission

commit 1afb8e61fc
Merge: 15c1779 8dd0b7d
Author: Antoine Nguyen <tonio@ngyn.org>
Date:   Thu Nov 3 12:17:16 2022 +0100

    Merge pull request #424 from Spitfireap/restore

    Backup & restore system

commit 8dd0b7d497
Author: Spitap <dev@asdrip.fr>
Date:   Thu Nov 3 10:57:03 2022 +0100

    Last camelCase

commit 554611b366
Author: Spitap <dev@asdrip.fr>
Date:   Thu Nov 3 10:54:06 2022 +0100

    review fix

commit 15c17796f2
Merge: ce8e7e6 84d1363
Author: Antoine Nguyen <tonio@ngyn.org>
Date:   Fri Oct 28 09:43:30 2022 +0200

    Merge pull request #446 from Spitfireap/fix-ssl-min-protocol

    fixed ssl_min_protocol setting

commit 84d13633a1
Author: Spitap <dev@asdrip.fr>
Date:   Thu Oct 27 22:37:47 2022 +0200

    fixed ssl_min_protocol setting

commit ce8e7e6027
Merge: 8e8ae5f fe7df27
Author: Antoine Nguyen <tonio@ngyn.org>
Date:   Thu Oct 27 17:56:37 2022 +0200

    Merge pull request #445 from Spitfireap/dovecot-fixes

    Fixes ssl permission error, updated ssl_protocol parameter

commit e01265a4ee
Merge: a5fba03 235ef3b
Author: Spitap <dev@asdrip.fr>
Date:   Thu Oct 27 17:44:37 2022 +0200

    Merge branch 'tighter-config-file-perm' of https://github.com/Spitfireap/modoboa-installer into tighter-config-file-perm

commit a5fba03264
Author: Spitap <dev@asdrip.fr>
Date:   Thu Oct 27 11:13:47 2022 +0200

    tighter config file permission

commit fe7df276fc
Author: Spitap <dev@asdrip.fr>
Date:   Thu Oct 27 17:25:39 2022 +0200

    Check dovecot version greater

commit 8f34f0af6f
Author: Spitap <dev@asdrip.fr>
Date:   Thu Oct 27 17:00:58 2022 +0200

    Fixes ssl permission error, updated ssl_protocol parameter

commit 8e8ae5fb9c
Merge: 67f6cee fefbf54
Author: Antoine Nguyen <tonio@ngyn.org>
Date:   Thu Oct 27 16:49:20 2022 +0200

    Merge pull request #439 from stefaweb/master

    Update config_dict_template.py for default max_servers value

commit 235ef3befb
Author: Spitap <dev@asdrip.fr>
Date:   Thu Oct 27 11:13:47 2022 +0200

    thighter config file permission

commit 67f6cee8ea
Merge: b84abbb 53f7f8e
Author: Antoine Nguyen <tonio@ngyn.org>
Date:   Tue Oct 25 19:32:37 2022 +0200

    Merge pull request #442 from Spitfireap/patch-1

    Set $max_server to 2 to avoid amavis crash

commit 5c9d5c9a03
Author: Spitap <dev@asdrip.fr>
Date:   Tue Oct 25 16:58:57 2022 +0200

    DKIM keys restore, Radicale backup/restore, fixes

commit 4c1f8710b5
Author: Spitap <dev@asdrip.fr>
Date:   Tue Oct 25 16:04:55 2022 +0200

    Added dkim key backup

commit e34eb4b337
Author: Spitap <dev@asdrip.fr>
Date:   Tue Oct 25 13:59:28 2022 +0200

    fix database path

commit 53f7f8ef9d
Author: Spitfireap <45575529+Spitfireap@users.noreply.github.com>
Date:   Wed Oct 19 08:19:40 2022 +0000

    Update config_dict_template.py

commit 35778cd614
Merge: 6726f5b b84abbb
Author: Spitfireap <45575529+Spitfireap@users.noreply.github.com>
Date:   Tue Oct 18 17:17:48 2022 +0200

    Merge branch 'modoboa:master' into restore

commit fefbf549a4
Author: Stephane Leclerc <sleclerc@actionweb.fr>
Date:   Thu Oct 6 13:36:13 2022 +0200

    Update config_dict_template.py for default max_server value

commit 6726f5b1a2
Author: Spitap <dev@asdrip.fr>
Date:   Mon Sep 26 13:39:28 2022 +0200

    Improved path generation, path mistake proofing

commit a192cbcbd0
Author: Spitap <dev@asdrip.fr>
Date:   Mon Sep 19 16:40:25 2022 +0200

    Updated doc, default path on conf file

commit 5bed9655ea
Author: Spitap <dev@asdrip.fr>
Date:   Mon Sep 19 15:53:19 2022 +0200

    fixed typo

commit 6b096a7470
Author: Spitap <dev@asdrip.fr>
Date:   Mon Sep 19 15:50:03 2022 +0200

    Simplified db dumps restore

commit e30add03fd
Merge: d75d83f 1f8dd1b
Author: Spitap <dev@asdrip.fr>
Date:   Mon Sep 19 15:39:05 2022 +0200

    Update from master

commit d75d83f202
Author: Spitap <dev@asdrip.fr>
Date:   Mon Sep 19 15:13:44 2022 +0200

    more refactoring

commit f3811b4b39
Author: Spitap <dev@asdrip.fr>
Date:   Mon Sep 19 14:59:43 2022 +0200

    refactoring

commit b0d56b3989
Author: Spitap <dev@asdrip.fr>
Date:   Thu Sep 15 11:32:57 2022 +0200

    PEP formating

commit 53e3e3ec58
Author: Spitap <dev@asdrip.fr>
Date:   Fri Aug 5 15:20:11 2022 +0200

    Better UX, use of os to concatenate path

commit e546d2cb23
Author: Spitap <dev@asdrip.fr>
Date:   Wed Jul 27 16:32:59 2022 +0200

    Better UX

commit 70faa1c5cb
Author: Spitap <dev@asdrip.fr>
Date:   Wed Jul 27 15:58:41 2022 +0200

    Fixed backupdir index

commit 563979a7dd
Author: Spitap <dev@asdrip.fr>
Date:   Wed Jul 27 15:51:22 2022 +0200

    fixed mail backup/restore

commit ee2ccf0647
Author: Spitap <dev@asdrip.fr>
Date:   Wed Jul 27 14:35:48 2022 +0200

    Fixed postfix install, added restore to readme

commit 2077c94b52
Author: Spitap <dev@asdrip.fr>
Date:   Tue Jul 26 17:05:00 2022 +0200

    Fix amavis config file not copied to right location

commit 4a7222bd24
Author: Spitap <dev@asdrip.fr>
Date:   Tue Jul 26 16:53:24 2022 +0200

    Fixed nginx call to uwsgi

commit e7b6104195
Author: Spitap <dev@asdrip.fr>
Date:   Tue Jul 26 16:39:41 2022 +0200

    fixed install within class

commit 4a00590354
Author: Spitap <dev@asdrip.fr>
Date:   Tue Jul 26 16:20:03 2022 +0200

    fixed restore disclamer

commit 15768c429e
Author: Spitap <dev@asdrip.fr>
Date:   Tue Jul 26 12:07:42 2022 +0200

    Restore workflow done

commit 439ffb94c4
Author: Spitap <dev@asdrip.fr>
Date:   Mon Jul 25 18:54:47 2022 +0200

    initial commit

commit 37bc21dfd3
Author: Spitap <dev@asdrip.fr>
Date:   Tue Jul 26 10:36:08 2022 +0200

    Backup postewhite.conf instead of custom whitelist

    Postwhite.conf contains a custom host list

commit 26204143af
Merge: 2097055 d495afd
Author: Spitap <dev@asdrip.fr>
Date:   Mon Jul 25 22:10:26 2022 +0200

    Merge branch 'master' into backup

commit 20970557de
Author: Spitap <dev@asdrip.fr>
Date:   Mon Jul 25 22:05:35 2022 +0200

    Allow to disable mail backup

commit 632c26596e
Author: Spitap <dev@asdrip.fr>
Date:   Mon Jul 25 21:52:15 2022 +0200

    Update backup readme

commit 9e1c18cd6b
Author: Spitap <dev@asdrip.fr>
Date:   Thu Jul 21 19:09:53 2022 +0200

    Fix argument passed as list instead of string

commit db6457c5f5
Author: Spitap <dev@asdrip.fr>
Date:   Thu Jul 21 19:07:18 2022 +0200

    better path handling

commit 579faccfa5
Author: Spitap <dev@asdrip.fr>
Date:   Thu Jul 21 19:00:32 2022 +0200

    added an automatic bash option (no path provided) or a path provided bash (for cron job)

commit 5318fa279b
Author: Spitap <dev@asdrip.fr>
Date:   Thu Jul 21 18:00:50 2022 +0200

    bash option

commit 74de6a9bb1
Author: Spitap <dev@asdrip.fr>
Date:   Thu Jul 21 17:31:56 2022 +0200

    Reset pgpass before trying to backup secondary dbs

commit 54185a7c5a
Author: Spitap <dev@asdrip.fr>
Date:   Thu Jul 21 17:26:40 2022 +0200

    Fix database backup logic issue

commit 1f9d69c37c
Author: Spitap <dev@asdrip.fr>
Date:   Thu Jul 21 17:21:59 2022 +0200

    Fix copy issue

commit 8d02d2a9fb
Author: Spitap <dev@asdrip.fr>
Date:   Thu Jul 21 17:09:23 2022 +0200

    added safe mkdir in utils, use utils.mkdir_safe() in backup

commit 6f604a5fec
Author: Spitap <dev@asdrip.fr>
Date:   Thu Jul 21 16:53:56 2022 +0200

    Fix loop logic

commit 568c4a65a0
Author: Spitap <dev@asdrip.fr>
Date:   Thu Jul 21 16:51:32 2022 +0200

    fix none-type passed to os.path

commit dc84a79528
Author: Spitap <dev@asdrip.fr>
Date:   Thu Jul 21 14:12:35 2022 +0200

    Note : capitalize affects only first letter

commit 304e25fa3c
Author: Spitap <dev@asdrip.fr>
Date:   Thu Jul 21 14:10:57 2022 +0200

    Fix getattr

commit 070efd61c4
Author: Spitap <dev@asdrip.fr>
Date:   Thu Jul 21 14:08:39 2022 +0200

    Fix import

commit 9917d8023e
Author: Spitap <dev@asdrip.fr>
Date:   Thu Jul 21 14:02:41 2022 +0200

    Edited README, fix backup run process

commit 27b9de6755
Author: Spitap <dev@asdrip.fr>
Date:   Thu Jul 21 13:48:44 2022 +0200

    database backup

commit 56ed214fb5
Author: Spitap <dev@asdrip.fr>
Date:   Tue Jul 19 19:06:53 2022 +0200

    Starting work on backup system
This commit is contained in:
Spitap
2022-12-22 18:47:23 +01:00
parent d9ced63e99
commit 6b4302b566
23 changed files with 863 additions and 74 deletions
Download Patch File Download Diff File Expand all files Collapse all files

231
modoboa_installer/scripts/backup.py Normal file
View File

@@ -0,0 +1,231 @@
"""Backup script for pre-installed instance."""
import os
import pwd
import shutil
import stat
import sys
import datetime
from .. import database
from .. import utils
from ..constants import DEFAULT_BACKUP_DIRECTORY
class Backup:
"""
Backup structure ( {optional} ):
{{backup_directory}}
||
||--> installer.cfg
||--> custom
|--> { (copy of) /etc/amavis/conf.d/99-custom }
|--> { (copy of) /etc/postfix/custom_whitelist.cidr }
|--> { (copy of) dkim directory }
|--> {dkim.pem}...
|--> { (copy of) radicale home_dir }
||--> databases
|--> modoboa.sql
|--> { amavis.sql }
|--> { spamassassin.sql }
||--> mails
|--> vmails
"""
def __init__(self, config, silent_backup, backup_path, nomail):
self.config = config
self.backup_path = backup_path
self.nomail = nomail
self.silent_backup = silent_backup
def validate_path(self, path):
"""Check basic condition for backup directory."""
path_exists = os.path.exists(path)
if path_exists and os.path.isfile(path):
utils.printcolor(
"Error, you provided a file instead of a directory!", utils.RED)
return False
if not path_exists:
if not self.silent_backup:
create_dir = input(
f"\"{path}\" doesn't exist, would you like to create it? [Y/n]\n").lower()
if self.silent_backup or (not self.silent_backup and create_dir.startswith("y")):
pw = pwd.getpwnam("root")
utils.mkdir_safe(path, stat.S_IRWXU |
stat.S_IRWXG, pw[2], pw[3])
else:
utils.printcolor(
"Error, backup directory not present.", utils.RED
)
return False
if len(os.listdir(path)) != 0:
if not self.silent_backup:
delete_dir = input(
"Warning: backup directory is not empty, it will be purged if you continue... [Y/n]\n").lower()
if self.silent_backup or (not self.silent_backup and delete_dir.startswith("y")):
try:
os.remove(os.path.join(path, "installer.cfg"))
except FileNotFoundError:
pass
shutil.rmtree(os.path.join(path, "custom"),
ignore_errors=False)
shutil.rmtree(os.path.join(path, "mails"), ignore_errors=False)
shutil.rmtree(os.path.join(path, "databases"),
ignore_errors=False)
else:
utils.printcolor(
"Error: backup directory not clean.", utils.RED
)
return False
self.backup_path = path
pw = pwd.getpwnam("root")
for dir in ["custom/", "databases/"]:
utils.mkdir_safe(os.path.join(self.backup_path, dir),
stat.S_IRWXU | stat.S_IRWXG, pw[2], pw[3])
return True
def set_path(self):
"""Setup backup directory."""
if self.silent_backup:
if self.backup_path is None:
if self.config.has_option("backup", "default_path"):
path = self.config.get("backup", "default_path")
else:
path = DEFAULT_BACKUP_DIRECTORY
date = datetime.datetime.now().strftime("%m_%d_%Y_%H_%M")
path = os.path.join(path, f"backup_{date}")
self.validate_path(path)
else:
if not self.validate_path(self.backup_path):
utils.printcolor(
f"Path provided: {self.backup_path}", utils.BLUE)
sys.exit(1)
else:
user_value = None
while user_value == "" or user_value is None or not self.validate_path(user_value):
utils.printcolor(
"Enter backup path (it must be an empty directory)", utils.MAGENTA)
utils.printcolor("CTRL+C to cancel", utils.MAGENTA)
user_value = utils.user_input("-> ")
def config_file_backup(self):
utils.copy_file("installer.cfg", self.backup_path)
def mail_backup(self):
if self.nomail:
utils.printcolor(
"Skipping mail backup, no-mail argument provided", utils.MAGENTA)
return
utils.printcolor("Backing up mails", utils.MAGENTA)
home_path = self.config.get("dovecot", "home_dir")
if not os.path.exists(home_path) or os.path.isfile(home_path):
utils.printcolor("Error backing up Email, provided path "
f" ({home_path}) seems not right...", utils.RED)
else:
dst = os.path.join(self.backup_path, "mails/")
if os.path.exists(dst):
shutil.rmtree(dst)
shutil.copytree(home_path, dst)
utils.printcolor("Mail backup complete!", utils.GREEN)
def custom_config_backup(self):
"""
Custom config :
- DKIM keys: {{keys_storage_dir}}
- Radicale collection (calendat, contacts): {{home_dir}}
- Amavis : /etc/amavis/conf.d/99-custom
- Postwhite : /etc/postwhite.conf
Feel free to suggest to add others!
"""
utils.printcolor(
"Backing up some custom configuration...", utils.MAGENTA)
custom_path = os.path.join(
self.backup_path, "custom")
# DKIM Key
if (self.config.has_option("opendkim", "enabled") and
self.config.getboolean("opendkim", "enabled")):
dkim_keys = self.config.get(
"opendkim", "keys_storage_dir", fallback="/var/lib/dkim")
if os.path.isdir(dkim_keys):
shutil.copytree(dkim_keys, os.path.join(custom_path, "dkim"))
utils.printcolor(
"DKIM keys saved!", utils.GREEN)
# Radicale Collections
if (self.config.has_option("radicale", "enabled") and
self.config.getboolean("radicale", "enabled")):
radicale_backup = os.path.join(self.config.get(
"radicale", "home_dir", fallback="/srv/radicale"), "collections")
if os.path.isdir(radicale_backup):
shutil.copytree(radicale_backup, os.path.join(
custom_path, "radicale"))
utils.printcolor("Radicale files saved", utils.GREEN)
# AMAVIS
if (self.config.has_option("amavis", "enabled") and
self.config.getboolean("amavis", "enabled")):
amavis_custom = "/etc/amavis/conf.d/99-custom"
if os.path.isfile(amavis_custom):
utils.copy_file(amavis_custom, custom_path)
utils.printcolor(
"Amavis custom configuration saved!", utils.GREEN)
# POSTWHITE
if (self.config.has_option("postwhite", "enabled") and
self.config.getboolean("postwhite", "enabled")):
postswhite_custom = "/etc/postwhite.conf"
if os.path.isfile(postswhite_custom):
utils.copy_file(postswhite_custom, custom_path)
utils.printcolor(
"Postwhite configuration saved!", utils.GREEN)
def database_backup(self):
"""Backing up databases"""
utils.printcolor("Backing up databases...", utils.MAGENTA)
self.database_dump("modoboa")
self.database_dump("amavis")
self.database_dump("spamassassin")
def database_dump(self, app_name):
dump_path = os.path.join(self.backup_path, "databases")
backend = database.get_backend(self.config)
if app_name == "modoboa" or (self.config.has_option(app_name, "enabled") and
self.config.getboolean(app_name, "enabled")):
dbname = self.config.get(app_name, "dbname")
dbuser = self.config.get(app_name, "dbuser")
dbpasswd = self.config.get(app_name, "dbpassword")
backend.dump_database(dbname, dbuser, dbpasswd,
os.path.join(dump_path, f"{app_name}.sql"))
def backup_completed(self):
utils.printcolor("Backup process done, your backup is available here:"
f"--> {self.backup_path}", utils.GREEN)
def run(self):
self.set_path()
self.config_file_backup()
self.mail_backup()
self.custom_config_backup()
self.database_backup()
self.backup_completed()