Backups in ISPmanager: Technical details

Data storage format

• Separate users are kept in a storage in separate files.
• Large archives can be divided into small volumes (the default value is 100MB) allowing to decrease required free disk space, and not to use the whole archive in case of partial data extraction.
• Storage format - .tgz. It allows extracting data using a third-party application.
• We use our own utility isptar, which creates a copy in the .tgz format and saves a list of files with displacements.

General architecture

A backup process includes a number of interconnected application.

backup2/backup2_pro

It creates a backup queue, if it is not present, extracts strings, one by one, and starts the backup2/backup2_pro process for each string. The process gets user backup settings, forms a draft of a file in the .system directory (real data will be saved into the files after starts), and starts the backup process with isptar.

isptar

An archiver, which reads data from a hard drive and packs them into .tgz archives.

backup2_cp

Manages storages (local, FTP, Dropbox, Amazon). It reads an archive generated by isptar, and uploads it to storage. backup2_cp is started from isptar and backup2/backup2_pro (to upload information about a backup copy file listing). It also checks free disk space in a storage.

backup2_system

Backup of such data as: user settings in a control panel, databases, mailboxes, domain names, web-domains, etc. backup2_system extracts data and saves it as special files in the .system directory. backup2/backup2_procreates a required number of files in that directory. Then, isptar starts the backup2_system process for each file.restore2/restore2_proExtracts data from the archive.

backup2_download

Forms a backup copy as a separate file for download. If a backup copy is saved into several volumes, they will be combined into one.

backup2_import

Allows to import an archive formed by backup2_download, and upload it to storage.

backup2_cgi

Allows to download a part of the archive as a separate .tgz file. E.g. it can be a dump of separate databases or a number of files and directories.

Free disk space control

ISPmanager Business can back up several users simultaneously. To check available disk space, the daemon backup2_cp --server (daemon) will start. This process gets information about occupied disk space from the .info files that are kept on MASTER, and checks that total data size in the storage has never exceeded the limit.

(daemon) does not process requests for space reservation itself, they are received by clients backup2_cp --client - processes that are started on every backup server and MASTER. A client will get a request from the backup2_cp --put process through UNIX socket and passed it through PIPE to (daemon).

If a local storage is used, the backup size limit is set for each server. Thus, if you have 2 backup servers, and the limit is set to 1 TB, your backup copies can occupy 1 TB (if all users will be backed up to one node), and 2 TB if backups are located on different nodes. All user backups are always created on one and the same node. If there is insufficient disk space on one node, old backups will be deleted on all nodes, even if some of them have enough disk space.

All user are allowed to store the same number of backups, and this number doesn't depend on a cluster where they are kept.

The number of backup copies may exceed the limit (the default value is 14: 7 daily differential and 7 full backups). First, a backup copy will be created, and then an old one will be deleted. The old copy can be deleted earlier if the storage has insufficient disk space. But the last backup copy cannot be deleted before a new one is created.

If there is insufficient space for a full backup, the backup process will fail.

Errors

Every time backup2/backup2_pro starts the user backup process, information about a backup copy is added into var/ispmgr.backup.cleanup. When starting and completing, backup2_cp --server (daemon) will look through the file, and will delete all files associated with those backups. If the backup process completes successfully, backup2/backup2_pro will delete the record from var/ispmgr.backup.cleanup that was created at the beginning.

Conversion of backup copies

To convert backup copies from dar, execute the /usr/local/mgr5/sbin/backup2_conv command. dar backups will be placed into the old directory located in the working directory specified in the panel's configuration file.

The default path to old backups after conversion is /usr/local/mgr5/var/backup/ispmgr/old

Old files are never deleted from the storage after conversion. You can delete them manually (before you start, check that the backup process completed successfully), or make sure that you have enough disk space.

During the conversion,  backup copies will be created next to user backup directories causing issues with quotas.