ISPtar Backup Module

Data storage format

• Separate users are kept in a storage in separate files.
• Large archives can be divided into small volumes (the default value is 100MB) allowing to decrease required free disk space, and not to use the whole archive in case of partial data extraction.
• Storage format - .tgz. It allows extracting data using a third-party application.
• We use our own utility isptar, which creates a copy in the .tgz format and saves a list of files with displacements.

General architecture

A backup process includes a number of interconnected application.

backup2/backup2_pro

It creates a backup queue, if it is not present, extracts strings, one by one, and starts the backup2/backup2_pro process for each string. The process gets user backup settings, forms a draft of a file in the .system directory (real data will be saved into the files after starts), and starts the backup process with isptar.

isptar

An archiver, which reads data from a hard drive and packs them into .tgz archives.

backup2_cp

Manages storages (local, FTP, Dropbox, Amazon). It reads an archive generated by isptar, and uploads it to storage. backup2_cp is started from isptar and backup2/backup2_pro (to upload information about a backup copy file listing). It also checks free disk space in a storage.

backup2_system

Backup of such data as: user settings in a control panel, databases, mailboxes, domain names, web-domains, etc. backup2_system extracts data and saves it as special files in the .system directory. backup2/backup2_procreates a required number of files in that directory. Then, isptar starts the backup2_system process for each file.restore2/restore2_proExtracts data from the archive.

backup2_download

Forms a backup copy as a separate file for download. If a backup copy is saved into several volumes, they will be combined into one.

backup2_import

Allows to import an archive formed by backup2_download, and upload it to storage.

backup2_cgi

Allows to download a part of the archive as a separate .tgz file. E.g. it can be a dump of separate databases or a number of files and directories.

Free disk space control

All user are allowed to store the same number of backups, and this number doesn't depend on a cluster where they are kept.

The number of backup copies may exceed the limit (the default value is 14: 7 daily differential and 7 full backups). First, a backup copy will be created, and then an old one will be deleted. The old copy can be deleted earlier if the storage has insufficient disk space. But the last backup copy cannot be deleted before a new one is created.

If there is insufficient space for a full backup, the backup process will fail.

Errors

Every time backup2/backup2_pro starts the user backup process, information about a backup copy is added into var/ispmgr.backup.cleanup. When starting and completing, backup2_cp --server (daemon) will look through the file, and will delete all files associated with those backups. If the backup process completes successfully, backup2/backup2_pro will delete the record from var/ispmgr.backup.cleanup that was created at the beginning.

Conversion of backup copies

To convert backup copies from dar, execute the /usr/local/mgr5/sbin/backup2_conv command. dar backups will be placed into the old directory located in the working directory specified in the panel's configuration file.

The default path to old backups after conversion is /usr/local/mgr5/var/backup/ispmgr/old

Old files are never deleted from the storage after conversion. You can delete them manually (before you start, check that the backup process completed successfully), or make sure that you have enough disk space.

During the conversion,  backup copies will be created next to user backup directories causing issues with quotas.