Data backups with Duplicity
- Title: Data backups with Duplicity
- Author(s): Javier Alcazar
- Date: 27 Agosto 2014
- Version(s): 1.0
- Zentyal profiles: All
Duplicity incrementally backs up files and directory by encrypting tar-format volumes with GnuPG and uploading them to a remote (or local) file server. This is the general format of the command:
duplicity [full|incremental] [options] source_directory target_url
You can installing just typing:
# apt-get install duplicity
In the following examples, we decided to exclude the same directories that where configured as excluded directories by default in Zentyal, but we also included, as example, the /home/user folder. Feel free to reconfigure them based on your needs.
Contents |
File System Backups
Full copy example:
duplicity full --volsize 25 --exclude=/proc --exclude=/sys --exclude=/backups --exclude=/dev --exclude=/proc --exclude=/sys --exclude=/mnt --exclude=/media --exclude=/tmp --exclude=/var/spool --exclude=/var/cache --exclude=/var/tmp --include=/home/user --exclude=/ / file:///backups --no-encryption
/backups is excluded because is the designated destination folder
Incremental copy example:
duplicity incremental --volsize 25 --exclude=/proc --exclude=/sys --exclude=/backups --exclude=/dev --exclude=/proc --exclude=/sys --exclude=/mnt --exclude=/media --exclude=/tmp --exclude=/var/spool --exclude=/var/cache --exclude=/var/tmp --include=/home/user --exclude=/ / file:///backups --no-encryption
Checking the collection status:
duplicity collection-status file:///backups --no-encryption
Remove old backups, keeping last n:
duplicity remove-all-but-n-full 2 --force file:///backups --no-encryption
Restoring files from backup:
duplicity --no-encryption --file-to-restore home/user/documents/somefile.txt file:///backups /home/user/documents/somefile.txt
In this case, the original source and the destination have the same path, so we will be overwriting the file, but you may prefer to do something like:
duplicity --no-encryption --file-to-restore home/user/documents/somefile.txt file:///backups /restore/somefile.txt
Automating the backup
You can script the previous commands in order to easily manage them with cron. Just remember to add execution rights to the scripts (chmod u+x script_file) and include them to /etc/cron.d/backups (for daily use) like:
0 17 15 * * root nice -n 10 /root/script_full 0 17 1,2,3,4,5,6,7,8,9,10,11,12,13,14,16,17,18,19,21,20,22,23,24,25,26,27,28,29, 30,31 * * root nice -n 10 /root/script_incremental 0 23 15 * * root nice -n 10 /root/script_clean_old_backups
1 .---------------- minutes (00 - 59) 2 | .------------- hours (00 - 23) 3 | | .---------- day of month (01 - 31) 4 | | | .------- month (01 - 12) 5 | | | | .---- day of week (0 - 6 where 0 = Sunday) 6 | | | | |
* * * * * user command
This cron configuration will create an incremental backup each day at 17:00, but only full backup on 15th of each month. Also, it will clean all previous backups on 15th at 23:00.
Backups over Rsync
First of all, make sure that the user has permissions to connect using SSH, to achieve this, you will generate a SSH key and then, copy the id to the destination remote machine:
Change to superuser in the CLI
$ sudo su -
Then you generate the keys for the *root* user:
# ssh-keygen
Finally you copy the SSH id to the remote machine:
# ssh-copy-id user@remote.domain.com
Now you can use duplicity with Rsync:
duplicity full --volsize 25 --exclude=/proc --exclude=/sys --exclude=/dev --exclude=/proc --exclude=/sys --exclude=/mnt --exclude=/media --exclude=/tmp --exclude=/var/spool --exclude=/var/cache --exclude=/var/tmp --include=/home/user --exclude=/ / rsync://user@remote.domain.com//path/to/folder --no-encryption
Backup over SCP
Here is an example of how to use duplicity with scp :
duplicity full --volsize 25 --exclude=/proc --exclude=/sys --exclude=/dev --exclude=/proc --exclude=/sys --exclude=/mnt --exclude=/media --exclude=/tmp --exclude=/var/spool --exclude=/var/cache --exclude=/var/tmp --include=/home/user --exclude=/ / scp://user@remote.domain.com//path/to/folder --ssh-askpass --no-encryption
- Make sure that the destination folder has the proper rights in order to be managed by the configured user
- Remember to accept the fingerprint by manually launching the command in order to use scp as duplicity backup method:
The authenticity of host 'localhost' can't be established. SSH-RSA key fingerprint is 8a:f3:b0:3a:dd:c3:64:13:20:3a:1c:b6:54:80:18:0e. Are you sure you want to continue connecting (yes/no)? Please type 'yes' or 'no': yes Local and Remote metadata are synchronized, no sync needed. Last full backup date: none --------------[ Backup Statistics ]-------------- StartTime 1409052568.46 (Tue Aug 26 13:29:28 2014) EndTime 1409052571.85 (Tue Aug 26 13:29:31 2014) ElapsedTime 3.39 (3.39 seconds) SourceFiles 225 SourceFileSize 54620986 (52.1 MB) NewFiles 225 NewFileSize 54620986 (52.1 MB) DeletedFiles 0 ChangedFiles 0 ChangedFileSize 0 (0 bytes) ChangedDeltaSize 0 (0 bytes) DeltaEntries 225 RawDeltaSize 54280963 (51.8 MB) TotalDestinationSizeChange 23693038 (22.6 MB) Errors 0 -------------------------------------------------