Simple Rsnapshot backup over FTPS

22 Aug 2013, 6 minute read

In this post I’ll walk you through setting up a simple backup solution using Rsnapshot and CurlFTPfs to retrieve files from an FTP/FTPS server. The result is a flexible backup solution with multiple retrieval points.

Note that while this example will work with FTP it is not recommended because the protocol is not encrypted.

Update 23/08/2013: There were a hundred other things I wanted to talk about in this post but decided to leave out for the sake of simplicity, but as @m0untainpenguin points out the feeling of safety that hard links give you is misleading. There is only ever one copy of the file on disk, so if that gets corrupted you’re out of luck. A great reason to do regular full backups, but that’s a walkthrough for another post!

I have tested this on a clean installation of Ubuntu 13.04, but let me know if anything doesn’t work as expected in the comments.

Many of the commands within this tutorial must be executed as root. You can always just add “sudo” in front of it if you have problems.

A simple FTP server for testing

While not technically part of the setup, I didn’t have ready access to an FTP server to test these notes. Installing an FTP server for testing is as simple as:

apt-get install vsftpd

Out of the box, vsftpd serves your home directory and can be connected to using your standard username and password.

Installing an FTP server with access to your home directory is a pretty bad idea without configuring it properly

Users and groups

We don’t want the backup to run as root because it’s always a good idea to restrict things as much as possible, so let’s create a backup user and group:

addgroup backup
adduser rsnapshot --ingroup backup --disabled-password

Mounting an FTP folder locally

CurlFTPfs presents the FTP server as a local directory using FUSE. It’s trivial to install it:

apt-get install curlftpfs 

We want to connect to our remote server using credentials, so let’s add that to the .netrc file in the home directory of the backup user:

cat << EOF > /home/rsnapshot/.netrc
machine ftp.host.com  
login myuser  
password mypass 
EOF

# Don't let anyone else see your FTP credentials
chmod 600 /home/rsnapshot/.netrc

Mounting the remote directory is also straightforward, here we’ll use /media/remote_server. Note that by default the directory is only accessible to the user who mounted it.

mkdir /media/remote_server
chown rsnapshot /media/remote_server

# Ubuntu doesn't have the right permissions on /etc/fuse.conf, so let's fix that
chmod 644 /etc/fuse.conf

Mounting and unmounting is easy:

# mount
curlftpfs ftps://ftp.host.com /media/remote_server -r -o ssl

# unmount
fusermount -u /media/remote_server

Now you have the remote folder mounted locally we can move on to the backup solution.

Rsnapshot configuration

Right, we’ll need to install Rsnapshot first.

apt-get install rsnapshot

We’re going to keep our backups in /var/rsnapshot for this example, but feel free to put them wherever you like:

mkdir -p /var/rsnapshot/private/snapshots
chmod 0700 /var/rsnapshot/private
chmod 0755 /var/rsnapshot/private/snapshots
chown -R rsnapshot /var/rsnapshot/

While we’re at it, let’s touch the log file and lock directory:

touch /var/log/rsnapshot.log
chown rsnapshot /var/log/rsnapshot.log
mkdir /var/run/rsnapshot/
chown rsnapshot /var/run/rsnapshot/

Next we need to set the configuration file. There are many options, so it’s a good idea to read the documentation, but this will get you up and running:

Now we can test the configuration (be sure to run as the rsnapshot user):

rsnapshot -t sync

We could run it without -t to carry out a backup, however that’ll fail right now because the FTP directory isn’t mounted (yet).

We want to mount the FTP folder every time we do a backup, so we’ll need a wrapper script to do that (/var/rsnapshot/doftpsync.sh):

#!/bin/sh
# -r = mount read-only
# -o ssl = Force SSL
echo Mounting the FTP servr
curlftpfs ftps://ftp.host.com /media/remote_server -r -o ssl

# Do some backup here :)
#echo performing the backup
/usr/bin/rsnapshot sync

# Finally, unmount and disconnect from the FTP server
fusermount -u /media/remote_server

Make the script executable:

chmod 755 /var/rsnapshot/doftpsync.sh

And finally run the script to test it. Check the contents of /var/rsnapshot/private/snapshots to check it worked as expected. # Running regularly with cron

As the backup user, let’s execute the Rsnapshot command at the relevant intervals using cron. To edit the user’s crontab just use:

su - rsnapshot
crontab -e

And add in the following commands (feel free to change the intervals to meet your needs of course):

# You only need to run 'sync' if the 'syncfirst' option is true
0 * * * *       /var/rsnapshot/doftpsync.sh

# Every hour on the 50th minute
50 * * * *       /usr/local/bin/rsnapshot hourly
# Once a day at 23:40
40 23 * * *       /usr/local/bin/rsnapshot daily
# Every Sunday at 23:30
30 23 * * 0       /usr/local/bin/rsnapshot weekly
# The first day of every month at 23:20
20 23 1 * *       /usr/local/bin/rsnapshot monthly

Mounting a read-only backup for recovery (optional)

Having a read-only copy of your backup is a great idea. Having one that even root can’t write to is even better. How much less stressful is it when you never have to think “damn, I copied that the wrong way and overwrote the backup”. Fortunately it’s easy to do with NFS (this is largely adapted and taken from the Rsnapshot HOWTO).

First we need to create a location to mount the read-only snapshots and modify the permissions so that no-one but the rsnapshot user can look inside the private directory:

mkdir /var/rsnapshot/readonly
chmod 0755 /var/rsnapshot/readonly

Then we add the NFS export of our private directory to /etc/exports:

/var/rsnapshot/private/snapshots/ 127.0.0.1(ro,no_root_squash)

You’ll need to restart the NFS service now:

service nfs restart

Now we have the NFS export, we can mount it. Add the following to /etc/fstab:

localhost:/var/rsnapshot/private/snapshots/ /var/rsnapshot/readonly/ nfs ro 0 0

Finally you can mount the directory:

mount /var/rsnapshot/readonly/

And of course, to test! Try creating a file inside the /var/rsnapshot/readonly/ directory and watch it fail.

Summary

You should now have a fully fledged enterprise-grade backup system making a backup of your hosted data, but there’s no time to rest! Encrypting your external drive is a great idea, just in case it gets stolen.

Of course you could always automate most of the above installation (and maintenance) by using Puppet and the Rsnapshot module.


Discuss!

Was this post useful? Why not help others find it by sharing on twitter. While you're there, get in touch and let me know what you think!

Some notes on backup I've moved

Hi, I'm Ryan

I care about code, building great teams and the community. I run Slate Horse but I'm still a full-stack developer. Previously co-organiser of JSOxford and OxRUG.


Get in touch