mserver.log

I have found myself in the position that I would like to have some files I have on my server to have a backup made regularly. Luckily I have a friend which also has a server at home. I heard about rsync but found it insecure so I looked into tunneling it over ssh and I found a solution for it. Problem still was to regularly do a backup, so I needed some scheduling, cronjobs! Then another issue raised, what about the ssh session, I can’t put my password in the script so I needed ssh key-authentication. As you can see already it was quite a ride to get all things sorted out correctly. That is why I decided to share this.

Note: you don’t need root permissions to do this, maybe to install rsync if not installed yet.

Ok, where to begin? Well the most important thing is to have key-authentication working, which sometimes can be rather difficult.

For better reference we will now define a naming for the source and the destination servers:

- a = source

- b = destination

Let’s continue:

On a open a ssh session from host a to host b to allow your ssh to get familiar with the host key of host b. This will put the key in ~/.ssh/known_hosts. This will avoid your cronjob hanging later on.

On host a:

ssh-keygen -t rsa

If you are asked for a pass phrase, leave it empty. Use the default given directory as a place to store the key.

cd ~/.ssh/
 
ls -lh

matthias@a:~/.ssh$ ls -lh
total 16K
-rw------- 1 matthias matthias 1.7K 2007-08-26 19:38 id_rsa
-rw-r--r-- 1 matthias matthias  395 2007-08-26 19:38 id_rsa.pub

Now you’ll need to upload ‘id_rsa.pub’ to host b:

sftp matthias@b
sftp> cd .ssh
sftp> put id_rsa.pub
Uploading id_rsa to.pub /home/matthias/.ssh/id_rsa.pub
id_rsa.pub                                                    100% 1675     1.6KB/s   00:01
sftp> exit

You could delete the public key on host a now for security reasons, it’s no longer used on this host:

rm id_rsa.pub

On host b:

cd ~/.ssh/
cat id_rsa.pub >> authorized_keys

This will enable you to log in with the private key earlier created. You will also need the right permissions on your files, because of the ‘StrictModes yes’ set by default in any sshd configuration, you should not want to modify this value, instead assign the right permissions:

chmod 700 ~/.ssh
chmod 644 authorized_keys
chmod 644 id_rsa.pub

Talking about permissions, ‘StrictModes yes’ forces you to have a 755 mask on your home directory:

chmod 755 ~

Ok, test your key authentication by opening a ssh session from host a to be, as you will see you will not need to put your password! It may take longer to log in than normally. If this didn’t work, leave a comment.

Ok, so much for the key authentication, next thing to do is creating some directory structure to have one directory where you put synchronisable files in.

Example, on host a and b:

mkdir ~/sync

On host a:

mkdir ~/sync/scripts

Which will hold the script that will do the synchronisation. I like to put it there, but of course you could take it somewhere else. Choice is up to you!

Now open ~/sync/scripts/sync_script.sh with you favourite editor, it should contain something like:

#!/bin/bash
 
lockfile="/home/user/sync/scripts/sync.lock"
 
if test -e $lockfile; then
echo "another rsync is already in process, will schedule new files for next run !!"
else
touch $lockfile
 
rsync -avz --delete --exclude ".*" -e ssh /home/user/sync/ user@b:/home/user/sync
 
rm $lockfile
fi

replace ‘user’ with your username and make sure the lockfile can be created! As stated before this needs some tweaking.

Don’t forget to do:

chmod u+x ~/sync/scripts/sync_script.sh

So this is the synchronisation script! you could run it by issuing:

~/sync/scripts/sync_script.sh

You will probably see your sync script being rsync-ed together with your lock file depending on your configuration. Now you can simply put some files on host a in the ’sync’ directory and once the script is called it will synchronise your files!

One thing left to do, scheduling:

crontab -e

Example:

*/5 * * * * /home/matthias/sync/scripts/sync_script.sh > /dev/null 2>&1

Which will call the script every 5 minutes, which means my files get synchronised every 5 minutes. You might wonder about race-conditions, but due to the ‘lockfile’ these are prevented.

Well that’s about it, I find it to be useful. Of course you could tweak it some more, but it’s a nice start!

Tags: cron, Linux, rsync, ssh, synchronisation

This entry was posted on Tuesday, July 1st, 2008 at 10:21 pm and is filed under General Linux. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.