Using Rsync and SSH - Keys & Validating

This document covers using cron, ssh, and rsync to backup files over a local network or the Internet. Part of my goal is to ensure no user intervention is required when the computer is restarted (for passwords, keys, or key managers).

First of all you need the following packages installed:

• rsync 
• openssh 
• cron (or vixie-cron) 

Please note these instructions may be specific to Red Hat Linux versions 7.3, 9, and Fedora Core 3, but I hope they won't be too hard to adapt to almost any *NIX type OS. The man pages for 'ssh' and 'rsync' should be helpful to you if you need to change some things (use the "man ssh" and "man rsync" commands).

First, I'll define some variables. In my explanation, I will be synchronizing files (copying only new or changed files) one way, and I will be starting this process from the host I want to copy things to. In other words, I will be syncing files from /remote/dir/ on remotehost, as remoteuser, to /this/dir/ on thishost, as thisuser.

I want to make sure that 'rsync' over 'ssh' works at all before I begin to automate the process, so I test it first as thisuser:

$ rsync -avz -e ssh remoteuser@remotehost:/remote/dir /this/dir/ 

and type in remoteuser@remotehost's password when prompted. I do need to make sure that remoteuser has read permissions to /remote/dir/ on remotehost, and that thisuser has write permissions to /this/dir/ on thishost. Also, 'rsync' and 'ssh' should be in thisuser's path (use "which ssh" and "which rsync"), 'rsync' should be in remoteuser's path, and 'sshd' should be running on remotehost.

Configuring thishost

If that all worked out, or I eventually made it work, I am ready for the next step. I need to generate a private/public pair of keys to allow a 'ssh' connection without asking for a password. This may sound dangerous, and it is, but it is better than storing a user password (or key password) as clear text in the script. I can also put limitations on where connections made with this key can come from, and on what they can do when connected. Anyway, I generate the key I will use onthishost (as thisuser):

$ ssh-keygen -t dsa -b 1024 -f /home/thisuser/cron/thishost-rsync-key
Generating public/private dsa key pair.
Enter passphrase (empty for no passphrase): [press enter here]
Enter same passphrase again: [press enter here]
Your identification has been saved in /home/thisuser/cron/thishost-rsync-key.
Your public key has been saved in /home/thisuser/cron/thishost-rsync-key.pub.
The key fingerprint is:
3a:24:d6:a2:14:de:3b:aa:12:da:e5:7c:cd:01:aa:10 thisuser@thishost

and now we have a key with no password in the two files mentioned above. Make sure that no other unauthorized user can read the private key file (the one without the '.pub' extension).

This key serves no purpose until we put the public portion into the 'authorized_keys' file on remotehost, specifically the one for remoteuser:

/home/remoteuser/.ssh/authorized_keys 

I use scp to get the file over to remotehost:

$ scp /home/thisuser/cron/thishost-rsync-key.pub remoteuser@remotehost:/home/remoteuser/ 

and then I can prepare things on remotehost.

Configuring remotehost

I 'ssh' over to remotehost:

$ ssh remoteuser@remotehost

remoteuser@remotehost's password: [type correct password here]
$ echo I am now $USER at $HOSTNAME
I am now remoteuser at remotehost

to do some work.

I need to make sure I have the directory and files I need to authorize connections with this key:

$ if [ ! -d .ssh ]; then mkdir .ssh ; chmod 700 .ssh ; fi
$ mv thishost-rsync-key.pub .ssh/
$ cd .ssh/
$ if [ ! -f authorized_keys ]; then touch authorized_keys ; chmod 600 authorized_keys ; fi
$ cat thishost-rsync-key.pub >> authorized_keys 

Now the key can be used to make connections to this host.

Now that I have the key with no password in place and configured, I need to test it out before putting it in a cron job (which has its own small set of baggage). I exit from the ssh session to remotehost and try:

$ rsync -avz -e "ssh -i /home/thisuser/cron/thishost-rsync-key" remoteuser@remotehost:/remote/dir /this/dir/

If it asks for a password, I will check permissions on the private key file (onthishost, should be 600), on 'authorized_keys' and (on remotehost, should be 600), on the '~/.ssh/' directory (on both hosts, should be 700), and on the home directory ('~/') itself (on both hosts, should not be writeable by anyone but the user).

If things still aren't working out, some useful information may be found in log files. Log files usually found in the /var/log/ directory on most linux hosts, and in the/var/log/secure log file on Red Hat-ish linux hosts. The most useful logfiles in this instance will be found on remotehost, but localhost may provide some client side information in its logs. If you can't get to the logs, or are just impatient, you can tell the 'ssh' executable to provide some logging with the 'verbose' commands: '-v', '-vv', '-vvv'. The more v's, the more verbose the output. One is in the command above, but the one below should provide much more output:

$ rsync -avvvz -e "ssh -i /home/thisuser/cron/thishost-rsync-key" remoteuser@remotehost:/remote/dir /this/dir/ 

Hopefully, it will always just work flawlessly so I never have to extend the troubleshooting information listed here.