Results 1 to 15 of 15
  1. #1
    Join Date
    Oct 2012
    Location
    Europe and USA
    Posts
    991

    Arrow [GUIDE] Incremental rsync backup/sync over SSH without password

    Hello WHT members!

    In this guide I will explain how you can setup an automatic script and cron job for retrieving backups from your remote server (dedicated or VPS) to your local computer using the well-known utility rsync and SSH protocol, without typing the root password.

    With rsync you can get incremental backups of your data. This way, you retrieve only new or changed files since the previous backup.

    Normally, each time you run rsync over SSH to connect and get backups from a remote server, you need to type the root password. This prevents the automatic backups via cron jobs. With this guide I will explain how to create the required authorization keys, in order to login with them automatically.

    So let's start!

    What you need:
    1) A local computer (or virtual machine) running Linux (Debian/Ubuntu/CentOS preferred)
    It doesn't matter if the local Linux distribution is different from the Linux distribution on your server. Since both systems run Linux we can proceed.
    2) A hard disk formatted as ext3 or ext4 with enough space to backup your files and databases from your server. This disk should be mounted to your local Linux system via /etc/fstab

    Steps:
    All commands below should be run via SSH as root

    1) First verify that rsync is installed on your local Linux system. If it is running Debian or Ubuntu, type:
    Code:
    apt-get install rsync
    If it is running CentOS, type:
    Code:
    yum install rsync
    2) Now that we have rsync installed, we must create the SSH authentication keys, so that we can connect to the remote server via SSH without asking for password.

    On your local Linux system run:
    Code:
    ssh-keygen -t dsa
    when asked "Enter file in which to save the key"
    type carefully:
    Code:
    /root/.ssh/id_dsa_myserver1
    and press Enter. You will see a prompt: "Enter passphrase"
    Don't type anything, just press Enter

    The next question is "Enter same passphrase again:"
    Again don't type anything and just press Enter.

    Now we have created the authorization key and we need to send it to the remote server.

    3) In the following command replace 101.102.103.104 with the IP address of the remote server.
    On your local Linux system run:
    Code:
    scp /root/.ssh/id_dsa_myserver1.pub root@101.102.103.104:/root/
    you will be asked for the root password on your remote server, type it and press Enter.

    4) Connect to your remote server and type, as root, the following commands:
    Code:
    cat /root/id_dsa_myserver1.pub >> /root/.ssh/authorized_keys
    chmod 644 /root/.ssh/authorized_keys
    5) Back to your local Linux system, now test the passwordless SSH connection by using this command:
    Code:
    ssh -i /root/.ssh/id_dsa_myserver1 root@101.102.103.104
    (again replace 101.102.103.104 with the IP address of your server)

    You should be connected to your server without asking a root password!

    6) Now it's time to set a rsync job using this passwordless SSH connection. Back to your local Linux system, run this command:
    Code:
    rsync -uptorgvlHaz -e "ssh -i /root/.ssh/id_dsa_myserver1" root@101.102.103.104:/home /media/localbackupdisk/
    In the above example, replace:
    101.102.103.104 with the IP address of your remote server
    /media/localbackupdisk/ with the real path of the hard disk where you want to save the backups

    Now your local computer should connect to the remote server without asking for password and start retrieving a backup of the entire home directory, to your local disk. Depending on the size of your files on the server, this first backup will need some time to complete. But the next backup tasks will need much less time, as they will be retrieving only new or changed files.

    7) Set a cron job, running automatically the above command, every day at 5 am:
    Code:
    nano /var/spool/cron/root
    paste this command in a separate line:
    Code:
    0 5 * * * rsync -uptorgvlHaz -e "ssh -i /root/.ssh/id_dsa_myserver1" root@101.102.103.104:/home /media/localbackupdisk/ > /dev/null 2>&1
    Again, replace:
    101.102.103.104 with the IP address of your remote server
    /media/localbackupdisk/ with the real path of the hard disk where you want to save the backups

    Save the file by pressing:
    Ctrl+O
    Enter
    Ctrl+X

    If you can an error "nano: command not found", type:
    apt-get install nano
    or:
    yum install nano

    After saving the cron file, type this command to reload the cron service and read the new cron job:
    On CentOS
    Code:
    /etc/init.d/crond restart
    On Debian or Ubuntu:
    Code:
    /etc/init.d/cron restart
    That was all! That was a big post, but actually the process is easy. Feel free to ask any questions.
    Last edited by NetworkPanda; 10-29-2012 at 05:40 AM.
    NetworkPanda :: Web Hosting SSD Powered :: Reseller Hosting
    Instant activation, fast servers, NVMe SSD disks, cPanel, Softaculous 1-click apps installer, daily backups
    Multiple hosting locations: USA, Canada, France, UK, Germany, Italy, Spain, Poland, Finland

  2. #2
    Great! Thanks for sharing such a useful steps. It will help a lot!

  3. #3
    Join Date
    Oct 2012
    Location
    Europe and USA
    Posts
    991
    You're welcome!

    I have been using this method for more than 6 years to backup our servers and it works.
    I thought to share it after seeing lots of people writing posts that they dedicated/VPS is offline or it was shut down or crashed and they didn't have local backup of their sites.
    NetworkPanda :: Web Hosting SSD Powered :: Reseller Hosting
    Instant activation, fast servers, NVMe SSD disks, cPanel, Softaculous 1-click apps installer, daily backups
    Multiple hosting locations: USA, Canada, France, UK, Germany, Italy, Spain, Poland, Finland

  4. #4
    Nice tutorial
    thanks for share

  5. #5
    Join Date
    Jul 2011
    Location
    Sittingbourne, Kent, UK
    Posts
    197
    Nice tutorial, not to be a buzzkill the obvious but you really should try to avoid using passwordless keys where possible for obvious reasons.
    RackSRV Communications Limited
    UK specialists in Dedicated Servers & Server Colocation
    Company: 06856870 VAT: GB 934 7073 15 Tel: 0330 111 4444

  6. #6
    Join Date
    Mar 2009
    Location
    Miami, Florida
    Posts
    20,777
    Quote Originally Posted by Lee-RackSRV View Post
    Nice tutorial, not to be a buzzkill the obvious but you really should try to avoid using passwordless keys where possible for obvious reasons.
    Hello,
    You do have a point as if someone exploits the root account or gets their hands on the key, your backup server can easily become compromised. Of course, using the keyless method has advantages when it comes to automation.

    A few ways to make the solution a bit more secure would be;

    - Do not use the root user on the remote backup server, rather a less priviliaged user with a jailed shell access
    - Run the rsync on the backup server to connect to the production server and pull the data rather then push. RSync can work both ways.
    - The backup server should be tightly locked down with only ssh running. Ideally it should not be on a public network and accessible only to the production server and a management network
    Keith I Myers
    KMyers.me The rantings of a lunatic
    Join me on Technical.chat

  7. #7
    Join Date
    Feb 2013
    Location
    Canada
    Posts
    127
    Great tutorial and good information. Very easy to set up as well.

  8. #8
    Join Date
    Jul 2002
    Location
    EARTH
    Posts
    34
    Very nice post. Thank you so much!

    1) Really helped me and may be I too tired but I think in rsync command above, /media/localbackupdisk/ should come earlier than root@ip part to push the files to backup server.

    2) Also something that can help other silly people like me is don't miss the "ssh -i /root/.ssh/id_dsa_myserver1" part or you will keep getting password prompts.

    3) Never work when you get too tired or you will make silly mistakes like me I did today.

    4) 5 stars for such a great post. I must admit you are so good at teaching that you made a tough job really easy.

    Thanks again!

  9. #9
    Join Date
    Oct 2012
    Location
    Europe and USA
    Posts
    991
    Quote Originally Posted by smily View Post
    Very nice post. Thank you so much!

    1) Really helped me and may be I too tired but I think in rsync command above, /media/localbackupdisk/ should come earlier than root@ip part to push the files to backup server.

    2) Also something that can help other silly people like me is don't miss the "ssh -i /root/.ssh/id_dsa_myserver1" part or you will keep getting password prompts.

    3) Never work when you get too tired or you will make silly mistakes like me I did today.

    4) 5 stars for such a great post. I must admit you are so good at teaching that you made a tough job really easy.

    Thanks again!
    You are welcome smily, thank you for your remarks. Regarding the "ssh -i /root/.ssh/id_dsa_myserver1" part, yes, it is important because it calls the identification file which you created earlier. So you bypass the password prompts when calling this file.
    NetworkPanda :: Web Hosting SSD Powered :: Reseller Hosting
    Instant activation, fast servers, NVMe SSD disks, cPanel, Softaculous 1-click apps installer, daily backups
    Multiple hosting locations: USA, Canada, France, UK, Germany, Italy, Spain, Poland, Finland

  10. #10
    Join Date
    Jul 2002
    Location
    EARTH
    Posts
    34
    Thanks again NetworkPanda for such a useful article and I have one question:

    Does rsync command "rsync -uptorgvlHaz -e" also includes synchronization of both folders like --delete command does? I mean if something is deleted from backup folder, will it also be deleted from the remote folder when Rsync cron runs?

    Thanks!
    HitBiz.Net (HITBIZ INTERNATIONAL LLC, a registered US based business)
    Domain Names and reliable (raid protected and with daily offsite backups and many more options) shared web hosting, reseller accounts and dedicated servers.
    Safety & uptime of our customers' data is our main concern.

  11. #11
    Join Date
    Oct 2012
    Location
    Europe and USA
    Posts
    991
    Quote Originally Posted by smily View Post
    Thanks again NetworkPanda for such a useful article and I have one question:

    Does rsync command "rsync -uptorgvlHaz -e" also includes synchronization of both folders like --delete command does? I mean if something is deleted from backup folder, will it also be deleted from the remote folder when Rsync cron runs?

    Thanks!
    No, but if you want this to be done, you can add the --delete, --delete-excluded and --delete-after options:

    rsync -uptorgvlHaz --delete --delete-after --delete-excluded -e "ssh -i /root/.ssh/id_dsa_myserver1" etc.

    The --delete-after option tells rsync to start removing deleted files after the backup of new or modified files is complete. I think this is important, before if there are a lot of files/folders to delete, then the deletion may delay the backup process for several hours, unless you add the --delete-after option.
    NetworkPanda :: Web Hosting SSD Powered :: Reseller Hosting
    Instant activation, fast servers, NVMe SSD disks, cPanel, Softaculous 1-click apps installer, daily backups
    Multiple hosting locations: USA, Canada, France, UK, Germany, Italy, Spain, Poland, Finland

  12. #12
    Hello all.
    I setting up an incremental backup for one of my VPS with centos 5. The "destination" of the backup is another VPS with centos 5 too.

    While I lunch the following code on the "destination" VPS

    Code:
    cat /root/id_dsa_myserver1.pub >> /root/.ssh/authorized_keys
    I got this error

    Code:
    bash /root/.ssh/authorized_keys: No such file or directory
    What is missing on my backup server?
    Thanks

  13. #13
    Join Date
    Oct 2012
    Location
    Europe and USA
    Posts
    991
    Quote Originally Posted by HostingWM View Post
    Hello all.
    I setting up an incremental backup for one of my VPS with centos 5. The "destination" of the backup is another VPS with centos 5 too.

    While I lunch the following code on the "destination" VPS

    Code:
    cat /root/id_dsa_myserver1.pub >> /root/.ssh/authorized_keys
    I got this error

    Code:
    bash /root/.ssh/authorized_keys: No such file or directory
    What is missing on my backup server?
    Thanks
    You will need to create the .ssh directory

    Code:
    mkdir /root/.ssh
    NetworkPanda :: Web Hosting SSD Powered :: Reseller Hosting
    Instant activation, fast servers, NVMe SSD disks, cPanel, Softaculous 1-click apps installer, daily backups
    Multiple hosting locations: USA, Canada, France, UK, Germany, Italy, Spain, Poland, Finland

  14. #14
    Join Date
    Jul 2013
    Location
    United Kingdom
    Posts
    23
    thanks for the share!

  15. #15

    Backup home dir

    Hello,

    When I want to backup the user in the /home dir from the original server, what command do I need to execute?

    I tried 2 commands, 1 did what it had to do, but put the /home dir from the original server in a /home/home dir on the backup server

    The other command deleted all the linux files so I had to install linux again.

Similar Threads

  1. Replies: 0
    Last Post: 10-19-2011, 11:16 AM
  2. Cpanel Remote Sync incremental Backup Plugin
    By Syslint in forum Software & Scripts Offers
    Replies: 27
    Last Post: 10-22-2010, 03:51 PM
  3. Replies: 18
    Last Post: 08-26-2010, 10:19 AM
  4. CPanel Incremental Backup (rsync) & RAM
    By mainarea in forum Hosting Security and Technology
    Replies: 0
    Last Post: 03-01-2005, 03:26 PM
  5. ssh/rsync/cron: How to connect without password?
    By jojo235 in forum Hosting Security and Technology
    Replies: 2
    Last Post: 02-08-2004, 02:19 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •