Results 1 to 15 of 15
-
10-29-2012, 05:34 AM #1Web Hosting Master
- Join Date
- Oct 2012
- Location
- Europe and USA
- Posts
- 991
[GUIDE] Incremental rsync backup/sync over SSH without password
Hello WHT members!
In this guide I will explain how you can setup an automatic script and cron job for retrieving backups from your remote server (dedicated or VPS) to your local computer using the well-known utility rsync and SSH protocol, without typing the root password.
With rsync you can get incremental backups of your data. This way, you retrieve only new or changed files since the previous backup.
Normally, each time you run rsync over SSH to connect and get backups from a remote server, you need to type the root password. This prevents the automatic backups via cron jobs. With this guide I will explain how to create the required authorization keys, in order to login with them automatically.
So let's start!
What you need:
1) A local computer (or virtual machine) running Linux (Debian/Ubuntu/CentOS preferred)
It doesn't matter if the local Linux distribution is different from the Linux distribution on your server. Since both systems run Linux we can proceed.
2) A hard disk formatted as ext3 or ext4 with enough space to backup your files and databases from your server. This disk should be mounted to your local Linux system via /etc/fstab
Steps:
All commands below should be run via SSH as root
1) First verify that rsync is installed on your local Linux system. If it is running Debian or Ubuntu, type:
Code:apt-get install rsync
Code:yum install rsync
On your local Linux system run:
Code:ssh-keygen -t dsa
type carefully:
Code:/root/.ssh/id_dsa_myserver1
Don't type anything, just press Enter
The next question is "Enter same passphrase again:"
Again don't type anything and just press Enter.
Now we have created the authorization key and we need to send it to the remote server.
3) In the following command replace 101.102.103.104 with the IP address of the remote server.
On your local Linux system run:
Code:scp /root/.ssh/id_dsa_myserver1.pub root@101.102.103.104:/root/
4) Connect to your remote server and type, as root, the following commands:
Code:cat /root/id_dsa_myserver1.pub >> /root/.ssh/authorized_keys chmod 644 /root/.ssh/authorized_keys
Code:ssh -i /root/.ssh/id_dsa_myserver1 root@101.102.103.104
You should be connected to your server without asking a root password!
6) Now it's time to set a rsync job using this passwordless SSH connection. Back to your local Linux system, run this command:
Code:rsync -uptorgvlHaz -e "ssh -i /root/.ssh/id_dsa_myserver1" root@101.102.103.104:/home /media/localbackupdisk/
101.102.103.104 with the IP address of your remote server
/media/localbackupdisk/ with the real path of the hard disk where you want to save the backups
Now your local computer should connect to the remote server without asking for password and start retrieving a backup of the entire home directory, to your local disk. Depending on the size of your files on the server, this first backup will need some time to complete. But the next backup tasks will need much less time, as they will be retrieving only new or changed files.
7) Set a cron job, running automatically the above command, every day at 5 am:
Code:nano /var/spool/cron/root
Code:0 5 * * * rsync -uptorgvlHaz -e "ssh -i /root/.ssh/id_dsa_myserver1" root@101.102.103.104:/home /media/localbackupdisk/ > /dev/null 2>&1
101.102.103.104 with the IP address of your remote server
/media/localbackupdisk/ with the real path of the hard disk where you want to save the backups
Save the file by pressing:
Ctrl+O
Enter
Ctrl+X
If you can an error "nano: command not found", type:
apt-get install nano
or:
yum install nano
After saving the cron file, type this command to reload the cron service and read the new cron job:
On CentOS
Code:/etc/init.d/crond restart
Code:/etc/init.d/cron restart
Last edited by NetworkPanda; 10-29-2012 at 05:40 AM.
★ NetworkPanda :: Web Hosting SSD Powered :: Reseller Hosting
★ Instant activation, fast servers, NVMe SSD disks, cPanel, Softaculous 1-click apps installer, daily backups
★ Multiple hosting locations: USA, Canada, France, UK, Germany, Italy, Spain, Poland, Finland
-
10-29-2012, 09:11 AM #2Disabled
- Join Date
- Aug 2012
- Posts
- 3,105
Great! Thanks for sharing such a useful steps. It will help a lot!
-
10-29-2012, 09:56 AM #3Web Hosting Master
- Join Date
- Oct 2012
- Location
- Europe and USA
- Posts
- 991
You're welcome!
I have been using this method for more than 6 years to backup our servers and it works.
I thought to share it after seeing lots of people writing posts that they dedicated/VPS is offline or it was shut down or crashed and they didn't have local backup of their sites.★ NetworkPanda :: Web Hosting SSD Powered :: Reseller Hosting
★ Instant activation, fast servers, NVMe SSD disks, cPanel, Softaculous 1-click apps installer, daily backups
★ Multiple hosting locations: USA, Canada, France, UK, Germany, Italy, Spain, Poland, Finland
-
01-10-2013, 06:05 PM #4Disabled
- Join Date
- Oct 2012
- Posts
- 10
Nice tutorial
thanks for share
-
01-31-2013, 01:19 PM #5Junior Guru
- Join Date
- Jul 2011
- Location
- Sittingbourne, Kent, UK
- Posts
- 197
Nice tutorial, not to be a buzzkill the obvious but you really should try to avoid using passwordless keys where possible for obvious reasons.
RackSRV Communications Limited
UK specialists in Dedicated Servers & Server Colocation
Company: 06856870 VAT: GB 934 7073 15 Tel: 0330 111 4444
-
01-31-2013, 01:35 PM #6Web Hosting Master
- Join Date
- Mar 2009
- Location
- Miami, Florida
- Posts
- 20,777
Hello,
You do have a point as if someone exploits the root account or gets their hands on the key, your backup server can easily become compromised. Of course, using the keyless method has advantages when it comes to automation.
A few ways to make the solution a bit more secure would be;
- Do not use the root user on the remote backup server, rather a less priviliaged user with a jailed shell access
- Run the rsync on the backup server to connect to the production server and pull the data rather then push. RSync can work both ways.
- The backup server should be tightly locked down with only ssh running. Ideally it should not be on a public network and accessible only to the production server and a management network
-
02-18-2013, 01:15 PM #7WHT Addict
- Join Date
- Feb 2013
- Location
- Canada
- Posts
- 127
Great tutorial and good information. Very easy to set up as well.
-
07-08-2013, 03:24 AM #8Junior Guru Wannabe
- Join Date
- Jul 2002
- Location
- EARTH
- Posts
- 34
Very nice post. Thank you so much!
1) Really helped me and may be I too tired but I think in rsync command above, /media/localbackupdisk/ should come earlier than root@ip part to push the files to backup server.
2) Also something that can help other silly people like me is don't miss the "ssh -i /root/.ssh/id_dsa_myserver1" part or you will keep getting password prompts.
3) Never work when you get too tired or you will make silly mistakes like me I did today.
4) 5 stars for such a great post. I must admit you are so good at teaching that you made a tough job really easy.
Thanks again!
-
07-12-2013, 10:09 PM #9Web Hosting Master
- Join Date
- Oct 2012
- Location
- Europe and USA
- Posts
- 991
★ NetworkPanda :: Web Hosting SSD Powered :: Reseller Hosting
★ Instant activation, fast servers, NVMe SSD disks, cPanel, Softaculous 1-click apps installer, daily backups
★ Multiple hosting locations: USA, Canada, France, UK, Germany, Italy, Spain, Poland, Finland
-
07-13-2013, 04:38 PM #10Junior Guru Wannabe
- Join Date
- Jul 2002
- Location
- EARTH
- Posts
- 34
Thanks again NetworkPanda for such a useful article and I have one question:
Does rsync command "rsync -uptorgvlHaz -e" also includes synchronization of both folders like --delete command does? I mean if something is deleted from backup folder, will it also be deleted from the remote folder when Rsync cron runs?
Thanks!HitBiz.Net (HITBIZ INTERNATIONAL LLC, a registered US based business)
Domain Names and reliable (raid protected and with daily offsite backups and many more options) shared web hosting, reseller accounts and dedicated servers.
Safety & uptime of our customers' data is our main concern.
-
07-13-2013, 06:12 PM #11Web Hosting Master
- Join Date
- Oct 2012
- Location
- Europe and USA
- Posts
- 991
No, but if you want this to be done, you can add the --delete, --delete-excluded and --delete-after options:
rsync -uptorgvlHaz --delete --delete-after --delete-excluded -e "ssh -i /root/.ssh/id_dsa_myserver1" etc.
The --delete-after option tells rsync to start removing deleted files after the backup of new or modified files is complete. I think this is important, before if there are a lot of files/folders to delete, then the deletion may delay the backup process for several hours, unless you add the --delete-after option.★ NetworkPanda :: Web Hosting SSD Powered :: Reseller Hosting
★ Instant activation, fast servers, NVMe SSD disks, cPanel, Softaculous 1-click apps installer, daily backups
★ Multiple hosting locations: USA, Canada, France, UK, Germany, Italy, Spain, Poland, Finland
-
07-14-2013, 04:47 PM #12Newbie
- Join Date
- Apr 2012
- Location
- Thailand
- Posts
- 16
Hello all.
I setting up an incremental backup for one of my VPS with centos 5. The "destination" of the backup is another VPS with centos 5 too.
While I lunch the following code on the "destination" VPS
Code:cat /root/id_dsa_myserver1.pub >> /root/.ssh/authorized_keys
Code:bash /root/.ssh/authorized_keys: No such file or directory
Thanks
-
07-14-2013, 08:40 PM #13Web Hosting Master
- Join Date
- Oct 2012
- Location
- Europe and USA
- Posts
- 991
★ NetworkPanda :: Web Hosting SSD Powered :: Reseller Hosting
★ Instant activation, fast servers, NVMe SSD disks, cPanel, Softaculous 1-click apps installer, daily backups
★ Multiple hosting locations: USA, Canada, France, UK, Germany, Italy, Spain, Poland, Finland
-
07-16-2013, 10:13 AM #14Newbie
- Join Date
- Jul 2013
- Location
- United Kingdom
- Posts
- 23
thanks for the share!
-
09-04-2013, 09:58 AM #15New Member
- Join Date
- Sep 2013
- Posts
- 1
Backup home dir
Hello,
When I want to backup the user in the /home dir from the original server, what command do I need to execute?
I tried 2 commands, 1 did what it had to do, but put the /home dir from the original server in a /home/home dir on the backup server
The other command deleted all the linux files so I had to install linux again.
Similar Threads
-
Remote Rsync Backup accounts - FTP/SSH/SFTP/RSYNC - 50% Off First Two Months
By Jamesc in forum Backup ServicesReplies: 0Last Post: 10-19-2011, 11:16 AM -
Cpanel Remote Sync incremental Backup Plugin
By Syslint in forum Software & Scripts OffersReplies: 27Last Post: 10-22-2010, 03:51 PM -
cPremote v2.0 - cPanel/WHM remote rsync incremental backup plugin. Offer this feature
By Syslint in forum Software & Scripts OffersReplies: 18Last Post: 08-26-2010, 10:19 AM -
CPanel Incremental Backup (rsync) & RAM
By mainarea in forum Hosting Security and TechnologyReplies: 0Last Post: 03-01-2005, 03:26 PM -
ssh/rsync/cron: How to connect without password?
By jojo235 in forum Hosting Security and TechnologyReplies: 2Last Post: 02-08-2004, 02:19 PM