Web Hosting Talk







View Full Version : BQ Internet - Various rsync configurations


HostXNow
11-28-2009, 11:53 AM
The rsync configurations/commands BQ Internet provide are:

Back up entire server: rsync -avz --exclude=/proc -e ssh / yourusername@yourusername.bqbackup.com : server1
Back up home directories: rsync -avz -e ssh /home yourusername@yourusername.bqbackup.com : server1
Back up specific users: rsync -avz -e ssh ~bob ~bill ~sarah yourusername@yourusername.bqbackup.com : server1

Backing up the entire server would cover the MySQL databases, but it is really necessary to backup the entire server? I think not...

Just backing up the home directories and /var/lib/mysql (Mysql Databases) would be better. But does anyone know the commands needed to do that?

Any help would be appreciated...

Chris

RSanders
11-28-2009, 12:16 PM
Hello,

Backing up the entire server is GREAT. If you have a failure, you can just replace your system, create your partitions, copy your data, install your boot loader.

But to answer your question,

rsync (options) source destination

rsync -a /home/ you@backupserver.com:/some/backup/home/
This will backup /home/ to /some/backup/home/ on another server. If you do not use root, your permissions will get scrambled. In this case, use tar or a file image on your backup server and not raw rsync.

rsync -a /var/lib/mysql/ you@backupserver.com:/some/backup/mysql/
This will copy your mysql files, but it is much better to do a dump and back those up.

The -a means archive, or save all the important details like permissions
The -v means verbose, shows you every file (also try --progress)
The -z means compression. If your doing this on a lan or busy server, you probably have more pipe than CPU and can skip compressing this.

This script is wonderful,
http://worldcommunity.com/opensource/
It will do mysql backups, rotate backups, FTP backups and make breakfast :)

Thanks,
Rob

MH-Andy
11-28-2009, 12:30 PM
Although we don't use BQ Internet, we use our own backup servers for MySQL Backup's we run this every six hours, on top of daily backups to another server too.

rsync -avz -e "ssh -p 22" /var/lib/mysql RemoteUser@12.12.12.12:BackupDirectory

Saved us more than once.

RSanders
11-28-2009, 01:23 PM
Ah yes...

-e ssh is certainly required to connect to remove hosts, my bad for typing off the top of my head.

The "ssh -p 22" is only needed if you need to pass variables to ssh, i.e. "ssh -p 2202' to use ssh on port 2202

I still recommend dumping mysql and backing that up as well. That has saved us more than once as well on very active large databases.

bqinternet
11-28-2009, 01:57 PM
is really necessary to backup the entire server? I think not...

If you know exactly what directories you need to back up, then it is not necessary to back up the whole server. By default, we recommend backing up the whole server so that you don't miss anything.

The home directory would contain most important files, but as you noted, it won't include MySQL databases. What about emails? What about configuration files? What about control panel user data?

bear
11-28-2009, 02:06 PM
Scott, what about creating a few simple scripts to back those up using rsync and making those available? I'm sure those new to rsync would love to have a simple script they could call via cron (instead of all the commands in the cron itself) that would simplify things once they've set up keys and all that.

RSanders
11-28-2009, 02:15 PM
If you know exactly what directories you need to back up, then it is not necessary to back up the whole server. By default, we recommend backing up the whole server so that you don't miss anything.

The home directory would contain most important files, but as you noted, it won't include MySQL databases. What about emails? What about configuration files? What about control panel user data?

The golden rule is, if you didn't back it up you will need it. If you have full backups, nothing will break :)

HostXNow
11-28-2009, 03:11 PM
By default, we recommend backing up the whole server so that you don't miss anything.

Ok, I will backup the whole server instead.

I used:

echo "15 22 * * * root rsync -avz --exclude=/proc -e ssh / exammpleuser@exampleuser.bqbackup.com : serverexample789" >> /etc/crontab

and after got:

root@server [~]#

What is the command to run rsync now?

Thanks

RSanders
11-28-2009, 03:16 PM
I used:

echo "15 22 * * * root rsync -avz --exclude=/proc -e ssh / exammpleuser@exampleuser.bqbackup.com : serverexample789" >> /etc/crontab

What is the command to run rsync now?

Thanks

Log in as root and run
rsync -avz --exclude=/proc -e ssh / exammpleuser@exampleuser.bqbackup.com (exammpleuser@exampleuser.bqbackup.com): serverexample789

This will have the same result as your cron. You should also be able to see it execute in your logs.

I exclude /dev and /sys on my own machines as well.

HostXNow
11-28-2009, 03:24 PM
Log in as root and run
rsync -avz --exclude=/proc -e ssh / exammpleuser@exampleuser.bqbackup.com (exammpleuser@exampleuser.bqbackup.com): serverexample789

This will have the same result as your cron. You should also be able to see it execute in your logs.

I exclude /dev and /sys on my own machines as well.

Cheers Rob

Now I don't know whether I should exclude /dev and /sys too?

Scott recommends to backup whole server?

Which is best to do guys? Backup all apart from /proc or can I exclude /dev and /sys too? If so, how do I do that ... like --exclude=/proc/dev/sys ?

I want the most secure method but the quickest one in case I need to restore files

Thanks

RSanders
11-28-2009, 03:38 PM
Cheers Rob

Now I don't know whether I should exclude /dev and /sys too?

Scott recommends to backup whole server?

Which is best to do guys? Backup all apart from /proc or can I exclude /dev and /sys too? If so, how do I do that ... like --exclude=/proc/dev/sys ?

I want the most secure method but the quickest one in case I need to restore files

Thanks
I recommend following your service providers guidelines and documentation. There are numerous ways to do it, and the rest are at your own discretion.

Just remember it's rsync options source destination. i.e.
--exclude/proc --exclude=/sys
also
rsync -avz -e ssh exammpleuser@exampleuser.bqbackup.com : serverexample789/path/to/good_file /local/path/to/file
The second coping remote files back.

Thanks,
Rob

HostXNow
11-28-2009, 03:40 PM
Ok, I will leave it as it is.

Thanks.

bqinternet
11-28-2009, 03:45 PM
Now I don't know whether I should exclude /dev and /sys too?

You can if you wish, but it's not necessary. The /proc directory is a bit different, because the contents constantly change, resulting in the accumulation of tens or hundreds of thousands of files in the backup after a while. That is why the Rsync syntax in our instructions specifically excludes /proc.

can I exclude /dev and /sys too? If so, how do I do that ... like --exclude=/proc/dev/sys ?

You can include multiple --exclude= options in the same command:

rsync ... --exclude=/proc --exclude=/sys ...

Nickos
11-28-2009, 08:12 PM
rsync -avPe "ssh -p 22" /var/lib/mysql RemoteUser@12.12.12.12:BackupDirectory

Then ctrl+z, then jobs, then jobs %1 and then you can disconnect from the shell.Job will be done in the background. :)

This is simple way, you need to find some backup script with cron if you want to be sure everything is running as it should.Maybe even sending you an email if everything went well.