Page 1 of 2 12 LastLast
Results 1 to 25 of 41
  1. #1
    Join Date
    Apr 2005
    Posts
    697

    incremental rsync

    Hi guys,

    I have two folders:

    /misc
    /misc-backup

    how do I go about making /misc-backup a mirror of /misc correctly?

  2. #2
    You could do it like this:

    Code:
    rsync -av --delete /misc/ /misc-backup/
    || ServaxNet (AS46974)
    || Server Management and Managed Hosting Experts Since 2004

  3. #3
    Join Date
    Apr 2005
    Posts
    697
    right, but does it run in the background? what if /misc a file gets overwrite on /misc which is a new file, would it update /misc-backup? also, how can we make this auto start when the server gets rebooted.

  4. #4
    Since you're using it to make a backup of a directory, I'm guessing you're going to just be making a cron out of it.

    The rsync command would delete any files that are not there anymore, and replace any files that have changed.

    As to your auto start when server gets rebooted.. Again I'm assuming you'd put this as a cron.
    || ServaxNet (AS46974)
    || Server Management and Managed Hosting Experts Since 2004

  5. #5
    Join Date
    Feb 2005
    Location
    Australia
    Posts
    5,849
    Bear in mind though that mirrors don't make good backups. What if you accidentally delete a file in /misc... Oops, it's gone from misc-backup too. And are the two directories on the same drive? Oops, a hard-drive failure just destroyed them both. Or even if they're on two different drives, what if a kiddie gets into your server and deletes everything? What was that rm -rf command again?

    With a little extra setup rsync is great for remote backups and keeping multiple snapshots, which will protect you from all of these problems. Check out rsnapshot or Mike Rubel's tutorial for more info.
    Chris

    "Some problems are so complex that you have to be highly intelligent and well informed just to be undecided about them." - Laurence J. Peter

  6. #6
    Join Date
    Apr 2003
    Location
    Melbourne, AU
    Posts
    539
    ti_nhatrang, what situations do you expect your backups (done your way) to cover?
    WK Woon
    CTO | http://www.aflexi.net - A flexible Network
    Building the next generation CDN platform - DEMO .... coming soon

  7. #7
    Join Date
    Apr 2005
    Posts
    697
    One folder is mounted so that's fine. I'm just looking for things to be mirrored so I can have the 2nd server bring up the files directly rather than going off the mounted folder.

  8. #8
    Join Date
    Jun 2006
    Location
    Tucson
    Posts
    23
    to run in the background, place a nohup in front of the command and & at the end, that way you can log off and it'll still run:

    nohup rsync -av --delete /misc/ /misc-backup/ &

  9. #9
    Join Date
    Apr 2005
    Posts
    697
    I try doing that in cron but it doesn't start, can you please help me figure this out on how to make this run in /etc/crontab?

    Thank you.

  10. #10
    Join Date
    Jun 2006
    Location
    Tucson
    Posts
    23
    You may have to specify the full path to the commands in the script or cron, usually located under:

    /usr/local/bin/rsync
    or
    /usr/bin/rsync

    you can find the location by keying in:

    which rsync

    and it'll display the path.

    You may also have to restart crond. The output of cron is normally sent somewhere like: /var/spool/mail/root (or whichever user you are executing the cron as)

  11. #11
    Join Date
    Apr 2005
    Posts
    697
    /usr/bin/rsync nohup rsync -av --delete /misc/ /misc-backup/ &

    That's what I have in crontab but when I restart the server, it doesn't seem to run.

  12. #12
    Join Date
    Jan 2006
    Location
    Athens, Greece
    Posts
    1,481
    */10 * * * * /usr/bin/rsync -av /misc/ /misc-backup/ > /dev/null 2>&1

    Every 10 minutes.

  13. #13
    Join Date
    Mar 2007
    Location
    Dublin
    Posts
    17
    Use something like rsnapshot or rdiff-backup to make your life a little easier.
    Want me for contracting? Please read

    http://www.brassy.net/contracting/

  14. #14
    Join Date
    Apr 2005
    Posts
    697
    Quote Originally Posted by Steve_Arm View Post
    */10 * * * * /usr/bin/rsync -av /misc/ /misc-backup/ > /dev/null 2>&1

    Every 10 minutes.
    Hmm, Wow, excuse my newbie thinking but I didn't know you actually have to run it multiple times. I thought it does it automatically and run in the background as a process. I thought it sits there and wait to fetch new/delete files.

  15. #15
    Join Date
    Apr 2005
    Posts
    697
    Quote Originally Posted by taec View Post
    Use something like rsnapshot or rdiff-backup to make your life a little easier.
    I would love to make my life a little easier. Can you please tell me how those will change?

    Bottom line is I want both folders to have the identical files/folders at all times.

  16. #16
    Join Date
    Aug 2003
    Location
    Gods Own Country
    Posts
    892
    Hope this link will be of some help
    http://gentoo-wiki.com/HOWTO_Backup
    Blessen Cherian
    Follow me on twitter.com/blessenonly
    Two decade in Web Hosting Industry

  17. #17
    Join Date
    Apr 2005
    Posts
    697
    Also, im rsync'ing 1TB of DATA, is this a good way to do it with this amount in size?

  18. #18
    Join Date
    Apr 2005
    Posts
    697
    Also, nohup is creating a logs which is getting pretty huge, how can we not have this log created?

  19. #19
    Join Date
    Feb 2005
    Location
    Australia
    Posts
    5,849
    Leave off the v flag (verbose) ie. rsync -a source destination.

    But you should be running it as a cron job.
    Chris

    "Some problems are so complex that you have to be highly intelligent and well informed just to be undecided about them." - Laurence J. Peter

  20. #20
    http://www.cis.upenn.edu/~bcpierce/unison/

    Should be already on your system, command called unison.

    Good luck,
    Pedram M.

  21. #21
    Join Date
    Apr 2005
    Posts
    697
    wow, so many tools out there... But my main question and hopefully to get a good recommendation on which tool I should use to sync 700GB-1TB of data?

  22. #22
    Join Date
    Nov 2004
    Location
    Australia
    Posts
    1,737
    rsync is fine with that. It does block checksums on large files and only transfers the parts of the files that have changed. However it does need to read and checksum the entire file to do that.

    Of course, optimally, you should have your 1Tb of data spread across many files. And yes, rsync will work beautifully, and optimally for that.

    I'd live with the logfiles getting large (but do remove -v) as you'll need them for debugging in the future when something goes wrong.

  23. #23
    Join Date
    Apr 2005
    Posts
    697
    I had to reformat my server because it through off my volggroup0... I'm trying again without the logs or -v, just the -a by itself.

  24. #24
    Join Date
    Apr 2005
    Posts
    697
    Quote Originally Posted by Steve_Arm View Post
    */10 * * * * /usr/bin/rsync -av /misc/ /misc-backup/ > /dev/null 2>&1

    Every 10 minutes.
    I don't think this is working for me. How do I check to see if the cron ran or not?

  25. #25
    Join Date
    Aug 2006
    Location
    Duluth, GA
    Posts
    201
    less /var/log/cron

Page 1 of 2 12 LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •