Results 1 to 9 of 9
-
06-01-2011, 11:52 AM #1Aspiring Evangelist
- Join Date
- May 2004
- Location
- Corona, CA USA
- Posts
- 386
What Method of HD Cloning Uses Least CPU
I have a live production server running my most important client and own sites that has developed critical errors on the third hard drive used for backups only (/dev/sdc). The machine is running CentOS 5.6 with WHM/cPanel, and of course I can haz r00t. It's in a 4 bay SuperMicro chassis that supports hot swapping. The 4th bay is currently unused. The machine has a Core 2 Quad Q9550. The failing drive is a WDC WD1002FAEX-00Z3A0 1TB drive.
My competent and reliable provider for this dedicated server has suggested that they can add an identical replacement 1TB drive for me to partition/format and then copy my retrievable backups. Since the fstab uses labels it should be trivial to do this.
Here finally is my question: what method of copying an entire hard drive will produce the least CPU load? I'm thinking tar might be the best choice. I expect using dd would be unwise with a drive producing read errors, and I'm guessing rsync would use more cpu than tar.
Suggestions?
Bonus question: what switches would you use with tar to clone a hard drive?██ Skeptic Hosting
██ Promoting a reality-based lifestyle choice
██ Hosting by invitation only, nothing for sale
-
06-01-2011, 12:43 PM #2Junior Guru Wannabe
- Join Date
- May 2011
- Location
- London, UK
- Posts
- 32
Hi srider,
I would choose Rsync.
Remember that with TAR, it will be a two-step process, you'll need to archive then extract the data.
With Rsync you can limit the IO/CPU load by adjusting the transfer rate.
Also use the verbose switch to get useful output which will help you identify any transfer errors, since the source is unreliable.
Obviously, don't use compression.
VeselinLast edited by kantsev; 06-01-2011 at 12:47 PM.
-
06-01-2011, 12:55 PM #3Russ
- Join Date
- Mar 2002
- Location
- Philadelphia, PA
- Posts
- 2,517
dd (disk dump).
-
06-01-2011, 02:06 PM #4Aspiring Evangelist
- Join Date
- May 2004
- Location
- Corona, CA USA
- Posts
- 386
Thank you, TAR is out. Assuming that the failing drive is mounted at /backup and the replacement at /backup2 would you use something like
rsync -rvpt --bwlimit=18000 /backup/* /backup2/ > /tmp/cloneresults.txt &
I wondered if it would be unwise to use dd with a drive that is failing. Would it handle read errors in a way that is better than rsync?
Thanks to both of you good people for your replies.██ Skeptic Hosting
██ Promoting a reality-based lifestyle choice
██ Hosting by invitation only, nothing for sale
-
06-01-2011, 02:54 PM #5Web Hosting Master
- Join Date
- Oct 2002
- Location
- Vancouver, B.C.
- Posts
- 2,699
All the TAR format does is concatenate files together, so computationally and effectively speaking, it would not be a two-step process; you'd just be copying the files over. The only overhead will be from piping the data, which is pretty minimal.
rsync on the other hand is always going to be computationally intensive because it has to compare data in multiple combinations. Regardless of how much you limit the IO/CPU, it's still taking up the same number of clock cycles; you're just spreading it out more. If anything, you'll use up even more clock cycles because you'll need more context switches between rsync and other processes over the course of the rsync run. Unless you're cloning to a drive that already has most of the data already, it's pointless to use rsync.
The best way to reduce CPU is to keep things simple. Use dump with -0 for a full backup, as this is what dump is specifically designed for. dd is a bit lower level than necessary. tar can be useful if you need some extra flexibility. If you don't need to preserve anything on the filesystem level (like permissions or modification times), even a recursive cp will work.ASTUTE INTERNET: Advanced, customized, and scalable solutions with AS54527 Premium Performance and Canadian Optimized Network (Level3, Shaw, CogecoPeer1, GTT/Tinet),
AS63213 Cost Effective High Performance Network (Cogent, HE, GTT/Tinet)
Dedicated Hosting, Colo, Bandwidth, and Fiber out of Vancouver, Seattle, LA, Toronto, NYC, and Miami
-
06-01-2011, 04:25 PM #6
If you haven't lost your primary data, then are the backups really important? On a failing drive, you might not be able to trust any of the data, first of all, and secondly, it can hang the system to try to recover from read errors. The cpu usage of many copying methods is reasonably low, but a server can behave unpredictably when reading a lot of data from a questionable drive.
IOFLOOD.com -- We Love Servers
Phoenix, AZ Dedicated Servers in under an hour
★ Ryzen 9: 7950x3D ★ Dual E5-2680v4 Xeon ★
Contact Us: sales@ioflood.com ★
-
06-01-2011, 04:54 PM #7Junior Guru Wannabe
- Join Date
- May 2011
- Location
- London, UK
- Posts
- 32
Hi srider,
hhw's comment states some useful facts on resource efficiency, so if you are looking for best utilization, it'd be beneficial to check the tools mentioned there.
Otherwise if you are still interested in Rsync, you could use:
rsync -av --bwlimit=18000 /backup/ /backup2/ > /tmp/cloneresults.txt
- you dont need the *, a trailing / is enough
- it will be better to leave the process in the foreground, so skip the &
- with this redirection, standard output will go to the TXT file and error will print on the console
- you could capture error output to a file too if you want, in case later you decide to review which files/folders failed to transfer and for what reason
rsync .... >standard.log 2>error.log
-
06-02-2011, 02:29 PM #8Aspiring Evangelist
- Join Date
- May 2004
- Location
- Corona, CA USA
- Posts
- 386
Thanks to all of you for helping a 34% senile old man.
We discovered that the 4th bay in my server was not cabled. So we put the new drive in bay 3 (sdc) and after I did fdisk and mkfs.ext3 with it I tried to use
dump -0u -f /dev/sdd1 /dev/sdc1
while the failing drive was mounted (and mounted) in an external USB enclosure.
A few minutes into that process it appears the failing drive gave up the ghost, as CentOS is no longer able to find an ext3 filesystem on it.
So the new drive will simply get filled up as new backups are made.
As funkywizard pointed out, nothing critical has been lost.██ Skeptic Hosting
██ Promoting a reality-based lifestyle choice
██ Hosting by invitation only, nothing for sale
-
06-02-2011, 03:08 PM #9Retired Moderator
- Join Date
- Aug 2003
- Location
- PA
- Posts
- 1,914
Originally Posted by srider
Originally Posted by funkywizard
Similar Threads
-
Website Cloning
By whitecollar in forum Other Offers & RequestsReplies: 16Last Post: 11-15-2006, 01:02 AM -
New CPU performance measuring method
By Opteron4P in forum Dedicated ServerReplies: 0Last Post: 08-22-2005, 06:51 AM -
Cloning - What is it?
By HelpQuick in forum Web Design and ContentReplies: 6Last Post: 10-10-2004, 04:13 AM -
Cloning
By Artashes in forum Web Hosting LoungeReplies: 10Last Post: 03-10-2004, 01:02 PM -
Method to check each user's CPU/RAM usage?
By wheimeng in forum Hosting Security and TechnologyReplies: 12Last Post: 10-12-2003, 11:33 PM