Results 1 to 15 of 15
-
09-15-2009, 03:52 PM #1Junior Guru
- Join Date
- Mar 2009
- Posts
- 245
Best way for transfer tons of files?
I have directory of 90 gb with tons of files which i need to transfer to another server.I usually tar directory with command tar -pczf bla.tar.gz directory,but in this case it will take ages first to archive files,then to transfer,and then to extract.Is there faster way which doesn't involve archiving?
-
09-15-2009, 04:06 PM #2Web Hosting Master
- Join Date
- Oct 2006
- Location
- /usr/src/linux/
- Posts
- 700
I'd use rsync and renice its pid to avoid hogging resources, but it will still take a while.
█ VPSnoc.com offers high quality Xen® OpenVZ & Windows® Virtual Private Servers at affordable prices.
█ 99.95% Uptime | 24/7/365 Support | Unmetered bandwidth.
█ Follow us: twitter.com/VPSnoc
-
09-15-2009, 04:27 PM #3Web Hosting Master
- Join Date
- Aug 2009
- Location
- Orlando, FL
- Posts
- 1,063
I agree. Rsync is your best bet. At least that way if the transfer gets interrupted you won't have to start all over again.
-=SKULLBOX.NET=-
-
09-15-2009, 04:54 PM #4Junior Guru
- Join Date
- Mar 2009
- Posts
- 245
Well,it would be nice if someone give a example command.Also what about scp?I read how that is also good.
-
09-15-2009, 05:24 PM #5Newbie
- Join Date
- Sep 2008
- Posts
- 17
Yea Rsync it`s by far the best way to transfer such amount of files.
FROGOST - Cheap Web Hosting
-
09-15-2009, 06:30 PM #6WHT Addict
- Join Date
- Dec 2002
- Posts
- 160
these days rsync is tunnelled over ssh so is the same as scp really.
use something like this (as root) from the server with the files on it, to a remote server, assuming you want to backup the whole /home/website/ directory preserving permissions/uid's and using ssh compression:
Code:rsync -avzp --progress --delete-after --numeric-ids /home/website/ root@newserver:/home/website/
or use the -n flag to just show you what it will do without actually doing it.
you have to be root on both machines to preserve the permissions, unless you only want to read/write a temp directory or $home.
usual disclaimer applies, try it on a dev box first or a test directory of unimportant files, also read up on rsync.Last edited by sej7278; 09-15-2009 at 06:35 PM.
-
09-21-2009, 02:41 PM #7Junior Guru
- Join Date
- Mar 2009
- Posts
- 245
I used following command:
scp -r directory root@ipadress:/home/bla
And it works fine.
Now there is following problem:i need to transfer 90 gb of files.I transfering it over LAN(both machines are at same datacenter)and that gives me max speed.But can i speed it up by repeating scp on another subdirectory?And also how do i put password without question which shows after i type command.I need that so i can put proccess into background.
-
09-21-2009, 09:52 PM #8Virtually Flawless ;)
- Join Date
- Apr 2009
- Location
- USA / UK
- Posts
- 4,577
→ RAM Host -- USA Premium & Budget Linux Hosting
█ Featuring Powerful cPanel Shared Hosting
█ & Premium Virtual Dedicated Servers
→ Follow us on Twitter
-
09-22-2009, 01:52 AM #9server automation specialist
- Join Date
- Apr 2009
- Location
- Nevada
- Posts
- 662
James B
►WWW.EZEELOGIN.COM |Setup your Secure Linux SSH Gateway►MEET PCI DSS & ISO 27001 Compliance|Manage & Administer Multiple Linux Servers Quickly & Securely.
-
09-22-2009, 03:24 AM #10Temporarily Suspended
- Join Date
- Jul 2009
- Location
- Manila
- Posts
- 301
Or you can use the bg command. You can check on this tutorial:
http://nolimitshostdesk.luv.ph/knowl...hp?article=113
I believe there is no way to "remember" passwords for scp. you need to have keys or use one scp to transfer the entire directory
-
09-22-2009, 05:40 AM #11Newbie
- Join Date
- May 2009
- Location
- L'viv Ukraine
- Posts
- 21
scp is suxx - doesnt keep file attributes
use rsync - its really rocks and you can restart it
from where its stopped every time its crashed ;-)
-
09-22-2009, 08:43 AM #12Support Facility
- Join Date
- Jun 2009
- Posts
- 2,335
Rsync is freely available and it is the fastest best mode to transfer the files. You will be happy to use it.
-
09-22-2009, 10:43 AM #13Newbie
- Join Date
- Sep 2009
- Posts
- 13
You can skip password prompt by using SSH Public Key Authentication.
Check if public key is already present in the source server.
Code:# cat /root/.ssh/id_rsa.pub
Code:scp /root/.ssh/id_dsa.pub root@destination.com:/root/.ssh/authorized_keys2
Code:$cd /root/.ssh/ $ssh-keygen -t rsa
Then copy the public key to destination server as explained above.
-
09-22-2009, 11:56 AM #14Newbie
- Join Date
- May 2009
- Location
- L'viv Ukraine
- Posts
- 21
hmmm...
ssh-keygen -t rsa
for dsa keys? weird...
-
09-22-2009, 01:37 PM #15Newbie
- Join Date
- Sep 2009
- Posts
- 13
You can use “dsa” instead of the “rsa” after the -t to generate a DSA key.
Code:$ssh-keygen -t dsa
Similar Threads
-
Ftp files transfer
By decybel in forum Hosting Software and Control PanelsReplies: 1Last Post: 09-26-2007, 01:16 PM -
Transfer files to new vps
By ltaylor in forum VPS HostingReplies: 4Last Post: 01-07-2007, 03:39 AM -
How can i Transfer files from one PC to another?
By Mike235 in forum Web Hosting LoungeReplies: 8Last Post: 11-26-2005, 07:38 PM -
$5.99 gets you 500 mb space 10gb transfer 2 domains and tons of features $5.99
By MarcD in forum Shared Hosting OffersReplies: 2Last Post: 02-24-2003, 11:22 AM -
tons and tons and tons of fraud
By ebo in forum Web Hosting LoungeReplies: 1Last Post: 03-29-2002, 03:04 AM