Web Hosting Talk : Web Hosting Main Forums : Colocation and Data Centers : Network wise: Sustain 200MB/s - 600MB/s data transfer speed between servers ?

[eva2000]

Networking noob for such setups

For colocation, what would be the most cost effective network setup solution for setting up a backup server to sustain 200MB/s to 600MB/s data transfer speeds between source and destination servers decked out with SSDs ?

Any ideas and suggestions would be greatly appreciated

[Spudstr]

A switch with Gigabit ports? Or am I confused by your question?

[virtuallynathan]

I believe he is saying MB/s - so either 10Gbit/s or bonded 1Gbit/s. If it's 200MB/s use 2x1Gbit/s in LACP/802.3ad. If it's 600MB/s, use 10Gbit, it will probably be more cost effective.

[Ionity]

A 10gig switch would probably be best. Depending on what type of traffic and what type of switching gear you use and it's L2-L4 LAG hashing capability... It may not really give you what you need. Plus to get up to 600MB/s you will need 5 or 6 x 1gig ports (gets almost as costly as 10gig gear plus a lot of added config and headache).

[[CTI] Todd]

Originally Posted by virtuallynathan

if it's 200MB/s use 2x1Gbit/s in LACP/802.3ad.

Maybe, depending on traffic patterns and hashing.

[virtuallynathan]

I've had no issues getting 200MB/s+ from 2x1Gbit/s on CentOS + Junos/IOS.

[[CTI] Todd]

It's not the total throughput that's in question.

[bhavicp]

Most cost-effective for you would be based on whether you have switch(s) which support 10Gbit at the moment (along with servers) or if it would require a lot of changes/new gear.

Otherwise it's pretty cheap to add another dual/quad 1Gbit NIC card to your servers and get 48 port gigbit switches and bond them. Configuration is pretty easy.

[KarlZimmer]

Originally Posted by virtuallynathan

I've had no issues getting 200MB/s+ from 2x1Gbit/s on CentOS + Junos/IOS.

With most ways the traffic is balanced, if all the traffic is from one server to one other server, you may run into issues, since it is often done using a hash of the MAC addresses.

[david_halliday]

If you are talking a single transfer wget, rsync, socket to socket a 802.3ad bundle of 1Gigs will not work. This protocol uses a hash of destination ip/mac address to calculate which of the gig links to use. As the mac address and ip are always the same it will always use the same link. An extension also uses a port number in the hash but this can be limiting also for backup solutions as they often use fixed ports.

Just go 10Gig and be done with it. You can get a switch with 48 x 1Gig + 4x 10 gig for $1300 (plus optics).

[serverhouse]

HI,

If your configuration is just for backups and it's only a single device feeding it you could just put 10G cards in each server and a bit of fibre in between, no need to switch it if it's a single device.

If you have a backup server and multiple feeds you may only need the backup server on 10G

But you also need to ensure your SSD's and IO paths are up to the job and you may want to consider what happens when the backup increases in size. I'm assuming you're trying to complete it in a short time window?

[osopolis]

My solution:
1x Dell 6224 with 2 Multimode SFP+ (10GBit) modules (or other cheap 10Gbit switchs) or! no switch and connect both directly
2x Intel X520 10Gbit Netowrk cards
SSD HDD and OS tuning might be needed

Cheers

[eva2000]

thanks guys for the useful input

Definitely rules out bonded gigabit NIC configs if it's 1 to 1 server transfers. The goal was as fast as possible transfers 'short time window'. Would have larger 1-2TB SATA disks on backup server to move and archive older backups off the SSD.

Never dealt with 10Gbit NIC/Switch/cabling gear, so not sure on brands, models or specifics, so any input there would be great too

[serverhouse]

Intel tend to make good NIC's

But if you're really after performance it'll depend on your O/S drive support etc.

Cables will need to be fibre and will be determined by the NIC and distance between the two.

[random321]

Assuming distances are short, wouldn't InfiniBand be just as good and much cheaper than 10G ethernet?