Hello all,

We're about to start off with a VPS, primarily for restructuring our internal infrastructure (we're a web/application hosting company), but may also sell VPS's).

We'll probably go for a Dell 2850 with Dual 3.0GHz Xeon's (2MB L2), 4GB RAM, redundant everything.

Now for the disks... we were initially going for 3x146GB 10krpm U320 on a PERC 4e/Di RAID5 with 256MB battery backed write cache. Now someone said that RAID5 with Virtuozzo is suicide and one should go for RAID10.

Is that what real-world figures tell you and what does SW-Soft recommend (I will ask them directly too)?

RAID5 gives us the option to add disks later on as we grow.

Go for RAID10, it's MUCH better for rebuilds and MUCH faster for writes.

On a PERC4 (LSI) controller RAID 5 is fine. The more disks you throw at it the faster it will get. Real world, real live performance is nearly identical between RAID 10 and RAID 5, especially in Virtuozzo.

Don't be fooled into thinking that your disks will be writing at 30MB/s all the time (if ever). It's not realistic. The truth is, that even on the heaviest servers you'll see writing is less than 20% of your gross disk activity, and it is extremely minimal to remotely require the use of RAID 10.

A 3 disk RAID 5 configuration is more than adequate, of course I'd rather run 6 x 73's but that's just me. You will get much higher read throughput the more disks you run. Also note that back in the 'old days' of the 2650 and 2550 Dells, RAID 5 was a NIGHTMARE. This was because of the Adaptec based (100Mhz Intel 80303) Perc3/Di controller. This is a piece of junk, and if I had some spares here I'd probably start a bonfire. These controllers would write betwen 5 and 8MB/second. Please note that since then, a lot has changed. The LSI based PERC4's are extremely fast. You WILL NOT be dissapointed with RAID 5 on such a controller.

People get easily caught up into thinking that RAID 10 is better bla bla bla, it's all about your controller. I can show you a single drive on a 29160 Adaptec that will score better benchmarks than a RAID 0 on a poor controller such as the PERC3/DI etc (or anything running on the i960 for that matter)

Another thing to consider is this. 4GB RAM on a 2850 is not much RAM. You should consider ordering your machine from Dell with the 'stock' memory (what are they giving 1GB now?) and locating the RAM from a 3rd party.

I recommend getting single ranked 1GB dimms, and running 6 pieces. These will not be that expensive if you find the right vendor. We purchased 48 pieces from Monarch Computers and were satisfied with the parts we received. The brand was 'Patriot' and comes with a lifetime warranty. Note also that the RAM we received from them, labeled Patriot, uses THE EXACT SAME IC'S that are on the OEM Dell DIMMS.

You're looking for DDR-2 PC3200 SINGLE RANKED 1GB DIMMS. DO NOT GET DOUBLE RANKED. The Dell 2850 only allows for 8 ranks. Getting 6 x double ranked dimms will put you out of bounds, (they are cheaper than single rank).

Anyway, I hope I made myself clear from my extremely extensive experience with 2850's and 6650/6850's. RAID 5, fine if you're running a PERC4. RAM, 4GB is not enough, get more.

Regards
Igor

http://www-03.ibm.com/servers/storage/disk/ess/pdf/raid5-raid10.pdf

http://www-03.ibm.com/support/techdocs/atsmastr.nsf/WebIndex/WP100350

In my personal opinion, running heavy Oracle databases is certainly a more difficult task for a disk subsystem than merely running Virtuozzo for web hosting.

:)

--

Attached picture is a graph from one of our IBM x360's, running RAID 5EE across 14 drives. You tell me, if you think this machine needs RAID 10 or not.

http://www.eurovps.com/wht/sda.gif

Originally posted by EuroVPS/Igor
Anyway, I hope I made myself clear from my extremely extensive experience with 2850's and 6650/6850's. RAID 5, fine if you're running a PERC4. RAM, 4GB is not enough, get more.
[/B]

Thank you very much for sharing your experience. Glad to hear RAID 5 will be fine. We intend to extend the array from 3 to 6 at some point as the server fills. Same with memory, we'll be getting relatively affordable 2x2GB dual ranks from Dell (they will match our Kingston pricing on the total server) and add 2GB single ranks as usage increases.

(will read the links posted now)

Too bad it's not an IBM you could use RAID 5EE which is much better. 5EE is so much better that we've stopped buying Dells - period.

We've looked at hp DL380/385 but are currently negotiating a deal with Dell that is looking good, esp. with 24x7 4hr response included.


...eurovps.com/wht/sda.gif


May I ask what you use for data collection and graphing? Cacti?

May I ask what you use for data collection and graphing? Cacti?

State secret. Email me for information.

Originally posted by EuroVPS/Igor
Attached picture is a graph from one of our IBM x360's, running RAID 5EE across 14 drives. You tell me, if you think this machine needs RAID 10 or not.

http://www.eurovps.com/wht/sda.gif [/B]

Actually, I can't really tell. The maximum read is ~5 MB/s with typical < 1 MB/s. That's quite slow, isn't it? We have RAID5 on Dell PowerEdge 2650 that can burst up to 120-150 MB/s (on PERC 3/DC, I believe) and even at its slowest time, it would do ~20-30 MB/s at the very least, and that's the write speed.

Also note that back in the 'old days' of the 2650 and 2550 Dells, RAID 5 was a NIGHTMARE. This was because of the Adaptec based (100Mhz Intel 80303) Perc3/Di controller.


This is surprising. We have two PE2650 running PERC 3/Di controller and we have never had any problem with it. The write/read performance was great. Did you test the PERC 3/Di with write through or write back? PERC 3/Di gave us ~ 70 MB/s write.

The rest of our PE2650 are running PERC 3/DC with battery backup and write-back enabled. No complaint here too.

You did not understand what I'm telling you. That is not a benchmark graph. That is a USAGE GRAPH of what is currently BEING UTILIZED on this Virtuozzo server.

When benchmarked this machine can write 65MB/second easily.

The point of the photograph was to prove that you don't need such radically increased write capabilities i.e. RAID 10 on a hosting or VZ server.

Regarding PERC3/Di, it's junk. We could never get write back enabled w/o a battery, overall the controller doesn't belong in a server, period. Dell finally got it right when they dumped Adaptec in favor of LSI. Dell support even admitted to us on the phone that "yeah we know it's bad, that's why we changed" etc etc.

BTW in case anyone finds this thread years down the road, if you flash back to 2.5 firmware on the PERC3/Di you will be able to enable write-back caching. 2.8 firmware DOES NOT ALLOW THIS w/o a battery.

Originally posted by EuroVPS/Igor
The point of the photograph was to prove that you don't need such radically increased write capabilities i.e. RAID 10 on a hosting or VZ server

I can't see that from the graph. It just shows that your server has very little I/O operations. Under this condition, you can perhaps get the same graph with IDE server. 5MB/s is very little, even and IDE can do it easily.

Anyway, I can't make any conclusion whether RAID10 is better than RAID5 from your graph since the graph does not show the limit of either what RAID5 or RAID10 or a bare SCSI can do.

Originally posted by EuroVPS/Igor
Regarding PERC3/Di, it's junk. We could never get write back enabled w/o a battery, overall the controller doesn't belong in a server, period.


You are not supposed to use write-back on a RAID controller without battery backup. You are risking a loss of data in the event that your server loss power. All the cache pending to be written to the drive will be lost too! Even on newer PERC card (PERC 3/DC, PERC 4/SC), I don't think you can set to write back without baterry backup (e.g. this option is not available under PERC 3/DCL).

BTW in case anyone finds this thread years down the road, if you flash back to 2.5 firmware on the PERC3/Di you will be able to enable write-back caching. 2.8 firmware DOES NOT ALLOW THIS w/o a battery

And that's a good thing! I won't even let my admin accidentally set to write-back without battery backup.

Again, you don't understand. This is a loaded, Virtuozzo server running 2.6.2 VZ on RHEL AS 3.0 base.

The DEMAND placed upon the machine by CUSTOMERS is no more than 4-5MB/second read, and maybe 1-2.2MB/second write.

Do you think you NEED RAID 10 to provide blistering 168MB/second write speed? Notice that you're reading 3-4X as much than you're writing, it makes all the sense in the world, a web server (which is what VPS customers are typically) are writing logs, and whenever someone updates a SQL database etc. When people visit a website, disks are being READ. RAID 5 allows for RAID 0-like-speeds on reads as data is striped across 10-12 disks.

Originally posted by EuroVPS/Igor
Again, you don't understand. This is a loaded, Virtuozzo server running 2.6.2 VZ on RHEL AS 3.0 base.


Believe me, I understand you. I will not go with RAID10 too on my Virtuzzo server since RAID5 is enough for me. But your graph only tells you about your server and that it's not very I/O intensive. From your graph I do agree that RAID10 is not needed for you, but it does not mean the same goes for everybody else. Their Virtuozzo may be much more I/O operations than mine/yours. And I can't just ask them to see my graph on my not so I/O intesive server and basically telling them that their Virtuozzo server will be not so I/O intensive as well.

So, to the original poster, I can't say whether you need RAID10 or not. But from my experience, RAID5 has been good. Do make sure you have battery backup and never-never set your RAID to write-back without battery backup (read your RAID manual)). I think your server configuration is great!

And Igor, will you stop PM-ing me everytime I am not agreeing with you. What's the problem? This is an open community, everybody is free to agree or disagree with anybody else. And yes, setting a write-back policy on your RAID without battery backup is wrong! I am surprised that you are very offended by this fact.

BTW,

If you plan to run on RAID5, at least have 4 drives. The more drives, the faster the read performance is. Even if you only need 3 drives, it's good ot put the extra drive as your HOT spare. In case one drive in your RAID5 fails, your HOT spare backup will take over. Remember that more than one failed drive in your RAID5 array will basically wipe out everything you have.

Dave:

The PERC3/Di doesn't have a capability for battery backup as delivered by Dell in the PowerEdge 2450.

Thus, it would be impossible to install such a battery on a PERC3/Di equipped 2450. Obviously this was remedied in the 2650.

BTW, Nice sig!

Igor, I don't care about PE 2450 or even 2550. I care about your comment telling people that PERC 3/DI is aweful in PE 2650. Our experience tells us otherwise, and I just state my experience. In fact, we use PERC 3/DI (with write-back and battery backup) on one of our VPS server and zero complaint from us/customers.

BTW, I hope you won't PM me again with your nasty and insulting message next time I state my experience that may not agree to yours. Please remember that it's not only you who are entitled to expressing experience here in WHT. And no, it's not only you who have a lot of experience with Dell servers. So, don't try to stop me from expressing my experience, be it in agreement or in disagreement with yours. Your prorfessionalism is highly appreciated. Thank you!

Nice signature Dave! MANY professional providers have quit using 2650's under 2.4 just because of the extreme unreliability and poor performance of the aforementioned controller. Just go to the RedHat lists and see for yourself.

I must have touched a nerve somewhere when I knocked your 2650's, I didn't even know you used the machines. It really doesn't matter, if it works for you that's great, for us it didn't work, so I stated as such. For you to come on here ranting and raving about me choosing to run write/back on not even knowing what I was talking about, and somehow trying to insinuate that we don't know how to setup our servers i.e. 'I would never let my admin set w/b on w/o a battery'.

remember that it's not only you who are entitled to expressing experience here in WHT.

Did I claim that I was the only one that was entitled to the "honor" of such postings? I don't think so..

BTW, keep posting the sig, great free advertisement!

Did I claim that I was the only one that was entitled to the "honor" of such postings? I don't think so..


So, refrain yourself from your childish personal PM attack, will you?

Write-back will improve your write speed. That's all I am saying. Our RAID5 write performance is great, probably because of the write-back (And we are safe to do this because of the battery backup). I clearly stated that we run some PE2650 with PERC 3/DI and write back enabled. To which you replied that PERC 3/DI is a junk. Perhaps that's just your way to badmouth your competitors publicly as well your attempt to insult me personally?

I don't care about how you set your write policy in your RAID card. Hopefully you don't have live customers on such system with write-back but no battery backup. If so, pray that you don't have loss of power during write session! I state this simple fact to prevent others from doing the same as you. So do no need to take this personally and to even PM me, for goodness sake!

I am stating the fact that it's very dangerous (and not recommended) to set your RAID controller to write back without battery backup. Thank goodness the new firmware disallow you to have write-back if you don't have battery backup. For you, this is a misfortune. For me, it's a great blessing as I don't want my admin/tech so set the write policy incorrectly. What do I do wrong to state this?

Thanks for your compliment and admiration on my signature. My signature is bound within WHT rule. So keep your insult to yourself. If you think it's a free advertisement, feel free to report it to mods.

Ok then, sorry for my opening post as this has lead to this rant with new posts every 10-20 minutes but no real *new* content begin added. So you've all said your thing and have different ideas. No problem there.

It's nice to hear you both agree that the PERC 4e/Di is a nice controller for my suggested RAID 5 setp.

I ordered with Dell and they gave me a good introduction discount (we've used hp exclusively until now).

I will post my experiences with this brick in a few weeks.