A lot of people have been telling me that IDE is better for web hosting now then SCSI. So I am doing this survey to find out what web hosts prefer.

So, why do they say IDE is better? To save some little cash? ;)

cheers,
:beer:

Originally posted by netsolutions
A lot of people have been telling me that IDE is better for web hosting now then SCSI. So I am doing this survey to find out what web hosts prefer.

:eek:
Who told you that IDE is better than SCSI in any way at all (except maybe for price per megabyte)? Even if you're going to put one popular site or if you have a webhosting business and are going to put tons of customers on your server, then you will probably get at least a few hundred hits per minute. Do you really think an IDE disk can handle the pressure and is as good as SCSI when it comes to multitasking? The only time I would recommend IDE drive in a webserver would be if you were going to put a so so popular site on your server and you were looking for the cheapest solution, or maybe as a backup drive.

This guy told me that IDE is way better because they run a lot cooler and are less prone to disk failure.

Whoever told you that needs to have their head checked....

IDE drives are single access and normally move at around 7200 rpms. This means that one process can write/ access the drive at once and while each access may be very quick, under deceent load, the line to get to the drive gets rather long. Because IDE is single access data can easily corrupt if multiple programs are trying to write to it at the same time.

SCSI on the other hand is multi-access which allows the drive to work as fast as calls are made. Normally SCSI drives are 10,000 or higher rpm and can handle many requests at the same time. SCSI drives corrupt far less and have great error/ sector checkings built in.

IDE's advantage is price... a 36 gig scsi drive runs for about $200 (ibm deskstar 7200 rpm series) and about $295 for a 10,000 rpm version (plus a controller card). 40 gig IDE drives are about $85-95 which is obviusly far cheaper but by no means better. Many hosts and servers use ide ebcause of the cost and simply hope their drives can cut it, most of the time they can but if you want anything big stay away from ide.

IDE is cheaper.
SCSI is better.

SCSI drives may require more airflow to stay cool, but this just means you need a decent enclosure. SCSI drives usually have 5-year warranties whereas IDE drives usually only have 3-year warranties, and the mean time between failure in hours of runtime is usualy 3x as long for SCSI drives.

I have seen servers running hot swap, RAID1, IDE drives with over 500 domains on them.

It is strictly a cost to storage ratio that makes IDE better. For everything else, SCSI is best. However, don't think you can not provide excellent service and host numerous sites on IDE drives. Just make sure you have back ups :)

These days most manufacturers produce SCSI and IDE from the same HDA (Head Disk Assembly), same controller chip and the only difference is the host interface. So in terms of sustained speed, it is going to be the same, as it depends on the HDA itself.

However, IDE has a shorter data path / simpler protocols as it goes through DMA, it transfers data to/from the disk very quickly.

SCSI disks on the other hand have a scsi controller build into the disk and thus data has to travel through SCSI command overheads.

However, since SCSI disks have this built in SCSI controller, they support among other things, command queueing. This allows the host controller to send the command and return immediately without having to wait for the command to be executed. Further, several commands can be sent for the SCSI controller on the disk to execute.

This feature alone will increase SCSI disks performance under multi tasking environment, when there are tens or hundreds of processes hitting the disk.

In short, SCSI is better simply because of multi tasking ability, and the ability for each SCSI disk to perform independently without having to wait for other disks, unlike IDE (on the same controller).

If there's any mistakes in my explanation please correct me. Thanks

Note though, if you are considering RAID, RAID1 makes things slower, but more reliable. RAID0 makes things faster but less reliable than a single disk. Raid 5 makes things faster and more reliable, but will only tolerate one disk failing. Raid 0+1 makes things faster and more reliable and can tolerate two disks failing (depending on which disk).

Then there's "El Cheapo Raid" that is..... copy your data regularly to another disk... hehehe ;)

What the poster neglected to mention was that said person said "IDE drives in a RAID5 array were much better than 2 x SCSI drives in a RAID1 array". Putting 4 or 5 drives in an IDE RAID5 array is better than having two 73Gb SCSI drives in a RAID1 array, not only is it faster (Discount writes, RAID5 with any disks is slower at writes than RAID1 will be), it is more reliable and can be had for the same price, you can get 5 x 60Gb drives + Controller card for the price you can get 2 x 73Gb SCSI drives for.

5x IDE means you have to use 2U at least.... and also means a lot more heat, also means pay more to host that 2U... and still doesn't solve the multi tasking load, reliability level is lower much much lower than raid1, because the possibility of failure is higher

Now the price might not be different by say $50-$100 ? but the reliability drops.... it now depends on what you want to achieve? Speed or reliability or something in the middle.

Yes, the chances of failure are multiplied, so then if you are that worried about more than one drive failing at the same time, put 4 drives in the array, put one in as hot spare, you can afford for 2 drives to die that way and still have 3 left in the array. And lets be honest, 2U only costs about an extra $25 in most places to colo.

Your argument against IDE RAID5 isn't really all that valid as it applies equally to a SCSI RAID5 array as well, so it's not a problem caused by IDE that means the chances of failure increase.

Originally posted by KDAWebServices
Your argument against IDE RAID5 isn't really all that valid as it applies equally to a SCSI RAID5 array as well, so it's not a problem caused by IDE that means the chances of failure increase.

Sorry I wasn't talking about IDE RAID vs SCSI RAID, simply IDE vs SCSI, and on another topic, about various types of raids...

Your idea for IDE RAID5 is quite appealing and I might try it out :beer:

Sorry, no you didn't mention it, but I did and so did the original poster in another thread where this has been discussed, they are considering RAID1 with SCSI, not a single drive.

I agree that for a single drive I would rather go with SCSI, but when it comes to RAID I would go with IDE simply because it provides all the same benefits as using SCSI, as I showed in the other thread, when you get 4-5 drives in a RAID5 array, you would max out the PCI (64bit, 66mhz) bus before you noticed any difference with SCSI.

So your saying IDE with RAID 1 is better than SCSI with RAID 1?

LOL, everyone seems to be going off topic here, 'netsolutions' is asking about IDE vs SCSI only. It depends what it's being used for... IDE with be fine for serving static webpages, but if you need to process pages such as PHP and using a database like MySQL then SCSI will noticeably faster than the IDE.

It's much cheaper, yeah, but it's FASTER, so it depends on what kind of budget you have.

SCSI drives are faster 'cos they've got their own processor in them, so as such it doesn't use the main CPU.

We use SCSI drives only as we need the speed and we don't need to store much, we might be currently using about 30% or less of our 9.1GIG SCSI drive. If you're using it to store loads of MP3s or just large files then an IDE will be more economical for you.

... that's what I think anyway :).

Alan

Yes he did ask about IDE vs SCSI, but I was just putting things into the context he was asking, which happend to be with RAID in the setup they are considering. With RAID anyway the controller card handles I/O so the CPU doesn't have too, so the load placed on the CPU by IDE drives is negated.

Originally posted by muppie
Raid 5 makes things faster and more reliable, but will only tolerate one disk failing.
Raid 5 is only faster when it comes to reading. Writing is actually slower due to (among other things) the writing of the parity checking blocks.

This has always been a problem at my place of work. The hardware guys always configure our Oracle servers with Raid 5. Me, as the DBA has to fight to make them reconfigure single drives for rollback space, redo logs and anything else that gets written to heavily.

The final layouts are usually:
1) O/S on Raid 1
2) Temp, swap, rollback and redo on 4 seperate physical disks
3) A couple of Raid 5 arrays for the main tablespaces.

But as this is a web hosting forum and not a database forum... this is probably irrelevent :D

Yep it has to be said in comparisson to a single drive the write speed on RAID5 is poor, but web servers do far more reads then writes (I think I mentioned this in the other thread on webhostchat, I am getting a feeling of deja vu).

Picking your RAID card is a key factor as depending on the card the write performance can vary by factors of 4 or 5.

How is the write speed of RAID1 compared to RAID5 ?

Since we are on the topic - Does anyone run software RAID1 on IDE?

Originally posted by KDAWebServices
web servers do far more reads then writes

That completely depends on the nature of your site(s).
If the server is not running low on memory probably most of the files are in the cache (at least for static HTML) so writing log files and such is causing most of the disk traffic!

IDE for home computers and SCSI otherwise.... :-)
This is a rule.... I gues... ;)

Have a great day :)

regards
amar

Originally posted by ho247
SCSI drives are faster 'cos they've got their own processor in them, so as such it doesn't use the main CPU.


I start to doubt about this claim ... Copy a large files (say ... 100MB total) on your scsi drives to other scsi or to itself. On separate window, run 'top' and see why this process also increase the CPU usage.

cheers,
:beer:

Originally posted by Mr. Amazon


:eek:
Who told you that IDE is better than SCSI in any way at all (except maybe for price per megabyte)? Even if you're going to put one popular site or if you have a webhosting business and are going to put tons of customers on your server, then you will probably get at least a few hundred hits per minute. Do you really think an IDE disk can handle the pressure and is as good as SCSI when it comes to multitasking? The only time I would recommend IDE drive in a webserver would be if you were going to put a so so popular site on your server and you were looking for the cheapest solution, or maybe as a backup drive.

This is rubbish. We do BILLIONS of hits per day and our IDE drives are vastly under utilized. The reason for this is that disk caching on linux is so effecient that the disk barely gets hit. With ram prices so low there is no excuse for not having 2+ GIG per web server. Most websites are much smaller than 2G therefore you hardly need a hard drive at all.

First off, use HARDWARE Raid, NOT SOFTWARE Raid..
I've seen waaay to many times where software raid causes alot of problems and you end up using tape backups to restore the mess. And yes, I'm sure their are plenty of stories on how people use software raid and never have any problems, but I can account for many, many problems that I've seen and heard about.
Thats why most enterprise storage products rely on hardware raid. Waaay better..

As for IDE or SCSI, depends on a couple things.. Speed, Performance, and Cost. If you want the first two and don't care about the third, then go SCSI. If you do care about the third, then go IDE.

If you compare the cost of SCSI and 7200rpm IDE disks, there is still a big difference. And with the IDE disks getting faster and faster, its becoming more of a viable option. However when you start really stressing the drives, SCSI still has better performance.
Unless you have tons of hits, multiple downloading, video streams, etc.. I wouldn't see where SCSI would be better..

As for backup software, you could use anything from a standard tar file (for linux) to a professional solutions like Veritas.

I've worked alot with the veritas suite of products and would recommend NetBackup (if you have *nix and Windows) or Backup Exec if you are just running Windows. Veritas is well known, not too expensive, and very reliable. And no, I don't work for them or anything like that.. I've just used their products and like them far better than something like Arcserve, Legato, etc..

If you need more info about the Veritas products, let me know.. :D
Sorry so long..
laterz,
raqworld

Would IDE last long when running 24 hours webserver taking multiple hits every seconds ?

The performance difference between our servers that we used to buy which ran IDE, and our new servers which us SCSI, is very noticable. I wouldn't recommend skimping on any part of a server, especially the drives.

-Brendan

BOTH!
:D

Originally posted by jakis
Would IDE last long when running 24 hours webserver taking multiple hits every seconds ?

Several sites hosted on my 7200 rpm IDE got about 27 million pages this month (projected). That's about 11 hits per second ... The IDE is kinda working hard, but has lasted for the last 5 months. Perharps I should start baking up those data, in case if it fails :)

cheers,
:beer:

Originally posted by HRBrendan
The performance difference between our servers that we used to buy which ran IDE, and our new servers which us SCSI, is very noticable. I wouldn't recommend skimping on any part of a server, especially the drives.

-Brendan

Brendan, will you please tell us (or at least me :D ) in what way do you notice the different? Server Load? Access times to server web pages? I am considering to upgrade most of my servers to 10000 rpm SCSI drives myself. Hope you can shed a light whether my plan is worthed. Also, how about the 7200rpm SCSI? Will it still give much better performance than 7200 rpm IDE?

cheers,
:beer:

We've had no problems with IDE drives, one server has done a serious amount of hits this month - not sure how many but the access_log files are over 7Gb so far so it's a pretty large amount.

I'll also wager that putting 4 IDE drives in a RAID 5 array is faster and cheaper than an equivelant SCSI drive of the same size, and being as the IDE in RAID is fault tolerant it should in theory also be more reliable and stand a good chance of being faster too.

IDE drives are cheap and easily to replace, but if you are planning on have a semi dependable company I would at least pay the extra money and go with the SCSI.

For web hosting, I don't think a SCSI drive will give noticeable performance improvement over IDE, this is for identical drives of course, with the only difference is the interface type (IDE & SCSI).

- The network bandwidth needed for web hosting purposes is FAR below the hard drive bandwidth provided by both interfaces. A heavily loaded web server needs bandwidth like 1 Mbit/s, thats about 300 GB/month. A modern IDE/SCSI drive provides more than 50 Mbit/s bandwidth. Even if counting other I/O activities like database, or logging, it is still plenty of bandwidth.

- Most modern IDE drives now support non CPU bound data transfer (DMA) just like SCSI. So the only disk activity that requires CPU is the kernel's filesystem code. This applies to both SCSI and IDE.

However, apart from interface differences, the best drives out there are using SCSI. You can't easily find a 10000 RPM IDE drives.

This is like asking "which is better, apples or oranges", only worse.

Fact: Some of the high-end SCSI drives have higher rotational speeds, reducing rotational latency. Counter: They also have noise, heat, and vibration issues, and RAM is where most of your disk work should happen in a real O/S (both Linux and FreeBSD do good disk caching - well, I think FreeBSD does better, but let's keep it down to one religious question at a time).

Fact: The SCSI standard supports command tags, which lets the drive be given multiple operations to complete in whatever order it determines ideal. This can give better performance if you have a large percentage of seek operations (with RAM caching, this does not represent typical Web hosting activity). Counter: The newest IDE drives have this also, and it's starting to be supported in some drivers.

Fact: The SCSI standard has little or no bus contention to worry about. A single IDE chain can only command one drive at a time, so Master and Slave on a single chain will have reduced performance. Counter: All modern motherboards have two separate IDE chains, and with drives being so large these days, most Web hosting servers are fine with one or two drives.

Fact: SCSI adapters are expensive and extremely picky. They are sensitive to cable positioning, cable quality, signal interference, sunspots (only sort of joking here), etc. Some won't work occasionally with a drive formatted by another controller. There are many, many variations of SCSI, most of which are somewhat inter-compatible, except when the connectors are different sizes (then you get into $40 converters). 50 pin, 68 pin, 80 pin, single-ended, differential, LVD, HVD, SCSI-1, SCSI-2, SCSI-3, Fast/Wide/Ultra/Ultra2/Ultra3, etc. Only some of the possibilities are listed at http://www.ramelectronics.net/html/scsi_connecters.html Counter: IDE/ATA has had a reputation for being an imperfect standard, but it's simple enough that the basics almost always work. You can drop almost any IDE/ATA drive onto a chain by itself as Master, and it will work. You just have to decide on the setting to make the BIOS happy, and Unix barely cares about that once it gets booted.

Fact: SCSI drives are generally more expensive, even in cases where the components and specifications are the same. You're buying the host adapter, too, even if it's built-in.

Fact: You can run software or hardware RAID on either kind of drive. This gives you the standard advantages of each RAID variation, regardless of the underlying drive type. Software RAID might even be best, assuming you have a good driver. The hardware RAID is just one more expensive specialized device that can misbehave ;)

Anything else I have is personal perception or experience. We use IDE in shared servers exclusively because we've been burned by SCSI unreliability many, many times. SCSI bus hangs, adapters that don't find the drives sometimes when they boot, cables that look like WWF championship belts, drives that mysteriously stop working with no warning given, etc. An IDE drive may fail completely on occasion (we had one go Friday night, for example), but usually the errors are straightforward, and a swap is quick and easy. And yes, you can do software RAID-1 on them. We're working on doing that for all of our servers soon.

Exception: The IBM Deskstar 75GXP. Horrible drive, don't touch it.

Rule: We have at least six hundred IDE drives happily purring away, very reliable overall, most of them with very little actual work to do thanks to RAM.

Kevin

Originally posted by sigma
We use IDE in shared servers exclusively because we've been burned by SCSI unreliability many, many times. SCSI bus hangs, adapters that don't find the drives sometimes when they boot, cables that look like WWF championship belts, drives that mysteriously stop working with no warning given, etc. An IDE drive may fail completely on occasion (we had one go Friday night, for example), but usually the errors are straightforward, and a swap is quick and easy. And yes, you can do software RAID-1 on them. We're working on doing that for all of our servers soon.

Most of what you have written is well founded, but SCSI is not unreliable. I've not had a single problem with SCSI which was not related to wrong termination or another problem of the technician who built the machine.
And I avoid software raid, hardware is the way to go for performance and reliability.

Originally posted by Walter

Most of what you have written is well founded, but SCSI is not unreliable. I've not had a single problem with SCSI which was not related to wrong termination or another problem of the technician who built the machine.
And I avoid software raid, hardware is the way to go for performance and reliability.

With all due respect, those are both based on your personal experience and opinions. You not having problems with SCSI doesn't mean that SCSI isn't prone to problems in other situations for other people. If you haven't experienced sensitivity to cable length, positioning, adapter type, etc, you are fortunate. Please refer to this amusing Web page to see that I am not alone in feeling this way: http://www.uni-mainz.de/~neuffer/scsi/fun.html

Hardware RAID is simply a drive controller with a CPU that runs software which performs the RAID functions. It may use its own internal techniques which render the individual drives unusable if the controller fails. Using software RAID, you should have the source, and there's no extra hardware component that can fail. The CPU impact is minimal for most RAID configurations. I'm thinking of vinum as a particular example. http://www.vinumvm.org/

Kevin

Originally posted by Walter
Most of what you have written is well founded, but SCSI is not unreliable. I've not had a single problem with SCSI which was not related to wrong termination or another problem of the technician who built the machine.


In contrast, it's nearly impossible to hook up an IDE drive incorrectly, or to select the wrong adapter, or the wrong species of goat.

Kevin

Originally posted by sigma
those are both based on your personal experience and opinions. You not having problems with SCSI doesn't mean that SCSI isn't prone to problems in other situations for other people.

:cool: I haven't said there are no issues, but show me a interface without issues. And of course there are IDE issues, too.

Please refer to this amusing Web page to see that I am not alone in feeling this way: http://www.uni-mainz.de/~neuffer/scsi/fun.html

If you send me to the fun page, why not send me to the rest of the page, too: http://www.uni-mainz.de/~neuffer/scsi/why_scsi.html
:)

Hardware RAID is simply a drive controller with a CPU that runs software which performs the RAID functions.

Wow, never thought about that!

It may use its own internal techniques which render the individual drives unusable if the controller fails.

Same for software: if the software fails, the drive is unusable.

Originally posted by sigma
In contrast, it's nearly impossible to hook up an IDE drive incorrectly, or to select the wrong adapter, or the wrong species of goat.

Get real, man.
If you use only built in IDE adapters, ok, you can't select the wrong adapter, but as soon as you try 3rd pary controllers (e.g. for raid) you have the same "problems" as with SCSI. And don't tell me anything about cables and what could go wrong.

Originally posted by Walter

Get real, man.
If you use only built in IDE adapters, ok, you can't select the wrong adapter, but as soon as you try 3rd pary controllers (e.g. for raid) you have the same "problems" as with SCSI. And don't tell me anything about cables and what could go wrong.

OK, I'm not looking for a fight. Of course you use the built-in controllers, with software RAID. And there are only three cable varieties (old ATA, UDMA66, UDMA100) and you only put one drive on one cable.

The rest is just sniping, so forget it. Use whatever you like!

Kevin