How many of you out there have a disaster plan of sorts?

I sure am glad that I have one. I refuse to use servers that do not have hardware raid 1. My requirement for this has proven valuable today, when one of my servers suffered hard drive failure.

One of my boxes started generating memory errors, so I rebooted it. It didn't come back up after reboot, so I called the datacenter, they hooked a monitor up, and saw that it was reporting a hardware failure on the raid array, and asked if it should continue booting off the good drive alone (without raid). The tech chose yes, and the box came back up.

Now, due to this failure, I have to go down to the datacenter tonight or tomorrow and rebuild the array with a new drive. And thanks to my hardware raid, I only have to do that, rather than telling my customers that all their data was lost due to hardware failure, and its going to take a day or 2 to get the system back up and running off of last weeks backup.

-----

How many of you are running hardware raid? What sorts of plans do you have in the case of hard drive failure?

I don't run raid. However, incase of hardware failure (such as the hard drive), while it may take a bit of time to install the new hardware, I would restore the offsite back up that was done earlier that week.

I don't consider raid very useful, sure it is a little quicker but I'd rather pay for better hardware (meaning better datacenter if you rent) and not have to deal with raid. I will be moving datacenters soon for this exact reason, better hardware to avoid unnecessary failure and much quicker replacement.

Originally posted by TheStarKillers
I don't run raid. However, incase of hardware failure (such as the hard drive), while it may take a bit of time to install the new hardware, I would restore the offsite back up that was done earlier that week.

I don't consider raid very useful, sure it is a little quicker but I'd rather pay for better hardware (meaning better datacenter if you rent) and not have to deal with raid. I will be moving datacenters soon for this exact reason, better hardware to avoid unnecessary failure and much quicker replacement.

Better hardware doesnt mean it wont fail. A hard drive that has been on, spinning at 7200rpm, 10,000rpm, or 15,000rpm for over a year will have a certain percent chance of failure. A $75 or $100 raid card is well worth it if you have a failure, and have any significant amount of downtime.

Sure, it doesn't mean it is not going to fail, the chance is less however. A raid card may be $100 if you colocate (which is mostly for the bigger hosts) but it is not if you rent. A failure in a good datacenter should not cause any significant downtime.

Originally posted by TheStarKillers
I don't run raid. However, incase of hardware failure (such as the hard drive), while it may take a bit of time to install the new hardware, I would restore the offsite back up that was done earlier that week.

I don't consider raid very useful, sure it is a little quicker but I'd rather pay for better hardware (meaning better datacenter if you rent) and not have to deal with raid. I will be moving datacenters soon for this exact reason, better hardware to avoid unnecessary failure and much quicker replacement.

Respectfully, restoring "...the offsite back up that was done earlier that week" doesn't cut the mustard for business or e-commerce sites. An online store can't afford to lose 3 or 4 days worth of customer and order information from their databases, nor will many of them have the expertise to recreate the database records manually from hard copy information (such as invoice emails).

RAID isn't the panacea that many companies present it as, but certainly a good RAID 1 or RAID 5 system is another piece of redundancy that lets you and your customers sleep at night. In this case amish_geek's clients were saved from potential data loss and got a swift recovery from a potentially catastrophic failure into the bargain.

Simon.

Yeap a hardware raid is quite necessary for me, I normally put a hardware raid 1 as well as a backup separate hard disk inside every server just in case. Raid helps during hard disk crash, the backup hard disk helps when we need to retrieve corrupted data etc

Hello,

Disaster recovery plans need to be designed carefully in order to minimize the discontinuity in service provision.

We for one use a combination of RAID, local backup, remote backup and offsite backups.

RAID:
Provides protection in case there is a hard-drive failure. Some people mistakenly consider raid to be a backup solution where in it is more of a replication solution where it maintains a mirror image of the primary hard-drive.

Local Backup:
Full daily backups (7 day archive) on a secondary hard-drive. Incase of data loss or corruption this backup can be used to restore files.

Remote Backup:
The backup file from the local backup gets copied every day to a Network Attached Storage in case both the RAID and local backups fail to materialize

Off-site Backup:
All backup files from remote backup storage (which pools backup files from all servers) are copied and taken off-site on weekly basis. This reduces the risk of data loss in case some thing happens to the site housing the servers.

Besides maintaining such a plan it is imperative that you do "restore rehearsals" on regular basis (whenever there is time to breath). These can be performed by duplicating a LIVE server on a dummy machine. This will insure that you will know exactly what needs to be done when disaster strikes.

Hope some of you will find this useful.

Thank you.

With regards,

Simon,

You make a good point, but the point depends on your target audience. It is not necessary for me because I do not currently host any large e-commerce websites with orders every hour. I will consider getting RAID later on.

Originally posted by TheStarKillers
Simon,

You make a good point, but the point depends on your target audience. It is not necessary for me because I do not currently host any large e-commerce websites with orders every hour. I will consider getting RAID later on.

Agreed that it depends on audience to a certain extent e.g. if your sole market is small personal websites then it's not as important (though it's still a good sleep at night feature). Your sig advertises small business hosting though, and you stated "I don't consider raid very useful", which seemed a contradiction and is why I brought the point up. It's immensely useful to anyone hosting sites where a few days of data loss is a big deal, and that would include most businesses. It might not even be order or customer data lost, it could be emails stored in webmail folders etc etc.

Simon.

if your using raid1, and one drive dies, the system SHOULD still boot fine. I dont know what type of raid controller you are using, but mine still boot and operate with a failed drive. Then i can pop out the dead one and pop in a good one, and the raid array will rebuild in the background without a shutdown or reboot.

AQHost,

The reason I said that is mostly because I've yet to see a hard drive fail. I know it happens but is pretty rare. The server I am currently using is new (since my data center ran out of them and I ordered just as they restocked). Once I move data centers soon, I will probably get RAID. Thank your for the information and helpful agreements.

Originally posted by ntwaddel
if your using raid1, and one drive dies, the system SHOULD still boot fine. I dont know what type of raid controller you are using, but mine still boot and operate with a failed drive. Then i can pop out the dead one and pop in a good one, and the raid array will rebuild in the background without a shutdown or reboot.

The system was using a Promise Fasttrack onboard IDE raid adapter. Upon booting, it gave a message "The array has failed, do you wish to continue booting on 1 drive."

It lets you boot and run just fine, it just gives you a message saying the array has failed, allowing you to shutdown and replace the drive. (I'm not using hot-swap drives).