Ok, first off I'd like to point out that I'm the Web Architect in my company, not the server technician or programmer. I'm not just trying to understand what is happening to my server and how I should vote/advise on a solution to our present problem.

This is our problem... as I understand it...

We have a RaQ hosted at 4WebSpace/Tera-Byte and when our programmer attempted to put a security update on the server it pretty much killed the server. (Turns out that 4WS runs its own DNS server so we weren't supposed to update the security on it - we screwed up, but how were we to know? - I used to read up on updates on the 4WS forum but it has been down for ages. But I digress...) We screwed it up by attempting that security upload and it caused our email service to be eliminated - but the Web sites were still working perfectly. We were also unable to administer the server through the control panel and had to contact tech support at Tera-Byte. This all happened Thursday afternoon at 3pm.

It seemingly took forever before TB's techs had a look at the box and in the meantime we lost all our web sites too. All our clients were without email and sites until we got another box set up by TB and we manually reset all our DNS info, site and email login, passwords, etc. Monday (that is 4 full days after we went down) TB's techs got our site info off of the old server and into a TAR file on the new one. Monday night my programmer manually moved each site from the TAR file and placed them into the client's server space, and when I checked early Tuesday morning (5th day) the sites were up and working perfectly BUT any emails sent to the clients were bounced back to the sender with this message:

> procmail: Quota exceeded while writing "/var/spool/mail/username"
> 550 5.0.0 <username@client.com>... Can't create output

The programmer assumed this was because he exceeded quotas when he was uploading all those sites at one time. (But he isn't sure and he can't get an answer out of TB's techs so we still don't know what really caused it and how to prevent it - If it sounds like I'm slamming TB's techs here, I don't mean to, but we are frustrated and our clients are screaming, and we don't know how to prevent this happening again because we still don't know what we did to cause it in the first place - but I digress again...)

Anyway, my programmer, in an attempt to fix the email, killed the server again - different server, different problem.

Now, when I go onto the admin screen I can see the list of users but there are no graphics whatsover (just the little red X in each box) and we can't administer anything. All the sites are down again and no email.

We called TB straight away and they said they would have a tech look at it within half an hour. It has been 24 hours and no word from them. We've called repeatedly and been told the ticket is still open, no update, no answers, no anything.

Ok, I just got a call from my programmer that the main tech guy will be looking at our problem within the hour. I hope it is true and he helps asap. I'm so sick of handling damage control with all our clients. As you can imagine, 6 days downtime is unacceptable so we have a LOT of explaining to do...

ANYWAY, assuming we get back on line again and our clients down run screaming to another hosting supplier and we get to stay in business... can anyone out there advise us on how to better manage this hosting side of our business???

How can we create a more redundant hosting solution? (without spending thousands a month on managed dedicated hosting).
Should we get a second server and mirror them? Should the servers be in 2 different networks (ex. one in TB & one in RackShack?). And if you back one server up to the other and one goes down, how do you get the DNS to point to the new server asap? I thought of moving to a raid server, but if the server goes down like this one did a second hard drive or even backed up files on tape or something isn't really going to help in that particular situation. Should we expect better tech support in the $99 a month server market? Any explanations as to how we screwed up the server and how to prevent it again? Is there a correct way to move the files from the TAR back into the server without screwing up the quotas???

Any help? Any advise? Anyone? :eek:

TIA
(and sorry for the long post)
Maxine

Well i think you just described every webhosts nightmare.

A simple advise is, spread your business over several servers, if one goes down like yours did, you'll only loose a part of your clients.
If you had a decent backup (not the one that comes with the cobalts), your sites could have been up in 24 hours or less, TB could have redirected your ip's to the new raq (assuming they would provide you with one fast) and everything would be back to normal from the outside.
Always be carefull with the updates Cobalt provides, try to find out what they're about before blindly installing them.

Just a few hints from someone that learned the hard way (and still is i'm afraid :( )