Web Hosting Talk : Web Hosting Main Forums : Dedicated Server : External Power (Webreseller)

[mediasmyth]

Isn't the whole point of External Power for situations like this?

Redundant systems only work when they're routinely tested. It's obvious that WR doesn't (or didn't) test their Backup Power system prior to this outage. Kinda like a skydiver jumping without checking his reserve chute.

I have two other hosting companies that I use that are located in the hit zone (NC & MD) for Isabel. None of them skipped a beat.

If Rob learns anything from this fiasco -- it's that testing needs to be part of WR's standard operating procedures.

Their phone line goes unanswered.
Still no ETA on restoration.
My clients are screaming.

I'm losing faith with each minute this outage continues. Too bad, other than the usual glitches -- WR is (was) a good host.

- RK

[goodness0001]

WR contact me and said about 2 more hours.

[smartbackups]

RK, have you given any merit to the thought that the phones are digital and hence require power?

Redundant systems only work when they work. All the testing and load tests do not guarantee squat when you have to run on emergency power for long period of time a 15 minute load test once a week doesn't prove anything. I have designed these systems for banks and hospitals. Both of which are federally required by law to have working systems. Guess what they still fail. Just read some news from when the NE had the power failure a few weeks back.

WR has had a run of bad luck. I am just as frustrated as anyone I have 7 servers there. Things happen, but am I looking at alternatives, you bet I am. Will I leave, I seriously doubt it. Rob is a stand up guy and extremely honest.

Here's your sign.

[AussieHosts]

We're on the other side of the world and can only guess at the impact this situation has caused from the snippets of footage we see on TV. If anything at all was to remain fully operational during the worse of it, I'd be surprised.

[mediasmyth]

smartbackups...

Yes, I understand that digital phones require power. That's precisely why I still maintain a traditional line to my office. (to backup my digital line)

I also understand that things fail. Stuff breaks.

Backup power systems require routine testing -- not to just make sure they work -- but to check the extended loads they may be required to carry in the event of an outage.

Recovering from an equipment failure should also be part any Emergency Power plan. (it's the same reason you have a spare tire in your trunk).

WR is a good company -- and for the most part I've been pleased with their service. It's not my intention to rub their nose in this problem. But to point out to Rob that their recovery from this event is taking way too long -- and it's time to re-think their emergency procedures.

[kathystover]

just FYI, I know Rackshack had a power outage problem this summer and they had their huge generators there, turned on and didn't miss a beat. Rackshack literally had pictures on their website showing their capabilities to handle the problems.

[shdesign]

Please see my post at: http://www.webhostingtalk.com/showth...68#post1478568

Power has been restored and (hopefully) a lesson learned!

Scott

[AussieHosts]

Originally posted by kathystover
just FYI, I know Rackshack had a power outage problem this summer and they had their huge generators there, turned on and didn't miss a beat. Rackshack literally had pictures on their website showing their capabilities to handle the problems.

True, it was an outstanding effort at RS. But it was only at RS. At businesses across the road and down the street there were no problems (other than staff heading out for a look, no doubt).

We don't see weather in those extremes down here. I've been in and out of a few datacentres here, and I don't know how well many would cope. Don't wish to find out either.

Hearing from a Webreseller rep on here shortly after things turned sour was reassuring. The rest just took patience.

G

[Matt]

Originally posted by smartbackups
RK, have you given any merit to the thought that the phones are digital and hence require power?

Redundant systems only work when they work. All the testing and load tests do not guarantee squat when you have to run on emergency power for long period of time a 15 minute load test once a week doesn't prove anything. I have designed these systems for banks and hospitals. Both of which are federally required by law to have working systems. Guess what they still fail. Just read some news from when the NE had the power failure a few weeks back.

This is not directed at webreseller, but that statement causes me concern. "Redundant systems only work when they work". What? The purpose of redundancy is so that their are multiple levels of protection should something occur. If your redundancy does not work, it is obviously not redundant.

That being said, I must also say that if a company is only doing a 15 minute load test they deserve what's coming. The purpose of a redundancy test is to create an environment that challenges the system. Once switched, you should remain in that mode for a minimum of six hours. This gives you the confidence that should something catastrophic occur, your systems are prepared for the load they will be asked to handle. Simply firing up your generators is not enough (and way too many companies think it is).

Obviously timing is critical in the server industry as your sites must be up 24/7 365 days a year. Many of you are probably saying now that it is crazy to do that sort of test. It's only crazy if you do not truly believe your system is redundant and capable of handling it. If you do lose power and the switch over fails, you will be wishing you ran a more intense test.


Once again, this is not directed at webreseller, just my thoughts on the purpose and application of redundancy.

[mediasmyth]

Matt...

Well said.

Any backup system is only as good as it's "worst case" testing protocol.

[kathystover]

At businesses across the road and down the street there were no problems

Well, apparently webreseller techs had power since they have been posting on this board in the other thread so there must be power on around the DC unless their techs really aren't......nevermind.

[shdesign]

Originally posted by kathystover
Well, apparently webreseller techs had power since they have been posting on this board in the other thread so there must be power on around the DC unless their techs really aren't......nevermind.

Unless they were operating on a laptop with battery power???
And yes, Kathy, their techs ARE onsite. While I was on the phone with Rob, Jon (second shift tech) arrived for work and I even passed a message to him through Rob.

I don't think that you are comparing "apples to apples" with your example. Fact is, there WAS a Hurricane that went through the area and more than a couple of blocks were without power and some 100,000 people are probably STILL without power..........

[kathystover]

I wish you luck and hope your business is not too adversely affected.

[Mark_TVI]

I know ServInt has been without power for over 20 hours from the hurricane. They still have everything up and running.

There is no substitute for a well designed and properly implemented emergency back-up system....

[goodness0001]

I am unsure of how an outside trasistor failure could have affected internal backup power supplies??