Results 1 to 17 of 17
Hybrid View
-
05-06-2004, 12:58 PM #1Aspiring Evangelist
- Join Date
- Mar 2002
- Location
- London & Kent, UK
- Posts
- 394
HostEurope London power outage - any better providers?
HostEurope (UK) are sending out emails to announce downtime to the London data centre of up to 6 hours on a Tuesday in a couple of weeks time. This is allegedly for maintainance to the power system, although I've not been able to speak to anyone who actually knows why they need downtime in order to do this. Given that they now have multiple power providers, (they learnt that lesson after the DC had hours of downtime on a couple of occasions when they had only one provider), and one would hope that they have onsite power generators too, I'm at a loss for why they would need to shutdown all power in order to perform maintainance. Certainly it should be part of the design of the DC that they can shut down providers individually in order that work can be completed, and also to be able to reroute power around components in order that critical pieces can be replaced.
This all sounds like another case of incompetence and bad planning on their part, something that they've demonstrated an aptitude for on several occasions in the past (physically losing a hard drive for over a day when trying to move it from one machine to another, for example),. To cap it all, making a change in the middle of the week (sure 1am may be night time here, but it's not in australia or for many of the places where our visitors come from) is ludicous as well, and it seems that they cannot power on machines remotely, and have to visit each machine in turn to restore power. Mickey mouse outfit? Well we know that they're not quite that, but it sure sounds like it, and certainly not "World Class" as they claim on the telephone. Thankfully we have servers in multiple DC's so that we can survive downtime on the main server without losing web presence, but that's not really the point.
So I'm wondering whether this is this par for the course, or if there any DC's that are designed to allow power maintaince to be performed safely without having to pull out the plug on the customers too?
-
05-06-2004, 01:41 PM #2Web Hosting Master
- Join Date
- Sep 2002
- Location
- London/Edinburgh
- Posts
- 1,131
Hello.
If anyone has a copy of this email, i would be interested in reading it.
Posting it in this thread would be great
Regards,
Thomas Currie█ Europhase UK Limited - XEN/OpenVZ UK/US Virtual Private Servers | R1Soft Backups for Web Hosts
█ Custom R1Soft Backups | UK/US Web Hosting | UK/US Virtual Private Servers
█ 99.9% Uptime SLA | Non Overloaded Service | Guaranteed Ram / Resources
█ Company: SC345142 VAT: GB-947754180 Tel: 0870 4710 895
-
05-06-2004, 04:53 PM #3Aspiring Evangelist
- Join Date
- Mar 2002
- Location
- London & Kent, UK
- Posts
- 394
See below.
Checking the SLA, it interestingly doesn't make exceptions for so-called planned maintainance, and claims 99.9% up time. So if the network is down for more than the amount allowed in the sla (about 5 hours based on 0.1% of May) then there should be grounds for compensation.
Generally HE are reliable, and support staff are helpful, but their problems always seem to be in how the manage occasional issues, and lack of planning to minimise potential downtime is the big failing. The "we apologise for *any* inconvenience" is also a bit cheeky, as it's hard to see how this isn't going to be a major inconvenience for any business that's serious about it's internet presence.
--
Planned Maintenance - London Datacentre - 18th May 2004
Dear Valued Customer,
We would like to advise you that we will be performing essential power maintenance
work in the LONDON datacentre between 1 AM and 3AM on TUESDAY 18TH MAY. This work is
vital to ensure we continue to maintain the best possible datacentre environment and
the highest quality network to support your service.
We anticipate that downtime will last no longer than 90 minutes for the data centre
work, but would advise that some customers may experience unavoidable downtime of up
to 6 hours whilst all servers are restarted.
To ensure this process runs as smoothly as possible, it would be beneficial if
customers could power down their own servers prior to the 1 AM start. Host Europe
engineers will personally power on all servers immediately all necessary work is
completed, and will aim to keep downtime to an absolute minimum.
We apologise for any inconvenience caused by this work. Maintaining an efficient,
resilient infrastructure is part of our commitment to delivering a strong and
efficient service for all of our customers going forward.
Yours sincerely,
Host Europe
-
05-06-2004, 06:39 PM #4Web Hosting Master
- Join Date
- Aug 2002
- Location
- UK
- Posts
- 1,040
It's also odd that they'd give less than 2 weeks notice for something as huge as this... they must've known before now that they would have to take the power down, so why didn't they give people more advance notice?
essential power maintenance work
That is so incredibly vague that it could really mean anything at all. It's strange that they can't be more specific.
Something just seems a little fishyRobin Balen
Gyron Internet Ltd - http://gyron.net/
UK colocation, managed hosting and connectivity services with 100% uptime SLAs
-
05-06-2004, 09:02 PM #5Web Hosting Master
- Join Date
- Aug 2002
- Location
- Trouble will find me!
- Posts
- 1,470
Would this have anything to-do with the fact they have been bought out by PIPEX?
Maybe there power system was not built on a n+1 redundant system and PIPEX are making changes to stabilise these issues ...^^ IM WITH STUPID!! ^^
"The only way to overcome fear, is to challenge it head on"
"The quickest way to get over a woman, is to get under another"
-
05-06-2004, 11:30 PM #6Web Hosting Master
- Join Date
- Sep 2002
- Location
- London/Edinburgh
- Posts
- 1,131
Shazzy,
Thats what I was thinking...
6 hours is extremely major, wonder if we could fish for more details!
Regards,
Thomas Currie█ Europhase UK Limited - XEN/OpenVZ UK/US Virtual Private Servers | R1Soft Backups for Web Hosts
█ Custom R1Soft Backups | UK/US Web Hosting | UK/US Virtual Private Servers
█ 99.9% Uptime SLA | Non Overloaded Service | Guaranteed Ram / Resources
█ Company: SC345142 VAT: GB-947754180 Tel: 0870 4710 895
-
05-06-2004, 11:56 PM #7Junior Guru Wannabe
- Join Date
- Jul 2003
- Posts
- 40
6 hours is nothing compared to FDC's UNANNOUNCED downtime of more than 12 hours.
-
05-06-2004, 11:58 PM #8Newbie
- Join Date
- May 2004
- Posts
- 29
at least they give their customer announcement asap.
fdc is worst
-
05-07-2004, 12:11 AM #9Web Hosting Master
- Join Date
- Sep 2002
- Location
- London/Edinburgh
- Posts
- 1,131
Hello.
FDC and HostEurope are on completely levels...
Dont even try to compare them
Regards,
Thomas Currie█ Europhase UK Limited - XEN/OpenVZ UK/US Virtual Private Servers | R1Soft Backups for Web Hosts
█ Custom R1Soft Backups | UK/US Web Hosting | UK/US Virtual Private Servers
█ 99.9% Uptime SLA | Non Overloaded Service | Guaranteed Ram / Resources
█ Company: SC345142 VAT: GB-947754180 Tel: 0870 4710 895
-
05-07-2004, 03:20 AM #10Aspiring Evangelist
- Join Date
- Mar 2002
- Location
- London & Kent, UK
- Posts
- 394
"6 hours is extremely major, wonder if we could fish for more details!"
It is, although they say 90 minutes for the work (although they've always underestimated these figures in the past), and 6 hours to flick the switch on everyones machines. Aside from the fact that there simply shouldn't need to be any power outage ever, what troubles me is that they aren't able to bring up machines rather quicker than that. They only have 3000 machines in the DC, and whilst they might want to avoid possible network storms from all machines coming up around the same time, (although such things shouldn't happen anyway), it shouldn't take 6 hours to get things restarted.
-
05-07-2004, 07:50 AM #11Web Hosting Master
- Join Date
- Aug 2002
- Location
- Trouble will find me!
- Posts
- 1,470
Originally posted by GetYourHitS
6 hours is nothing compared to FDC's UNANNOUNCED downtime of more than 12 hours.
Could you imagine Rackspace announcing such maintenance?
Something is fishy and I do think we are all being told the truth.^^ IM WITH STUPID!! ^^
"The only way to overcome fear, is to challenge it head on"
"The quickest way to get over a woman, is to get under another"
-
05-07-2004, 07:57 AM #12Aspiring Evangelist
- Join Date
- Mar 2002
- Location
- London & Kent, UK
- Posts
- 394
Something is fishy and I do think we are all being told the truth.Real-time intrusion protection and error reporting for PHP sites ioncube24.com
Software protection for website owners and PHP developers ionCube PHP Encoder
-
05-12-2004, 06:23 AM #13Junior Guru Wannabe
- Join Date
- May 2004
- Location
- Manchester, UK
- Posts
- 30
I personally think it's safe to assume that Pipex wanted to sort out the data centre as quickly as possible so they can move onto other issues.
I imagine the team at Pipex won't stand for anything less than excellence in the data centre. It is surprising to hear that this kind of data centre design bungling can happen...
Dave
-
05-12-2004, 06:36 AM #14CISSP-ISSMP, CISA
- Join Date
- Aug 2002
- Location
- Seattle
- Posts
- 5,525
Co-locate a 16kVA Symmetra LX, give it some time to charge, and you should be able to weather the outage
-
05-12-2004, 06:40 AM #15Been around for too long...
- Join Date
- Aug 2002
- Location
- DC
- Posts
- 3,643
Originally posted by s.h.a.zz.y
Would this have anything to-do with the fact they have been bought out by PIPEX?
Maybe there power system was not built on a n+1 redundant system and PIPEX are making changes to stabilise these issues ...
- Matt
-
12-01-2004, 02:04 PM #16New Member
- Join Date
- Oct 2004
- Posts
- 1
HE aka Pipex
I personally think that Pipex are just beefing themselves up to sell on the data centre to someone else.
Therefore they don't care about their clients in the HE centre.
Power Outages, DOS attacks, really how can that happen if you're well equipped, I mean that is what they sell to you don't they.
We have been with them for more than 1.5 years now, and they've now cut themselves back to a 9-5pm data centre, come on how can you really offer a co-lo 9-5pm, so the lady on the phone says to me, can't you predict when your server will crash.... hmmm let me see what are the lottery numbers for tonights draw???
And if you want out, as they have now changed the rules, well there is no way out, leave if you want but no refund. So we're stuck.
I say if you're even thinking about co-lo or any type of hosting, stay well away from Pipex.
-
12-01-2004, 05:08 PM #17Aspiring Evangelist
- Join Date
- Mar 2002
- Location
- London & Kent, UK
- Posts
- 394
LOL they lose hard drives too
can't you predict when your server will crash
Worse things also happened for us after this, and serve as a further warning about the HE/Pipex outfit.
The summary is that the drive started to fail on one of our servers, and they were asked to replace it. We checked to ensure that they could mount the old drive read only in addition to the new drive, allowing data to be copied across data easily, and get going quicker than using backups. No problem they said, and estimated 30 minutes to do it. I was optimistic at this point.
After 24 hours of downtime, the status was that they had physically lost the hard drive that they were replacing, and the server was still down. This wasn't good, and any faith in them had long since evaporated by this point.
Sometime later, support claimed to have found the drive, but said that the partition table was damaged and no data could be found. This I knew to be rubbish, so they were asked to mount it anyway so that we could examine the raw drive. Whether they believed that they had found the correct drive, or whether they thought that we knew even less than them I don't know, but the drive they had put in was someone elses, from a Windows machine, and so not even someone elses from a Linux machine. The drive hadn't been reformatted, and we were quickly able to tell them whose drive they had just installed in our server, and that they were claiming to be ours. So much for data security on drives taken out of customers machines!
A while later, our original drive was finally found, althuogh by then of course the machine was mostly going from backups.
This whole experience was pretty shocking, and I've since learnt from our ex-account manager that they were really running on a shoestring, and only just keeping their heads above water. I can't confirm that, but certainly the experiences when there have been problems would tend to bear that out.
I'm not sure that things are better now, and when just a week or so ago there were major network problems in the London DC that affected connectivity to servers from within the UK, although thankfully not from elsewhere, it took many hours before the problems were correctly resolved.
We no longer use HE/Pipex exclusively, and will be phasing out servers with them in due course. As with anything, buyer beware.