Results 51 to 73 of 73
Thread: SingleHop down! [merged]
-
12-19-2014, 07:34 AM #51Junior Guru Wannabe
- Join Date
- Jan 2014
- Posts
- 36
-
12-19-2014, 07:35 AM #52Web Hosting Master
- Join Date
- Nov 2001
- Location
- Chicago
- Posts
- 1,135
As you can imagine we have a lot of phone calls, emails, twitter and FB messages, and upset customers. We're doing our best, I fully recognize we could have communicated better here. What is most important at the moment is restoring service. We'll deal with everything else as soon as that is done including issuing a detailed RFO to impacted customers and bill of rights/SLA credits for the outage.
-
12-19-2014, 07:37 AM #53Web Hosting Master
- Join Date
- Nov 2001
- Location
- Chicago
- Posts
- 1,135
That is not correct. The OT details page is having some delays loading due to dependencies with the main SingleHop.com site, which we WILL fix first chance we have... however the OT main page has been updated since the issue started: http://ot.singlehop.com/
Our phones are slammed with callers. We are answering, but we are under very heavy load.
Facebook and Twitter ARE being updated, but direct messages were the first priority and there were a number of them.
I know you are frustrated and unhappy right now, we are too, please stand by and we'll get this resolved and share full details with all impacted customers.
-
12-19-2014, 07:40 AM #54WHT Addict
- Join Date
- Feb 2003
- Location
- hmm..
- Posts
- 174
-
12-19-2014, 07:46 AM #55WHT Addict
- Join Date
- Feb 2003
- Location
- hmm..
- Posts
- 174
http://www.downforeveryoneorjustme.com/singlehop.com
It's not just you! http://singlehop.com looks down from here.
-
12-19-2014, 07:47 AM #56New Member
- Join Date
- Dec 2014
- Posts
- 0
-
12-19-2014, 07:50 AM #57Web Hosting Master
- Join Date
- Nov 2001
- Location
- Chicago
- Posts
- 1,135
Everyone -- please stand by. We are working really hard on this and we will absolutely send a detailed REASON FOR OUTAGE explanation to everyone who was impacted as soon as we are done restoring service. In the meanwhile, what is most important, is we let the engineers do their thing.
-
12-19-2014, 07:57 AM #58WHT Addict
- Join Date
- Feb 2003
- Location
- hmm..
- Posts
- 174
Really? Your first update there was 39 minutes ago - HOURS after this outage started. No happy campers over on Twitter either.
https://twitter.com/SingleHop/status/545899556849057792
This is about communication - outages happen...(well they shouldn't like this)
Telling everyone what's going on after the fact is pretty much useless when an outage is extended.
-
12-19-2014, 07:58 AM #59Junior Guru Wannabe
- Join Date
- Jan 2014
- Posts
- 36
-
12-19-2014, 08:05 AM #60Premium Member
- Join Date
- Dec 2010
- Location
- Staffordshire, UK
- Posts
- 25
It's entirely possible to let your engineers continue working on this issue while your customer service team publish updates. As many others have commented, a full explanation is of little use afterwards. We all have clients waiting on the phone and submitting tickets/live chats wanting to know what's going on, and we currently have nothing to tell them.
The only people who actually know what is going on (I would hope) are you guys, SingleHop, so why not shed some light on the issue before we start losing customers?
If you only have one team of people who can either fix issues OR answer tickets and provide updates...then maybe it's time you started recruiting. No point having "transparency" as one of your main selling points if you never let us know what's going on during a downtime until after the event.
Also as one of the biggest cloud hosting companies in the US, I'd expect your website to handle a bit of additional traffic during a downtime, so there's absolutely no excuse for your status page not to be working.
Jay
-
12-19-2014, 08:29 AM #61Web Hosting Master
- Join Date
- Nov 2001
- Location
- Chicago
- Posts
- 1,135
Jay, everyone, I know and am first to admit that we could have communicated better here. We are going to send every customer who was impacted a detailed RFO (reason for outage) that explains exactly what happened, the exact time frame, and what we are doing to prevent it from happening again. That said, I make no excuses about our failure to communicate effectively here and we will be evaluating our process and procedure for that. Meanwhile I do want to report that service should be fully restored for all customers, and that is why we switched OT from Red to Yellow. We will absolutely credit anyone who was impacted. This is something that I take very seriously personally and I know the rest of the SH team does as well.
If you would like to talk to me personally about it just send me an email - dan -at- singlehop -dot- com.
Thanks,
Dan
-
12-19-2014, 08:35 AM #62Web Hosting Master
- Join Date
- Apr 2006
- Location
- Phoenix
- Posts
- 808
First and Foremost - most important, all services have been 100% restored for about 15-20 minutes. If your still experiencing issues, please contact us.
Second, and let me please be extremely clear on this. We are extremely disappointed internally with the way our communication was handled. Our engineers were so heads down fixing the problem that there was an unacceptable delay in getting our emergency social media team mobilized. We let our customers down, something that will not happen again. We pride ourselves on the level of transparency and availability to contact us. There are no excuses, only actions and we will be doing a full post mortem to better improve our internal process for communication. This will include better OT reporting and updates. Please stay tuned for updates.
During incidents like this, there is often times a number of possible root causes for the symptoms being displayed and it is rarely possible to pass on a specific cause while the networking team is diagnosing the problem. While I understand that one of the most frustrating part is not knowing specifics, you have my personal commitment that we will provide the most up to date, verified information as soon as we have it.
Now that service is restored, we can focus on the following important tasks:
1) A detailed RFO (Reason for outage) as well as steps that will be taken to prevent it ever happening again.
2) Emergency communication post-mortem and changes.
3) SLA credits for those affected.
Thank you for your understand and patience during this stressful time.
-
12-19-2014, 08:52 AM #63WHT Addict
- Join Date
- Dec 2012
- Location
- In Real World!!
- Posts
- 149
This is good seems that the machines are up at this moment. Let's see how it goes as they were up many time during past hours and went down.
Also the Leap Page is giving below error.
500 Internal Server Error - Sorry something went wrong, you will be redirected to your dashboard.
And OT Page is given warnings like below and taking a lot of time to load.
Warning: DOMDocument::load(http://status.singlehop.com/?feed=rss2&p=327) [domdocument.load]: failed to open stream: Connection timed out in /home/otsingle/public_html/status-details.php on line 115
Warning: DOMDocument::load() [domdocument.load]: I/O warning : failed to load external entity "http://status.singlehop.com/?feed=rss2&p=327" in /home/otsingle/public_html/status-details.php on line 115
Warning: DOMDocument::load(http://status.singlehop.com/?feed=rss2&p=326) [domdocument.load]: failed to open stream: HTTP request failed! in /home/otsingle/public_html/status-details.php on line 115
Warning: DOMDocument::load() [domdocument.load]: I/O warning : failed to load external entity "http://status.singlehop.com/?feed=rss2&p=326" in /home/otsingle/public_html/status-details.php on line 115
-
12-19-2014, 09:13 AM #64Web Hosting Master
- Join Date
- Nov 2001
- Location
- Chicago
- Posts
- 1,135
-
12-19-2014, 09:13 AM #65Web Hosting Master
- Join Date
- Apr 2006
- Location
- Phoenix
- Posts
- 808
All customer services have been restored, there is an error with our status page that we are working on. If your having any issues accessing your equipment please reach out to us.
-
12-19-2014, 10:26 AM #66WHT Addict
- Join Date
- Dec 2012
- Location
- In Real World!!
- Posts
- 149
Leap Portal is still giving error 500 however all the servers are working fine and that is a sign of relief. Hope to have Leap Working soon.
-
12-19-2014, 12:39 PM #67Web Hosting Master
- Join Date
- Jun 2001
- Location
- Princeton
- Posts
- 1,029
-
12-19-2014, 02:23 PM #68WHT Addict
- Join Date
- Feb 2003
- Location
- hmm..
- Posts
- 174
Leap3 is still down... geezz..
"500 Internal Server Error - Sorry something went wrong"
-
12-19-2014, 03:25 PM #69Web Hosting Master
- Join Date
- Apr 2006
- Location
- Phoenix
- Posts
- 808
Barko
All of our test accounts and many customers we have talked to are indicating that leap is 100% operational but it may be possible you have an outside case. We have found that for 99% of customers that the 500 error was caused by a session/cache issue. Can you please restart your browser and try again. If that doesn't work please open a ticket so we can fix it asap.
-
12-19-2014, 03:42 PM #70WHT Addict
- Join Date
- Feb 2003
- Location
- hmm..
- Posts
- 174
-
12-19-2014, 03:43 PM #71Web Hosting Master
- Join Date
- Apr 2006
- Location
- Phoenix
- Posts
- 808
Please don't wait! We want to get any issue resolved ASAP! Especially if your seeing drops in routing.
-
12-22-2014, 06:18 PM #72Junior Guru Wannabe
- Join Date
- Jan 2014
- Posts
- 36
---- SingleHop SLA ----
Network Uptime
Our public network will be online 100% of the time and you will be credited per our SLA for every minute it is down. Based on the amount of downtime, credit will vary from 5% to 100% of your invoice.
Uptime Credit
<100% 5%
<99% 10%
<98% 25%
<95% 50%
<90% 100%
According to their report the downtime was between 1:15am to 6:08am
http://uptime.is/99.3
LOL they will give us just 5% credit for downtime!
-
12-22-2014, 06:20 PM #73
CyberKeys what did you expected?
Rest of the time the service was working OK, wasn't it?
If you need some enormously high SLAs then you can always pay premium for that, depending on your requirements.
Similar Threads
-
SingleHop
By verzing in forum Dedicated ServerReplies: 14Last Post: 11-06-2012, 12:33 AM -
Singlehop spamming. Singlehop.com why?
By Everyday in forum Dedicated ServerReplies: 25Last Post: 10-24-2011, 11:39 PM -
SingleHop Down [ Merged ]
By CEO-90210 in forum Providers and Network Outages and UpdatesReplies: 53Last Post: 04-03-2009, 09:46 AM -
1Month Review Of Singlehop(Easr or West Singlehop is Best)
By rathin in forum Dedicated ServerReplies: 7Last Post: 08-20-2008, 01:27 PM -
Singlehop...
By -ahmed- in forum Dedicated ServerReplies: 4Last Post: 07-30-2008, 09:48 AM