Results 1 to 15 of 15
-
06-29-2009, 01:53 AM #1Newbie
- Join Date
- Jul 2007
- Posts
- 29
R1Soft on Steadfast.net --> No Minimum Level of Service
Heya all,
I guess this is an open ended question. I have R1Soft backup services with Steadfast.net as they have always had good service and great prices.
However 3 days ago I thought that I would run some maintenance on my backups and tried to log into the backup servers. When I tried to go to the control panel it said the system was down for maintenance... that was strange.
Checked my emails, no notification of maintenance time. Checked the news page at steadfast, no notice of maintenance. I logged a job and was told that the system had been down for roughly 3 days (when you add that together we are now at roughly 6 days as best I can estimate) that the system has been offline.
After many emails back and forth, I have found out they are running an fsck on the entire backup server and it has taken all this time to do and is still running.
I also voiced my concern that no notice whatsoever was provided of issues and I was told that I should have notifications set up when backups fail (which I do and have received no notice), but this still doesn't account for not notifying customers of maintenance to a server.
What I am wondering is:
- Should companies really be notifying their customers of maintenance to such critical systems proactivly (Singlehop does, I have my actual server with them, they notify me about anything if it may *remotely* affect my services, really happy with them BTW).
- Should they actually be running such a maintenance task which can take the system offline for so long without providing some other solution at least temporarily (they ignored that question when I sent it to them BTW)
- Am I being a jerk? Really I try to be nice and I have been polite in all my communication but seriously, I wanna know if my server has not been backed up for 3 days (well 6 now) LOL.
They have given me a free month next month, but that does not discount the fact that if my server was to go offline right now, I would loose almost a week of data... which is less than impressive for me.
What are your thoughts, is there some minimum level of service I should be able to expect?
Cheers
Stuart
(in the mean time, does anyone know where I can get some cheap backup space ~ 50GB)?
-
06-29-2009, 02:17 AM #2Web Hosting Master
- Join Date
- Mar 2003
- Location
- chicago
- Posts
- 1,781
i think a free month is as much as i would expect from any company in that situation.
-
06-29-2009, 02:31 AM #3Temporarily Suspended
- Join Date
- Feb 2004
- Location
- UK
- Posts
- 1,431
Hi
Maybe if the server was rebooted after a kernel update (or something), if it had been up for longer than a certain length of time, the system can force a fsck. It may be caught them by surprise... However I have to say that after 3 days I think they should have notified people...
Most customers would be blissfully unaware.
If you are concerned about data loss, I would look to backup your files manually to another backup location.
-
06-29-2009, 02:57 AM #4THE Web Hosting Master
- Join Date
- Jan 2003
- Location
- Chicago, IL
- Posts
- 6,957
This is a unique situation that did result from a scheduled maintenance, which is on our site and was sent out as our scheduled maintenances are. The primary part of the maintenance went fine, but it seems kernel updates done to a couple of the storage arrays at the same time caused some completely unexpected issues. It is extremely unusual that a kernel upgrade would end up causing data corruption, and with that part of the maintenance we were not expecting it, and thus did not really look for it. A couple hours later, we noted some odd behavior, took some of the storage arrays down to run fsck and the fsck seemed to actually increase data corruption. We then downgraded everything back to the way it was, ran fsck again, etc. and most things came back up. Right now, a vast majority of the customers are up, and most had had minimal downtime, considering the circumstances, but one of the arrays has been basically been running fsck forever now, though it is successfully repairing things.
Overall, this resulted from a scheduled maintenance, so it was sort of tied into that, and it seemed things would be resolved very quickly, as many things were. Then after that, with what was still being waited on to come up, we really had no ETA, it could come up in a couple minutes, though we really never expected it to take a couple days. Looking back though, I do certainly agree we should have handled this better and we have already had discussions internally about how to better handle this in the future.
So the basics:
1) This was the result of a scheduled maintenance, which was announced on the web site and via email, to those subscribed, as per our terms.
2) Yes, you should have been given more updates. We generally hate giving vague updates, but in some cases, it is simply necessary, as some information is certainly better than none. Also, if you asked for another option, you should have been offered to go to our standard FTP/rsync service, which was largely unaffected. Can you PM me the ticket number?
3) We will certainly honor our SLA, and any SLA requests submitted will be honored with a 100% service credit.
4) We have been working on additional storage options, including our own in-house SAN configurations across multiple locations. We are hoping to have additional redundancy available for our backup plans in the near future, which would make issues like this a thing of the past. The service we are offering as it is now though is specifically sold as a single storage location for backup data, offered at a lower price due to the evident single point of failure.Last edited by KarlZimmer; 06-29-2009 at 03:12 AM.
Karl Zimmerman - Founder & CEO of Steadfast
VMware Virtual Data Center Platform
karl @ steadfast.net - Sales/Support: 312-602-2689
Cloud Hosting, Managed Dedicated Servers, Chicago Colocation, and New Jersey Colocation
-
06-29-2009, 03:44 AM #5Newbie
- Join Date
- Jul 2007
- Posts
- 29
-
06-29-2009, 07:31 AM #6Aspiring Evangelist
- Join Date
- Dec 2006
- Posts
- 398
I know that I did not receive this email (and yes, spam folder was checked).
3) We will certainly honor our SLA, and any SLA requests submitted will be honored with a 100% service credit.
-
06-29-2009, 07:40 AM #7Newbie
- Join Date
- Jul 2007
- Posts
- 29
Well I am glad it was not just me that did not receive the email, (like you I checked all my spam folders etc too).
Stuart
-
06-29-2009, 07:42 AM #8Newbie
- Join Date
- Jul 2007
- Posts
- 29
System just came back online it would seem, 4 and a half days (give or take) between last backup to now, so I can now provide that value for clarification.
-
06-29-2009, 10:36 AM #9Junior Guru
- Join Date
- Feb 2007
- Location
- Chicago, IL
- Posts
- 205
-
06-29-2009, 11:30 AM #10THE Web Hosting Master
- Join Date
- Jan 2003
- Location
- Chicago, IL
- Posts
- 6,957
Well, here is the announcement: https://support.steadfast.net/index....&group=default It was certainly sent to everyone subscribed to receive such updates.
Can you PM me the ticket # of the billing ticket? All the billing tickets from Friday had been answered long ago.Karl Zimmerman - Founder & CEO of Steadfast
VMware Virtual Data Center Platform
karl @ steadfast.net - Sales/Support: 312-602-2689
Cloud Hosting, Managed Dedicated Servers, Chicago Colocation, and New Jersey Colocation
-
06-29-2009, 11:36 AM #11Aspiring Evangelist
- Join Date
- Dec 2006
- Posts
- 398
You may want to make people aware that they should subscribe to receive the updates, because I was completely unaware that such even existed. I would assume you'd just send out emails to those affected by maintenance period regardless of whether they are subscribed to a mailing list or not.
I sent an email to billing @ steadfast.net but never received a ticket number back. I'll just submit a new ticket via the ticket system directly.
-
06-29-2009, 12:24 PM #12THE Web Hosting Master
- Join Date
- Jan 2003
- Location
- Chicago, IL
- Posts
- 6,957
The announcements page is listed in the SLA and terms of service, where it is easy to subscribe to either an RSS feed or to email notifications. We do specifically link to it in our shared hosting and dedicated server welcome emails, but it seems it is not in the backup services welcome email, thus I have now added that. Thank you for bringing that to our attention.
Karl Zimmerman - Founder & CEO of Steadfast
VMware Virtual Data Center Platform
karl @ steadfast.net - Sales/Support: 312-602-2689
Cloud Hosting, Managed Dedicated Servers, Chicago Colocation, and New Jersey Colocation
-
06-29-2009, 01:25 PM #13Randy
- Join Date
- Aug 2006
- Location
- Ashburn VA, San Diego CA
- Posts
- 4,615
OT - Curious as to the size/type of array that needs 3 days to fsck?
Fast Serv Networks, LLC | AS29889 | DDOS Protected | Managed Cloud, Streaming, Dedicated Servers, Colo by-the-U
Since 2003 - Ashburn VA + San Diego CA Datacenters
-
06-29-2009, 02:02 PM #14THE Web Hosting Master
- Join Date
- Jan 2003
- Location
- Chicago, IL
- Posts
- 6,957
This one is only ~10TB, but the issue is with the data from rsyncs that have a lot of tiny files and then the hard linking between those files, etc. It has ended up then in a whole bunch of "multiply claimed blocks" which is taking a lot of time to process through.
Karl Zimmerman - Founder & CEO of Steadfast
VMware Virtual Data Center Platform
karl @ steadfast.net - Sales/Support: 312-602-2689
Cloud Hosting, Managed Dedicated Servers, Chicago Colocation, and New Jersey Colocation
-
06-30-2009, 12:15 PM #15Aspiring Evangelist
- Join Date
- Dec 2006
- Posts
- 398
This issue has been resolved for me. Thanks to Steadfast for their prompt assistance.
Similar Threads
-
R1Soft Backup Solution -Starting at $9.95 - Free Client License - Steadfast Networks
By KarlZimmer in forum Other Hosting OffersReplies: 4Last Post: 05-31-2009, 04:29 PM -
R1Soft Backup Solution - Starting at $9.95 - Free Client License - Steadfast Networks
By KarlZimmer in forum Other Hosting OffersReplies: 12Last Post: 01-26-2009, 12:35 AM -
SteadFast Bad Service
By avivronen in forum VPS HostingReplies: 34Last Post: 08-11-2008, 01:17 AM -
R1Soft CDP Backup Service / Hosting
By Kevin in forum Other Hosting OffersReplies: 3Last Post: 01-14-2008, 05:54 PM -
COMMENTS PLEASE: Steadfast Servers vs Coloquest Pure Level(3) bandwidth?
By helios3 in forum Dedicated ServerReplies: 9Last Post: 01-24-2005, 02:57 PM