
|
View Full Version : RackShack Moment of Truth For Me....
Shannon 04-27-2002, 02:01 AM First off, let me say this is NOT intended as flame.. really, it's not. I've been following RS for a while, since November or there abouts... I'm no Linux Newbie, and have administered everything from home cablemodem router/firewall boxes to 36 server IT departments for private business.. So I figured RS was a perfect fit, I wouldn't need any hand holding, knew what I was getting into, and figured I'd be okay. I've even recommended RS to fellow IT department co-workers who're looking for nice little 'hobby boxes' for this and that... so I'm no anti-RS fanatic.. I think they have a nice network, a nice location, and incredible pricing... my opinion has always been "If you know what your doing, and don't need the linux hand holding, you should be fine with them."... Maybe that opinion is going to come back to bite me in the rear end, I don't know yet... or maybe I'm just over-reacting and this is the way every dedicated solution is.. Maybe I'm just anxious because if this was a server directly under my control I'd be hung after an entire day of downtime.. ;)
Signed up for one of the plain Intel boxes late last month, during the special... everything has been great, prior to today I've had one reboot ticket that I needed to file. Nothing else.
But apparently my system has bad memory in it. Last night cPanel emailed me the 'warning' email with kernel messages from my log claiming all kinds of kernel errors (slab.c) allocating cache memory and deallocating it... anyway, after seeing those new messages, I check the server, and sure enough it's dead as a doornail.. I can ping it, but no web access, no ssh access, no ftp access, not even port 25 smtp access.... Hmmm.. not good.. So I file a Reboot TT and 20 minutes later it's closed stating "Rebooted and Server Up" (Which of course we all know means "I can ping it".. which is SOP pretty much anywhere).. I figured I'd atleast be able to ssh in, archive my data, ftp it off the box, and then file a ticket for "Possibly Bad RAM, please check"..
Well, it didn't happen that way.. nothing on the machine was responding still (except for pings) and I have no idea what's going on with the machine. At 11:51 I filed a ticket stating the RAM was probably bad, rebooting did not fix, nothing is responding, and included one of the kernel errors I had recieved from my logcheck the night before. That ticket was updated as follows:
11:51 - Opening of ticket.
12:12 - "Sent To DC" - RS
02:03 - "We will test the RAM with our Ramchecker, will involve downtime."
Now, obviously my first reaction was "Why not just swap out the possibly bad RAM with something good, and THEN test the memory? Depending on the RamChecker test-length I could be looking at hours down while it runs!".. but hey, who am I to tell someone else how to run their shop. So I sit back and wait a bit.. and a bit more.. and a bit more. Nothing. Not only am I not hearing anything, but I've been pinging the server fairly regularly, and it's never failed to respond, so it's NOT been powered off to have the Ram tested/swapped/whatever.
Around 5pm I pop into the chat room and am told that maybe I need to update the TT with the fact that nothing works (even thought it already said that at the beginning of the ticket).. but I decide that MAYBE, just maybe "Will involve downtime" translates into "Waiting for customer to okay downtime to run tests".. (Even though, since the first sentence in the TT by me states "Server is unfunctional!" I figured they'd assume it's okay to pull the box and fix it already)... So I update the TT : "Downtime for test is not an issue since it's already useless." (Blunt, but true.).. and I wait some more.
Fast forward to about midnight tonight their time (12 hours since ticket was submitted about the problem RAM).. I pop back into the support chat and and am told "they are up totheir neck in computer gear out there tonight...better them than me...and they're pretty efficient. They'll get it done."
I don't know what that means, maybe they're racing around trying to get servers up and running for tomorrows great "special" sale, who knows... But I mean, 12 hours to swap some RAM? (Or test some ram, or whatever it is their going to do?)... I've heard nothing actually regarding my ticket from anyone there since the 2:03 "Will test" message...
This is the first server I've placed live that isn't directly under my control where I can reach out and touch it on a regular basis... (Sadly I have no where near the bandwidth at my fingertips to do in-house shared hosting).. so maybe I'm just suffering from seperation anxity coupled with the stress of upsetting my clients...
What's everyone else's opinion? Am I over-reacting is this standard for the industry? Or is something amiss here? Yes, I know you get what you pay for, and I'm not expecting them to have dropped everything at 11:52 and replaced my ram by 11:53, but lets face it, if you can do a reboot in 20 minutes, a RAM swap should take 40, TOPS... ;) And even if it takes a few hours.. it'd be nice to keep the client informed of what's going on so they don't spend an entire day pulling their hair out and pondering how they're going to cope with a dead server. :(
What's everyone elses experience been?
2Grumpy 04-27-2002, 03:53 AM This sort of thing is exactly why my last 5 servers have went into a local colo where I pay $30 per 1U and $299 per megabit of bandwidth. I can be there in 20 minutes to solve problems. $240 for an APC Masterswitch from Ebay and $9 each for null modem cables so I can serial terminal 2 machines together and I'm set :)
And
I reciever almost as many complaints about my 3 servers at rackshack having slow network response as I do my one server at efreeservers, I've got 3 active servers at the colo (4 and 5 are brand new) and over 1000 web sites on them, and not a SINGLE "why is ftp so slow" trouble ticket or simliar for those accounts.
StarGate 04-27-2002, 05:10 AM ... with our first WHT special offer we had an issue like this too and handled it as follows.
As most of you know we just bought an Inc in New York some days ago with a DataCenter at NAC. Since then we are restructuring everything and you can imagine that everything is a little crazy, or at least has been in the first couple of days. This means that in the first days support did lagg but impoving fast day by day.
But now to the topic. Although it was the clients fault - he locked himself and all others out with a generic firewall - I offered to refund the client as I was aware that there is no excuse for support laggs, no matter how unmanged the server is.
Our policy is although having better pricing, better bandwith providers and more bandwith included then RS, to always keep the client informed via or helpdesk or other means, and of course to solve every issue in a timely manner which I can say we did achieve. In order to back the client up even further we have a 30-days money back guarantee on everything we sell - even on packages that are without setup.
So on the question of what happened to you is "standard issue in the industry" I can state a clear NO IT IS NOT.
Regards
Mxhub 04-27-2002, 07:38 AM "Why not just swap out the possibly bad RAM with something good, and THEN test the memory?
We had a bad swap memory experience on one of our RS server. Pretty bad experience during that period. We had a total about 20 tickets involve in "Reboot" . Yes. 20 more or less in just two weeks. If RS tech could seriously look into this problem and diagnose them quickly without having different tech checking the server, i guess our server would not go for so much downtime.
To those who wanted get new server from RS, just pray that you got a good memory batch or harddisk.
Shannon 04-27-2002, 05:00 PM My server came back up finally with new RAM less then an hour ago. I was in the support chat, and on the phone with customer service when Patrick said "Going to swap your RAM now, sorry for the delay" (or something similar).. within 15 minutes of that message, I was back up and running. I'm extremely happy with the service, ONCE the ball gets moving. ;)
How would I rate RS overall on the experience? Well, 27 hours to replace some RAM? I can't say I'd rate them real high... :( I know with the big sale and all going on this morning they were probably rather busy, but still... 27 hours for new RAM?
I'll probably be keeping my RS server for now.. but rest assured, I will be looking at procuring a box somewhere either: alot closer to me geographically... or somewhere a lot more serious about keeping their customers online.
So in closing, this still isn't a RS slamming thread, because I believe they do offer a nice service at a good price.. but I think that EVEN when you're only playing $99 a month, 27 hours is a bit too long to wait to replace some bad ram. ;)
Anyway, I'm off to try and get some rest, and then tomorrow I start shopping out rackmount servers to see what I'm up against here price wise. ;)
-Shannon
smacx 04-27-2002, 07:03 PM Originally posted by Dixiesys
a local colo where I pay $30 per 1U and $299 per megabit of bandwidth.
Could you let me know where exactily you get these prices?
BMurtagh 04-27-2002, 07:51 PM shannon is cool, :] i was having fun laughing at as she made fun of the horrible rs support. :)
where is RS getting all this BAD ram from , i just wanna know because they seem to have lots of it , and i dont ever want to accidentally buy some.
case
p.s. , this is in no way a bash against RS , but more of a bash against the ram they buy/sell
clocker1996 04-27-2002, 09:26 PM well it must be cheap hardware that they have then?
Shannon 04-28-2002, 01:32 AM shannon is cool, :] i was having fun laughing at as she made fun of the horrible rs support. She is actually a He, but after the first couple times letting people know that in RS chat, I just kinda figured "Ah heck, whatever works, just replace my ram!"... ;)
Although it was funny when I called RS tech support, she asked for my RS#, I gave it, she asked for my name, I gave it.. she asked one more time for the name, I repeated it, followed with "Yes, you are hearing correctly."
It's one of those things that comes up to bite me every so often, but after the first thousand telemarketer calls looking for "Ms. Shannon" I've learned to just enjoy the fact that I can go "Sorry, she died." and hang up. ;)
Walter 04-28-2002, 03:32 AM Originally posted by Shannon
So in closing, this still isn't a RS slamming thread, because I believe they do offer a nice service at a good price.. but I think that EVEN when you're only playing $99 a month, 27 hours is a bit too long to wait to replace some bad ram. ;)
What is a good price worth if you pay with 27 hours downtime?
web_res 04-28-2002, 01:12 PM After reading the responses of others and Patrick himself in the RS forums, I decided it would be best to contract Patrick himself rather then going through rs support.
The faulty memory that I suspected was an extra 512 mb of ram i had added several months ago. Well they decided to take it out (server was still functioning optimally) and go through to test it.
They did think it was faulty (never was told how they tested it) but apparently the tech in the area did not have access to the ram cabinet or something like that.
They put in the new ram the next day....
Now... I can see how mad I would get if my only stick of memory was faulty and would have to wait more then 18 hrs for the machine to come back up. Luckily for me it was the second stick i added later.
I think that they can change procedures in this case (assuming theyy don't have a large shortage of 512 mb ram) to make things a lot better for the customer.
Also umm in the process my hardware might have been swapped (very fast, didn't notice it till I saw the system date/time 3 months off). At first i didn't suspect the memory...
HRBrendan 04-29-2002, 01:19 AM Go with a dedicated server provider that offers a SLA on hardware replacement. Im going to bet having 6000 of the cheapest servers money can buy has something to do with the slow hardware replacement times.
-Brendan
2Grumpy 04-29-2002, 01:28 AM Nothing like the feeling of security you get seeing row after row of those little cheap white $23 cases knowing one of them is YOUR "server" eh? :)
http://www.atxcase.com/ I love the 1U case with 300W power supply, just ordered my 4th one last week :)
web_res 04-30-2002, 06:29 PM Originally posted by HRBrendan
Im going to bet having 6000 of the cheapest servers money can buy has something to do with the slow hardware replacement times.
-Brendan
It may seem like that but I think it is most likely due to shortage on highly trained personnel. In reality i don't think rs servers are as ultra cheap as some people claim...
dynamicnet 04-30-2002, 07:11 PM Greetings:
One of our managed service consultants found a statement on Rackshack's site that stated something to the effect of "hardware repairs are usually resolved in 24 hours."
Jake laughed stating, "usually?" "24 hours?"
Check out http://www.rackspace.com/infrastructure/sla/sla.php
Also see http://www.verio.com/company/policies/sla/dedserver/hardwaresla.cfm
Rackshack has a very good business model for what they provide.
However (imho), in order to provide other features and qualities, they would have to charge more.
Thank you.
Patrick-EV1 05-01-2002, 10:39 AM Originally posted by web_res
It may seem like that but I think it is most likely due to shortage on highly trained personnel. In reality i don't think rs servers are as ultra cheap as some people claim...
We're working on ways for every shift to be able to handle hardware problems quickly and effectively, including the recent purchase of a hardware RAM checker, which has reduced faulty memory issues by at least 95%, most bad sticks of RAM have been identified within 8 seconds on the checker, every stick of RAM we put in a box now goes through the full test. All the boxes in the recent sale went through hours of testing before each one was put up for sale. It's not a perfect system but we're modifying it everyday to help bring it closer. We're also working to train everyone on how to identify the beginning symptoms of a hardware issue, every reboot ticket we get, we search the account history to see if there's an unusually high pattern of reboots.
But sometimes because these arent managed servers, we rely on the user to give us enough information to warrant a hardware diagnostic, we hope the user wont sit on an issue such as the box crashing a day after they buy it when they havent done anything to it at all. ( No sites uploaded, etc.. ), or provide us with information such as "This is the last thing i was doing, and the server died" "The load average was xx and the server died for no reason" "I saw this in my error log, it looks to be a hardware issue, can you investigate it for me?" Once identified, we have no objections to replacing someone's hardware, giving you good working hardware also reduces our workload, so we're not fighting against anyone on this. :)
Just from my perspective. :)
richy 05-01-2002, 11:00 AM would you consider a burn in test, say running seti or some other similar program that would rag the hdd ram and cpu to death? i know its all extra time, but hire a couple of students from your local uni, i reckon they should be able to get through plenty of machines for minimal costs. i know for one id pay an extra 5 bucks setup to know the machine had been tested in this manner, whats 5 bucks for a piece of mind.
Patrick-EV1 05-01-2002, 11:43 AM What we ran on the other servers is a stress test, we shot the load average up to approx. 45-50 and let it run for a while.
cdream 05-02-2002, 06:36 PM Good host
clocker1996 05-02-2002, 06:45 PM Originally posted by cdream
Yea, i gave them a try and they are just a really bad hosting company. Their support is the worst I've ever seen in 8 years. I've had hosting with about 12 other hosts and by far Rackshack.net is the worst I've ever seen. Can't think of one good thing except their cheap prices.... Just remeber you get what you pay for. I had several cases where my server went down and would be down for days. Now I've had this also happen and other hosts but they either compensated with money or bent over backwards to get me up ASAP to include just putting me up on another box overnight. Trust me Rackshack doesn't give a d*m* about your business. You can call them until you blue in the face and all they have to say is "Sorry please put in a trouble ticket" there is no telling how much business I have lost because of them. Their cheap prices are not worth the business you will lose with them.
Thanks
Cdream
when was this..?
richy 05-02-2002, 07:45 PM umm 6000 cheap servers, i bet they got a hell of a discount from compaq but not enough to make em the cheapest servers, the white boxes seem fine on stability.
as for support, the boundries may be tight but the quality of what you get is very good. if your server was down for days what on earth were you doing? waiting for them to fix it? did you phone them and ask for a reboot, if it didnt work did you ask for a restore? did you use the trouble tickets and chat room? the occasional horror story happens but its generally the exception not the rule. if they wouldnt fix your server for you then maybe you should reassess the support package you get with a server next time you buy one. rackshack dont sell managed servers. in theory all they have to do is plug in a working machine mail you your ip and let ya to it. they arent here to manage it for ya. however their techs do take the time to assist people and are both polite and competant. mistakes are made, but i rarely see them repeated which assures me greatly.
afriq 05-03-2002, 07:35 AM Originally posted by HRBrendan
... Im going to bet having 6000 of the cheapest servers money can buy has something to do with the slow hardware replacement times.
-Brendan
I wouldn't call Compaq's or Cobalt RaQ's cheap servers ...
The problem lies with the number of support people, and their skill level (some are obviously more skilled than others).
bacid 05-03-2002, 06:23 PM Originally posted by afriq
The problem lies with the number of support people, and their skill level (some are obviously more skilled than others).
every ISP support is like this, unless its a very small company where support is done by the actual admins then you will always have a few morons in the group
janderk 05-04-2002, 06:45 PM Originally posted by bacid
every ISP support is like this, unless its a very small company where support is done by the actual admins then you will always have a few morons in the group Not entirely true.
Concerning rackshack where I have a Compaq server: I had a good and a bad experience. The bad one definitely needed more training. Starting with reading lessons.
However, I also have a few sites hosted at futurequest. And while they are growing larger they have the best support around. Never impolite, late, lazy, whatever. Companies like that still exist. I heard a rumor that they will offer dedicated servers in the future now that they own their own data center.
Jan Derk
bteeter 05-06-2002, 05:02 PM I've had nothing but good experiences with RS support. Call me lucky. :-)
The first experience was when I ordered my first box with them. It was essentially DOA. The hard disk had errors anytime I tried to save or do much of anything. Within hours, I had a new drive and was up and running.
More recently, I've had to request reboots on my machines there. All of which were done in less than 30 minutes, from the time the ticket was submitted to when I was back up and running. To me, thats pretty impressive.
Other dedicated server providers I work with take days and weeks sometimes to even respond to issues. I had a faulty hard disk in a server that took _3 Weeks_ to get replaced. Fortunaltely, it worked well enough to make it all that time, but I was not sleeping well during that time...
Its nice to have a provider that actually gives a s*** and can take care of problems fast. They've earned our business, and we'll be buying more in the future. :-)
Take care,
Brian
|