Results 1 to 33 of 33
  1. #1
    Join Date
    Feb 2004
    Location
    Fort Worth, TX
    Posts
    2,585

    ThePlanet (Another 10+ Hour Reload).. any questions?

    Welp unfortunately i'm back to an OS Reload. Mainly due to security reasons. We are going to move from RHE, to fbsd due to a pretty bad server compromise. Although we wanted to do this eventually anyways, but since this happend... wasn't a lot of time to move on it.

    Anyways, we called up to ThePlanet on 11/06 and said that we wanted an OS Reload with fbsd. They said no problem, to open an OS Reload ticket. My staff member called up TP again and explained we wanted an update version of fbsd, openssh, etc. The main reason was we've had several compromised servers just HOURS after they handed over the servers. Why you may ask? The software wasn't updated!

    So he calls them up and makes sure they add fbsd, cpanel/whm, and updated versions of openssh, blah blah blah. Once its up we hired a 3rd party company to secure the server. All is agreed to.

    I submit an OS Reload Ticket into the Orbit HelpDesk around 7-8:00a.m on 11/7. We have 2 hdds, so they said they have to charge to fix the second hdd to work with fbsd as it was using linux. Ok.. just give us a 80GB and we'll pay the $15.00 setup. Ok so that is seddled.

    So at 1:35p.m there is finally an update to the ticket, "Starting this reload now".

    Then goes 2:00, 3:00, 5:00, 8:00, 11:00, and now I finally responded at 11:48p.m

    "Ok so.. its into hour TEN now since we had a "Staring this reload now". I'd like to know what is going on. How does it possibly take TEN hours before we get a single update?????

    Am I going to get a refund of $25.00 for this?"

    Biggest concerns...

    #1 TP not releasing servers with updated software

    #2 Customer Support/Tech Support giving us the run around, and not responding to tickets

    #3 We called 10 times today, 6 times we could not get through... huh??

    #4 I opened up an abuse ticket with TP as one of the people who tried to install a rootkit on one of our servers was downloading the tools from a known hacking haven. <URL REMOVED> <--- ring a bell? They had exploits, viruses, hacking info, even a credit card generator. All TP did was instruct them to remove the credit card generator. This is obviously a haven for hackers.. but ya know.. the site is still up.

    I have already contacted Douglas from DN, and looks like i'll be spending our $8,000 in server cost per year with DN. As I, like a ton of ex-customers, have lost faith in this company. Its really sad to see, as TP was my favorite dedicated server provider for over a year and a half now. Unfortunately I wish I would of jumped ship a few months ago.


    P.S The kernel that was installed on this most recent server, is known for being one of the most compromiseable kernels for linux servers. Nice.....

    We've lost over 35 customers over this mess, which heck.. I don't blame them. If I was a customer of my service, i'd cancel also. And its really sad for us, as we have no control over this right now. But... looks like we'll go with another company, and i'm sure it will be better.

    I'll post back if in the morning, they still have yet released the server. I won't be too surprised if its not done then.

    MOD EDIT: url removed to avoid since its irrelative to the thread goal.
    Last edited by Bashar; 11-08-2005 at 02:54 AM.
    www.JGRoboMarketing.com / We Filter out the Bad Leads and Send you the Good ones!
    █ Office: (800) 959-0182 / Automated Lead Funnel Service

  2. #2
    Join Date
    May 2004
    Location
    Toronto, Canada
    Posts
    5,084
    Please remove that link.
    Andrι Allen | E: aallen(a)linovus.ca
    Linovus Holdings Inc
    Shared Hosting, Reseller Hosting, VPS, Dedicated Servers & Public Cloud | USA, Canada & UK - 24x7x365 Support

  3. #3
    Join Date
    Jun 2003
    Location
    Istanbul
    Posts
    661
    Agreed, the link have to be removed.

  4. #4
    Join Date
    Jan 2004
    Posts
    1,183
    Sorry but RHE is pretty secure you just need to know what you are doing.

    All my TP servers can with the latest up2date (kernel) installed.

    Yeat it have been ages since my last os reload but last os reload took about 1-3 hours if I remeber correctly.


    Any way good luck...! I hope you find what your looking for...

  5. #5
    Join Date
    Jun 2003
    Location
    Tampa FL
    Posts
    2,378
    was your server managed or unmanaged?
    Last edited by anon-e-mouse; 11-08-2005 at 10:29 AM.

  6. #6
    Join Date
    Nov 2002
    Location
    WebHostingTalk
    Posts
    8,878
    Quote Originally Posted by RealtorHost
    P.S The kernel that was installed on this most recent server, is known for being one of the most compromiseable kernels for linux servers. Nice.....

    We've lost over 35 customers over this mess, which heck.. I don't blame them. If I was a customer of my service, i'd cancel also. And its really sad for us, as we have no control over this right now. But... looks like we'll go with another company, and i'm sure it will be better.
    [/B]
    No, you do have control over it. When you box is released to you, it is your responsibility to ensure that it is up2date'ed and secured. This is not the responsibility of The Planet, unless you have a fully managed server.

    TAKE RESPONSIBILITY FOR YOUR BUSINESS.

    On the note of the O/S reloads, not much to say there... that has been an ongoing issue for quite some time. I am not looking forward to the day that I need an O/S reload, for fear that I will be in the same situation as you.

    Sirius
    I support the Human Rights Campaign!
    Moving to the Tampa, Florida area? Check out life in the suburbs in Trinity, Florida.

  7. #7
    I'm not happy with TP support too.

    Tickets take at least 1 day to be solved.

    The last ticket took 2 days and finally i did the job not TP technician (already asked them to use 1 hour admin at the beginning)

  8. #8
    Join Date
    Apr 2004
    Location
    Singapore
    Posts
    617
    Plan ahead. I had do quite a number of Reload at theplanet. And i had projected the time they usually reply with confirmation of start working on the OS reload.. yes.. it can be 10 hours..you just need to observe the time they had those mass reload sessions.

  9. #9
    Join Date
    Feb 2004
    Location
    Fort Worth, TX
    Posts
    2,585
    Ok... since the URL was removed (im sorry about that) we can all agree its not appropiate, and you'd think TP would take action.

    As I am writing this.. no other action has been taken. why???????

    TP says they release the server to you updated. When we received the server the kernel wasn't updated. Before we could even secure the server, it was compromised.

    And yes we do have a managed server with them....
    www.JGRoboMarketing.com / We Filter out the Bad Leads and Send you the Good ones!
    █ Office: (800) 959-0182 / Automated Lead Funnel Service

  10. #10
    Join Date
    Feb 2004
    Location
    Fort Worth, TX
    Posts
    2,585
    Quote Originally Posted by PremiumHost
    I'm not happy with TP support too.

    Tickets take at least 1 day to be solved.

    The last ticket took 2 days and finally i did the job not TP technician (already asked them to use 1 hour admin at the beginning)
    Same here, its taking days to get responces from TP. I'm extremly concerned right now about the future of this company. I can't wait until the end of the month, so I can move my customers to a stable data center, and start getting back to business.
    www.JGRoboMarketing.com / We Filter out the Bad Leads and Send you the Good ones!
    █ Office: (800) 959-0182 / Automated Lead Funnel Service

  11. #11
    Your lucky, One time a freind's company was hacked by some .pif's files. They shut them down for security reasons and took over 3+days to get an Secondary HD and OS reload. The Secondary HD was installed pretty fast, but the OS Reload has taken over 24hrs+. Everytime he called them up, They told "OS Reloads are not a guaranteed 24hrs installation, We try to get them done within 24hrs but sometimes they do take longer" and All they did was an automatic OS reload, why would they had that done that earlier is my freinds question.

    Sometimes, TP does mess up and most of them it is always when you need them the most. Isnt it great how that turns out?

    Ben

  12. #12
    Join Date
    Sep 2004
    Posts
    52
    It's been some time since I needed an OS reload there but they had been relatively quick unless I told them to wait till late afternoon or evevening. I have noticed some complaints of late mentioning slower ticket response times and there was recently some major changes in management there maybe related, maybe not.

  13. #13
    Join Date
    Mar 2003
    Location
    London, UK
    Posts
    170

    *

    If TP are putting unpatched servers online then you should take that up with their technical support. Managed or not, one should not be expected to rely on blind luck to get a server patched and rebooted before one of the thousands of automated scanners finds it and installs a backdoor.

    There really is no good argument for not performing a full system update before the machine receives connectivity to anything but the update server.

  14. #14
    Join Date
    Jul 2003
    Location
    Connecticut
    Posts
    3,038
    I thought getting support back on track was the reason they stopped sales?

  15. #15
    Quote Originally Posted by Defcon|Rich
    I thought getting support back on track was the reason they stopped sales?
    It doesn't happen overnight.

    Once you get your server back, yes, it is your responsibility to secure it yourself.

    Saying that it's their responsibility, is like saying a computer manufacturer like Dell or Gateway should be responsible for updating the system for every single microsoft update on every single brand new computer they make. Guess what? They are not responsible because of the time and effort that would have to go into updating every single brand new box.

    It's the same for TP. They'll probably update their reloading system maybe once every few months, but not al the time because an update might screw up their automated reload system, and they probably don't want it broken every other day.

    ~Garrett

  16. #16
    Our own experience is about 24+ hours the last time one of our clients had a harddisk failure at ThePlanet. Repeated calls, etc is useless. You would just be fed the standard line that reloads are done in the order it is received etc and that there is no guarantee of how long it takes.

    Well for sure they won't get any of our budget with that kind of procedures. To me, I fail to see how datacenters can treat OS reloads as something they can do on their own sweet time especially since in the case of a harddisk failure (or in this case, a bad root compromise), your server is effectively offline. We are even willing to pay more if that's needed to guarantee a SLA, but there isn't even such an option and to meet their promises of course. In fact, ev1 took about 10-12 hours (just 2 months ago) on one of the 2 last servers we have there when we had a harddisk failure and now we have zero servers there.
    ••• Like us on Facebook to qualify for discounts! •••
    ••• http://www.sprintserve.net •••
    ••• Offering: | Internap FCP Bandwidth! | Rebootless Kernel Updates! | Magento Optimized Hosting | Wordpress Hosting | •••
    ••• Services: | Managed Multiple Cores 64bit Servers | Server Management | •••

  17. #17
    Join Date
    Jul 2003
    Location
    Connecticut
    Posts
    3,038
    I don't know how other DC's do it but I would think if there are that many requests then allocating a few folks or hiring additional staff to just doing reload requests would be something to think about for TP as it appears to be something that gets asked for quite often.. I read a month or two ago where TP setup 500 servers for one company in 24 hours so according to that it seems like they are more then capable of getting this done.

    I am surprised that they tell you to 'get in line'. As you said a reload should be something that gets done asap. It's tough to run a business and be worried that if something goes wrong it might take a day or two to get sorted out. That's definatly not a good situation for anyone IMO.

  18. #18
    Join Date
    Aug 2005
    Location
    Oxford, England, UK
    Posts
    828
    Quote Originally Posted by sprintserve
    Our own experience is about 24+ hours the last time one of our clients had a harddisk failure at ThePlanet. Repeated calls, etc is useless. You would just be fed the standard line that reloads are done in the order it is received etc and that there is no guarantee of how long it takes.

    Well for sure they won't get any of our budget with that kind of procedures. To me, I fail to see how datacenters can treat OS reloads as something they can do on their own sweet time especially since in the case of a harddisk failure (or in this case, a bad root compromise), your server is effectively offline. We are even willing to pay more if that's needed to guarantee a SLA, but there isn't even such an option and to meet their promises of course. In fact, ev1 took about 10-12 hours (just 2 months ago) on one of the 2 last servers we have there when we had a harddisk failure and now we have zero servers there.

    Surely with a harddisk failure if it takes 24 hours thier SLA should compensate you, as hardware issues are expected to be resolved within X hours?

  19. #19
    Join Date
    Mar 2003
    Location
    London, UK
    Posts
    170
    Quote Originally Posted by GMoore7
    Saying that it's their responsibility, is like saying a computer manufacturer like Dell or Gateway should be responsible for updating the system for every single microsoft update on every single brand new computer they make. Guess what? They are not responsible because of the time and effort that would have to go into updating every single brand new box.
    I make no attempt to defend the distribution of unpatched systems to users who do not know how to administer them correctly. I have no evidence to suggest that Dell does ship systems that are vulnerable to commonly scanned exploits or do not have automatic updates enabled, so I cannot comment further on that.

    It's the same for TP. They'll probably update their reloading system maybe once every few months, but not al the time because an update might screw up their automated reload system, and they probably don't want it broken every other day.
    There are many differences in TP's situation. Major DC IP spaces are scanned with much greater regularity than any others, due to the greater availability of bandwidth. TP have direct, easy access to their systems up until the handover to the customer. The potential impact of a breach in one of TP's servers is far greater than the breach of a home user's computer.

    Perhaps the most important factor of all is good old-fashioned technical competence. Giving a server connectivity on your network when you know it to be vulnerable to commonly-scanned exploits is negligent and (evident by this thread) makes your company look incompetent. If my DC can't manage the simplest of procedures when the server is out of my control, why would I trust them to deal with similar situations when I have no access to the server? (following hardware failure, for example)

    The overheads of dealing with (potentially unknowingly) compromised systems on your network far outweighs the effort to either develop an unmanaged auto-update solution or to simply have somebody perform the step manually. And let's be honest here; on Linux it's as simple as running an update and reboot command in a one-off script. Perhaps I could be more understanding if this had been a Windows server.

    Edit: spelling

  20. #20
    Join Date
    Dec 2001
    Location
    127.0.0.1
    Posts
    3,640
    Quote Originally Posted by Alex Fernandez
    Surely with a harddisk failure if it takes 24 hours thier SLA should compensate you, as hardware issues are expected to be resolved within X hours?
    IIRC, it's 2 hours for the disk replacement (if your a TC customer), but I don't think they SLA the reload.

    YourHost, care to comment?

  21. #21
    Join Date
    Mar 2004
    Posts
    1,005
    Hopefuly YourHost will clear some things out
    Best Regards,
    Namesniper

  22. #22
    Join Date
    Aug 2005
    Location
    Oxford, England, UK
    Posts
    828
    Quote Originally Posted by mripguru
    IIRC, it's 2 hours for the disk replacement (if your a TC customer), but I don't think they SLA the reload.

    YourHost, care to comment?

    Surely SLA covers downtime, and a new disk is no use if the techs cant be arsed to install the OS onto it?

  23. #23
    Join Date
    Dec 2001
    Location
    127.0.0.1
    Posts
    3,640
    Quote Originally Posted by Alex Fernandez
    Surely SLA covers downtime, and a new disk is no use if the techs cant be arsed to install the OS onto it?
    I believe the SLA is only for the network, not the server.

  24. #24
    Join Date
    Oct 2005
    Posts
    62
    I too agree that a server should not be handed over until it has been updated against major security problems!

    And this is nothing like Dell sending unupdated windows.. When you're at home, you're behind your Router commonly, and nobody can connect to your computer directly.. You are also not running Telnet, FTP etc. services!
    And MOST of all.. if anything does happen, heck it's your responsibility because it's your computer and you can re-load the OS yourself.

    When the server is out of your hands it's much harder to secure remotely while making sure you don't lock yourself out accidently (such as a small misconfiguration in the firewall)

    I recently found out that my LT server came with a 'ltsvadmin' account which I was unaware of and apparently might have the same passwd on all LT servers.. this is yet another case of the host introducing security problems.. so now not only you have to secure it, you have to look for security holes they might have created before handing it to you.

    I believe the right thing to do would be for hosts to at least use unique passwords all around, and install firewall for everybody.. I had to install it myself.
    Then they can use firewall to limit connections only to your IP (if you can provide a static one) .. and when you're ready, you can open up your server yourself to the world.

  25. #25
    Join Date
    Aug 2005
    Location
    Oxford, England, UK
    Posts
    828
    There is a 99.5% Network Uptime Service Level Agreement (SLA), 2 Hour Hardware Replacement SLA, and a 99.9% Power SLA on this server
    The Guarantee
    99.9% uptime guarantee. We stand by our SLA. If we fail to deliver the level of service as promised, you get dollars off your bill. It's that easy.
    Taken from the SM website not so long ago (before the redesign, thanks Google).

    Funny, new site only has
    Each server solution in this series comes with a 99.5% network uptime SLA and a 99.9% power SLA
    - No mention of hardware anymore.

    Either way, they do/did state 2 hours hardware. Now, one would think that if its a primary hard drive they are replacing that is 2 hours for them to NOT only put new HDD in but to also install the OS again.

  26. #26
    Join Date
    Dec 2001
    Location
    127.0.0.1
    Posts
    3,640
    Quote Originally Posted by Alex Fernandez
    Either way, they do/did state 2 hours hardware. Now, one would think that if its a primary hard drive they are replacing that is 2 hours for them to NOT only put new HDD in but to also install the OS again.
    One would think so - but, I don't think TP thinks/cares.

  27. #27
    Join Date
    Aug 2005
    Location
    Oxford, England, UK
    Posts
    828
    Quote Originally Posted by mripguru
    One would think so - but, I don't think TP thinks/cares.

    Guess the lesson is: dont use TP/SM

  28. #28
    Join Date
    Dec 2001
    Location
    127.0.0.1
    Posts
    3,640
    Quote Originally Posted by Alex Fernandez
    Guess the lesson is: dont use TP/SM
    Yea, seems to be the lesson of the month .

  29. #29
    Join Date
    Jun 2003
    Location
    Istanbul
    Posts
    661
    Now and then while working with TP/SM beware of their TOS.
    Always keep your eyes at their TOS changes because you are due to their terms as their supporters forwarded me on every fault like that

    COMPANY SHALL NOT BE LIABLE FOR (i) ANY INDIRECT, INCIDENTAL, SPECIAL OR CONSEQUENTIAL DAMAGES, OR FOR ANY LOSS OF PROFITS OR LOSS OF REVENUE RESULTING FROM THE USE OF THE COMPANY’S SERVICES AND PRODUCTS BY CUSTOMER OR ANY THIRD PARTIES, OR (ii) ANY LOSS OF DATA RESULTING FROM DELAYS, NONDELIVERIES, MISDELIVERIES OR SERVICE INTERRUPTIONS COMPANY PROVIDES THE SERVICES AND PRODUCTS AS IS, WITHOUT WARRANTY OF ANY KIND, WHETHER EXPRESS OR IMPLIED COMPANY DISCLAIMS ALL IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE CUSTOMER SHALL BE SOLELY RESPONSIBLE FOR THE SELECTION, USE AND SUITABILITY OF THE SERVICES AND PRODUCTS AND COMPANY SHALL HAVE NO LIABILITY THEREFORE.
    http://www.servermatrix.com/legal/terms.html

    FYI: No longer TP/SM customer whom was paid at least $30k for the last 20months for monthly service fees.

  30. #30
    Join Date
    Aug 2005
    Location
    Oxford, England, UK
    Posts
    828
    Quote Originally Posted by dolay
    Now and then while working with TP/SM beware of their TOS.
    Always keep your eyes at their TOS changes because you are due to their terms as their supporters forwarded me on every fault like that



    http://www.servermatrix.com/legal/terms.html

    FYI: No longer TP/SM customer whom was paid at least $30k for the last 20months for monthly service fees.

    Actually, you as a customer dont need to keep an eye on the TOS changes, they have to notify you about any change and provide a reasonable amount of time for you to accept it / not accept and move away.

  31. #31
    Join Date
    Jun 2003
    Location
    Istanbul
    Posts
    661
    Quote Originally Posted by Alex Fernandez
    Actually, you as a customer dont need to keep an eye on the TOS changes, they have to notify you about any change and provide a reasonable amount of time for you to accept it / not accept and move away.
    Well you have to tell or ask that to the TP/SM responsibles , not to their customers.
    We have no control on their TOS and their changes.. anyways whatever they are doing now and then it is not my concern for more too.
    I only feel myself too stupid because i stay as their customer since that time and suggest you to read the TOS especially those caps and keep your eyes at their TOS if any changes appeared that you do not agreed

  32. #32
    Join Date
    Feb 2004
    Location
    Fort Worth, TX
    Posts
    2,585
    We never got an explanation why it took so long, just a very quick repsonce in a ticket basically emulationing, "Sorry for the delays, this os reload is done".

    My guess is it took so long as a person had to get on there and make sure everything was up to date, as we requested. I don't think their images are up to date. And this is why so many servers are getting compromised.

    I understand when people say if up to you to secure the servers. But I also understand how its not right when they hand you the server and its not secure to begin with. I mean c-mon a 2 hour compromise.. how does that happen? We didn't even have time to login and secure everything, before it was compromised.

    OpenSSH was 3 versions behind and the kernel was WAY exploitable. Its just *shaking my head* I dunno anymore...
    www.JGRoboMarketing.com / We Filter out the Bad Leads and Send you the Good ones!
    █ Office: (800) 959-0182 / Automated Lead Funnel Service

  33. #33
    I have no issues with an unpatched server as we secure them tightly on receiving it and do so on behalf of our clients. I do have issues with slow OS reloads. I just can't see why DCs will not invest in someone to do it if they have sufficient scale, or to have an escalation process (afterall, a total hardware failure means total downtime, vs someone who may just want to change his OS and is still up and running)

    It's fine though. We believe in voting with the wallet and we had done so.
    ••• Like us on Facebook to qualify for discounts! •••
    ••• http://www.sprintserve.net •••
    ••• Offering: | Internap FCP Bandwidth! | Rebootless Kernel Updates! | Magento Optimized Hosting | Wordpress Hosting | •••
    ••• Services: | Managed Multiple Cores 64bit Servers | Server Management | •••

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •