Results 1 to 32 of 32
  1. #1
    Join Date
    Nov 2006
    Location
    College Station, TX
    Posts
    185

    FutureHosting leaving softlayer in Dallas?

    I got an email from Futurehosting saying that they were going to migrate all of their VPSes from the current DataCenter in Dallas to a new DC in Dallas.

    Does this mean that they're ditching Softlayer?

    Anyone know who the new datacenter is?

  2. #2
    Join Date
    Nov 2006
    Location
    Long Island, NY
    Posts
    287
    New DC Appears to be Colo4Dallas.

    Wonder why they're moving away from softlayer.
    Robert @ SmoothVPS
    www.SmoothVPS.com - Since 2008

  3. #3
    Join Date
    Nov 2006
    Location
    College Station, TX
    Posts
    185
    I think it's more along the lines of "why haven't they moved away from softlayer sooner" -- Softlayer's had crappy uptime, lots of unannounced maintenance periods, and some big hours-long outages due to power or cooling issues.

  4. #4
    Join Date
    May 2004
    Location
    Toronto, Canada
    Posts
    5,083
    Moving from a dedicated provider like Softlayer to a colo provider like colo4dallas generally means moving to their own hardware.

    I am of course just speculating but it fits for me.
    André Allen | E: aallen(a)linovus.ca
    Linovus Holdings Inc
    Shared Hosting, Reseller Hosting, VPS, Dedicated Servers & Public Cloud | USA, Canada & UK - 24x7x365 Support

  5. #5
    Join Date
    Jun 2005
    Location
    European Union
    Posts
    387
    Good job Vik

  6. #6
    Join Date
    Jan 2006
    Posts
    1,915
    Or they are going thru a provider already there. Only Vik can confirm it. There are some there which can build servers, etc and do what Softlayer does on a smaller scale. Tailormadeservers for example..

    -Jay

  7. #7
    Join Date
    Jun 2003
    Location
    Tampa FL
    Posts
    2,378
    Quote Originally Posted by Coolraul View Post
    Moving from a dedicated provider like Softlayer to a colo provider like colo4dallas generally means moving to their own hardware.

    I am of course just speculating but it fits for me.
    I tend to agree with you there.
    Ceridius Networks Sales
    Email/MSN [email protected]
    Ceridius Networks - Reseller of Hivelocity Hosting
    Network Speed Test

  8. #8
    Join Date
    Nov 2006
    Location
    Long Island, NY
    Posts
    287
    It appears to be TMS According to the test ip showed on their Dallas information page
    Robert @ SmoothVPS
    www.SmoothVPS.com - Since 2008

  9. #9
    Join Date
    Jan 2008
    Posts
    519
    I have already migrated over to the new server, and based on my new IP's:

    Tailor Made Servers TAILORMADESERVERS

    On another note - I again want to say what an excellent job that group is doing. They communicated their intentions, laid out my options for migration, and executed.

  10. #10
    Join Date
    Sep 2002
    Location
    Top Secret
    Posts
    11,686
    Quote Originally Posted by karlkatzke View Post
    I think it's more along the lines of "why haven't they moved away from softlayer sooner" -- Softlayer's had crappy uptime, lots of unannounced maintenance periods, and some big hours-long outages due to power or cooling issues.
    Wow, such a resentful and unfactual statement there.

    Number of servers monitored in Softlayer: 7 , including my own
    Number of expected outages from softlayer in the past 6 months: 3
    Number of Unexpected outages from softlayer in the past 6 months? 0

    Softlayer has had a repeated excellent performance as far as network is concerned. Of course that's only speaking for Dallas, but Seattle is too new, and if you're using that network, you really need to reconsider options, given the slowness of it.

    In the long and short, Softlayer is HARDLY going 'downhill'. Their peformance is flat on with any major level network, even more stable . Downtimes happen, and if you're going to get petty over 5 minutes of downtime, or a router reboot, then you've got way too much time on your hands.

    For what they charge (even above it), softlayer is great. Please get your facts straight before posting misinformation in the future.
    WHMCS Guru - WHMCS addons, management, support and more.
    WHMCS Notifications Extended - Add slack, hipchat, SMS, pushover to WHMCS !!
    Always looking for Linux, WHMCS, Support Desk work. PM for details

  11. #11
    Join Date
    Jan 2008
    Posts
    519
    Quote Originally Posted by linux-tech View Post
    Wow, such a resentful and unfactual statement there.

    Number of servers monitored in Softlayer: 7 , including my own
    Number of expected outages from softlayer in the past 6 months: 3
    Number of Unexpected outages from softlayer in the past 6 months? 0

    Softlayer has had a repeated excellent performance as far as network is concerned. Of course that's only speaking for Dallas, but Seattle is too new, and if you're using that network, you really need to reconsider options, given the slowness of it.

    In the long and short, Softlayer is HARDLY going 'downhill'. Their peformance is flat on with any major level network, even more stable . Downtimes happen, and if you're going to get petty over 5 minutes of downtime, or a router reboot, then you've got way too much time on your hands.

    For what they charge (even above it), softlayer is great. Please get your facts straight before posting misinformation in the future.
    I am not going to post the e-mail from FutureHosting (they can do that if they choose) but in short, there have been several power failures and a Storage Area Network (SAN)failure. That is unacceptable to them, and hence the move. What some customers are saying, is that it should have be done earlier. There is no factual misinformation taking place.

    Nobody is saying that Softlayer services have diminished. It just appears that in a particular setting, there were issues.

  12. #12
    Join Date
    Sep 2002
    Location
    Top Secret
    Posts
    11,686
    Redmeds:
    Please actually read before you post, it helps one not look foolish.

    I wasn't addressing your post. I was addressing specifically the post that I quoted, and yes, he made some pretty huge mis-statements that needed to be corrected.
    Last edited by whmcsguru; 04-20-2008 at 12:30 PM.
    WHMCS Guru - WHMCS addons, management, support and more.
    WHMCS Notifications Extended - Add slack, hipchat, SMS, pushover to WHMCS !!
    Always looking for Linux, WHMCS, Support Desk work. PM for details

  13. #13
    Join Date
    Jan 2008
    Posts
    519
    Quote Originally Posted by linux-tech View Post
    Redmeds:
    Please actually read before you post, it helps one not look foolish.

    I wasn't addressing your post. I was addressing specifically the post that I quoted, and yes, he made some pretty huge mis-statements that needed to be corrected.
    I can read just fine.

    You made a statement that I did not agree with, as you commented on a statement by him that you did not agree with. This is a forum, where people comment.

    So, again I don't see the mis-statements.

    In the future, I don't need to be told how to view or post in forums. That is really arrogant on your part.

  14. #14
    Join Date
    Jan 2006
    Location
    Jersey
    Posts
    2,965
    Hmmm...I wondering why they are moving? May be they want to take advantage of Internap FCP
    Email: info ///at/// honelive.com

  15. #15
    Join Date
    Sep 2002
    Location
    Top Secret
    Posts
    11,686
    In the future, I don't need to be told how to view or post in forums. That is really arrogant on your part.
    Then post truths, not misconceptions, such as:

    Nobody is saying that Softlayer services have diminished.
    This is a misstatement, based on not reading posts. The post I referred to stated quite clearly that they HAD diminished.

    I don't care whether you "agree" with the argument or not, that is irrelevant. You did not thoroughly read the post, or thread, and made a statement that was based off of something that was incorrect.

    It's not being 'arrogant', it's being right. When blatant mis statements such as "Their service has gone to crap" or "their service is crap" are made, or even "they've had hours long outages" (something that is a flat out lie), you'd better believe that they're going to be corrected, from a professional perspective, not a childish one.

    Again, read statements before you make responses. Save yourself from looking foolish. Don't get into arguments where you know very little about the facts and the statements.

    I've been using, recommending and monitoring Softlayer for going on 3 years now, and a blanket statement such as was quoted in my response is INCORRECT and a flat out lie. Are there 'special' circumstances? Sure, there are always special circumstances, but those are unique and a strong minority.
    WHMCS Guru - WHMCS addons, management, support and more.
    WHMCS Notifications Extended - Add slack, hipchat, SMS, pushover to WHMCS !!
    Always looking for Linux, WHMCS, Support Desk work. PM for details

  16. #16
    Join Date
    Jul 2006
    Posts
    285
    The economics of running VPS servers on rented high-end hardware don't work out very well. It's hard to make money off of a server when you're paying $900 month for it. To keep your pricing competitive, you have to have 20 midrange VPS plans on such a high-end server just to break even. With colocation, you pay a lot up front for the hardware, but it pays off after 3-4 months. Then your profit is much higher than "very little" on each hardware node.
    Last edited by Iwannasite; 04-20-2008 at 02:07 PM.

  17. #17
    Join Date
    Jan 2006
    Posts
    1,915
    Quote Originally Posted by Iwannasite View Post
    The economics of running VPS servers on rented high-end hardware don't work out very well. It's hard to make money off of a server when you're paying $900 month for it. To keep your pricing competitive, you have to have 20 midrange VPS plans on such a high-end server just to break even. With colocation, you pay a lot up front for the hardware, but it pays off after 3-4 months. Then your profit is much higher than "very little" on each hardware node.
    Yes, that is correct. Now the question is, what is FH doing buying their own servers or using TMS rented servers? Buying servers takes more capital, etc but in the long run it allows you to manage/build out your own infrastructure much better. I don't see Colocation on TMS's website.

    -Jay

  18. #18
    Join Date
    Jan 2008
    Posts
    519
    Quote Originally Posted by linux-tech View Post
    Then post truths, not misconceptions, such as:


    This is a misstatement, based on not reading posts. The post I referred to stated quite clearly that they HAD diminished.

    I don't care whether you "agree" with the argument or not, that is irrelevant. You did not thoroughly read the post, or thread, and made a statement that was based off of something that was incorrect.

    It's not being 'arrogant', it's being right. When blatant mis statements such as "Their service has gone to crap" or "their service is crap" are made, or even "they've had hours long outages" (something that is a flat out lie), you'd better believe that they're going to be corrected, from a professional perspective, not a childish one.

    Again, read statements before you make responses. Save yourself from looking foolish. Don't get into arguments where you know very little about the facts and the statements.

    I've been using, recommending and monitoring Softlayer for going on 3 years now, and a blanket statement such as was quoted in my response is INCORRECT and a flat out lie. Are there 'special' circumstances? Sure, there are always special circumstances, but those are unique and a strong minority.
    The topic is "FutureHosting leaving softlayer in Dallas?". So, with that being said, we are discussing specifically Dallas and it's relationship with Softlayer.

    So, if you know why FutureHosting is leaving Softlayer or terminating their agreement, maybe you should shed a little of your professional knowledge into the discussion.

    Nobody said their service has down hill as you originally responded. What the person said was, that based on the problems that this particular "special circumstance" as you put it, he could not believe they did not leave sooner.

    He said the uptime had been crappy. That does not mean that Softlayer's service has gone to crap. What that means, is there are issues.

    subject now closed for me.

  19. #19
    Join Date
    Nov 2006
    Location
    College Station, TX
    Posts
    185
    Linux-Tech --

    Go ahead and believe what you want to. First off, Softlayer has had power problems specifically on the rows of racks that Futurehost is in. The SAN failure was something specific to FutureHost, and caused me some significant downtime as I was one of the nodes affected. I don't keep stats, unfortunately, but I'm sure someone at FutureHost does. All of the failures have specifically been an issue with the way that Softlayer maintains the area that Futurehost's machines are in. Don't assume that if you haven't had problems that means that someone else hasn't.

    I am VERY impressed with the way that FutureHost has been handling this, but I wish I had been migrated already since I'm going to upgrade to CentOS 5 as part of the process and move all my content manually.

  20. #20
    Join Date
    Sep 2002
    Location
    Top Secret
    Posts
    11,686
    And don't assume that just because you have that everyone else is. You're making general statements such as
    Softlayer's had crappy uptime, lots of unannounced maintenance periods, and some big hours-long outages due to power or cooling issues.
    that are frankly unfactual, unreliable, and lies. You don't like your uptime, that's fine, and that's more than enough of a reason to leave softlayer, but that is not reason for you to make a blanket statement such as you did, especially one that misconstrues the facts.

    The facts:
    Overall, softlayer has had excellent uptime for the past 2-3 years, moreso than any company in it's class.

    YOUR HARDWARE has nothing to do with softlayer's uptime, and everything to do with your hardware.

    Just because you've seen crappy hardware doesn't mean that "Softlayer's had crappy uptime", it means you've got pathetic hardware, and you need to stop trying to use the solution you're trying to use, because there's something wrong.

    Now, I'll be the first to admit Softlayer's got problems (unforseen, unrealistic account changes forced on clients anyone?), but the network itself, and the hardware (for the most part) is astounding. If you can't get your hardware problems resolved, then you need to fire whomever is contacting softlayer, because I've had absolute success getting them to fix poor hardware, in no time flat.

    When you come out and blatantly lie like you have about the performance of a DC that is top notch and above the rest in both performance and pricing, you'd better expect that to be torn apart and countered, every way possible. It's not about me "believing what I want", it's about me posting factual statements , correcting your blatant lies.
    WHMCS Guru - WHMCS addons, management, support and more.
    WHMCS Notifications Extended - Add slack, hipchat, SMS, pushover to WHMCS !!
    Always looking for Linux, WHMCS, Support Desk work. PM for details

  21. #21
    Join Date
    Dec 2004
    Location
    New York, NY
    Posts
    10,574
    Quote Originally Posted by Anantha View Post
    Hmmm...I wondering why they are moving? May be they want to take advantage of Internap FCP
    SoftLayer has been using FCP for quite some time too, and whether it is actually beneficial to the customer is debatable and dependent on how it is configured--but that's just my opinion of course.

  22. #22
    Join Date
    Nov 2006
    Location
    College Station, TX
    Posts
    185
    Linuxtech -

    YOUR HARDWARE has nothing to do with softlayer's uptime, and everything to do with your hardware.
    It's not *MY* hardware. It's hardware leased, as far as I know, FROM SOFTLAYER for their client, futurehosting.biz, who hosts a VPS for me. As far as I know, all of the downtime with the exception of the SAN outage were due to infrastructure failures. The four+ hour one (which, if you look back in time in the outage reports forum on this very site, you can see) was due to a failure in the breaker panel that all power must route through to reach the racks that Softlayer maintained for futurehosting. There was a chiller outage that was several hours long that was specific to one room, which also affected my server as the node had to be shut down. There was a router outage that went a bit longer than planned.

    that are frankly unfactual, unreliable, and lies. You don't like your uptime, that's fine, and that's more than enough of a reason to leave softlayer, but that is not reason for you to make a blanket statement such as you did, especially one that misconstrues the facts.
    Facts. You keep using this word. I do not think it means what you think it means.

    To be pedantic: My opinion is that my experience has been downright crummy with my server's dom0 hosted at SoftLayer, and the fact is it's been downright crummy for so many people that a major customer of Softlayer's has just decided to migrate away from Softlayer at such a rapid rate that they're offering bonus packages to the first X number of people who get migrated.

    This speaks volumes more than your 'facts' -- which are nothing more than your opinions.

  23. #23
    Join Date
    Sep 2002
    Location
    Top Secret
    Posts
    11,686
    It's not *MY* hardware.
    Ah, but it is. As long as YOU lease things, YOU pay the bills, it is YOUR HARDWARE. The only thing that SoftLayer is required to do is replace it, IF it is bad. Guess what? They do that, very, very well. They're not required to do a thing more than that for your individual hardware.

    From a network perspective:
    There have been no global , multi-hour outages whatsoever, in the past few months. I'm not even sure if there has been one in the past year. If there HAD, I would certainly know about it, considering I have a pretty in depth monitoring system setup that iss more than the typical first grade monitoring system put out by most hosts.

    From a hardware perspective:
    If 'futurehosting' went with more reasonable hardware expectations, maybe they'd find a bit better performance. There are known problems with multiple configurations, and you can't just blame a provider for your own unreasonable expectations. That's hardware/software/kernel related, and it's NOT softlayer's problem.

    I've contacted softlayer and had them do intense hardware inspections. Did they find a problem? No. Did they resolve it? Of course, they did. Swapping out hardware solved the problem. Don't tell me that softlayer won't do this, because they WILL I've seen them do it myself.

    So, is this a SoftLayer problem? No. It is a 'futurehosting' problem. Just because 'futurehosting' says "OOOH, softlayer is the problem" doesn't mean it really IS, it just means they can't figure it out, so they blame the provider. Pretty typical there.

    Again, I'll be the first to admit softlayer has problems, but your spreading of misinformation here is wrong. Just because YOU have experienced INDIVIDUAL problems doesn't mean GLOBAL problems exist, and is not a cause for you to make accusations such as you have.

    There's a difference between "softlayer has performed bad for us" and "softlayer has global issues". My experiences are on a 'global' level, from multiple networks, routes, and servers. Your experiences are individual , not global. Mine are just the opposite.

    Facts. You keep using this word. I do not think it means what you think it means.
    Actually, yes I do.
    I could provide logs, connectivity logs, but it would break confidentiality, so I won't. The point is that your argument is not factual, but argumentative, without any sole proof. On the other hand, mine is not without proof, though I won't show you the proof because it would violate confidentiality.
    WHMCS Guru - WHMCS addons, management, support and more.
    WHMCS Notifications Extended - Add slack, hipchat, SMS, pushover to WHMCS !!
    Always looking for Linux, WHMCS, Support Desk work. PM for details

  24. #24
    Join Date
    Nov 2006
    Location
    College Station, TX
    Posts
    185
    There have been no global , multi-hour outages whatsoever, in the past few months. I'm not even sure if there has been one in the past year. If there HAD, I would certainly know about it, considering I have a pretty in depth monitoring system setup that iss more than the typical first grade monitoring system put out by most hosts.
    So you agree with my point. The outages I've been experiencing on equipment hosted for my service provider for Futurehosting have been localized to their room (room SL-4) and/or racks. Thanks.

    Hey, I'm not saying your experiences have been bad. I'm saying mine have been. Maybe you can't see the difference between that. You're welcome to your own opinion, but I wish you would express it without descending to a level of discourse that one would expect from a fourteen year old whose favorite pokemon was just insulted.

    The point is that your argument is not factual, but argumentative, without any sole proof
    Here.

    http://www.webhostingtalk.com/showth...ight=Softlayer
    http://www.webhostingtalk.com/showth...ight=Softlayer
    http://www.webhostingtalk.com/showth...ight=Softlayer
    http://www.webhostingtalk.com/showth...ight=Softlayer
    http://www.webhostingtalk.com/showth...ight=Softlayer
    http://www.webhostingtalk.com/showth...ight=Softlayer

    ... and you know it's good when the topic's thread is "How often is softlayer down?"
    http://www.webhostingtalk.com/showth...ight=Softlayer

    You'll note, if you actually choose to go read those links, instead of choosing to retain your head's current firm impaction in your rectal cavity, that I've filtered out any that ended up being an upstream provider or client network issue.

    ... But of course, no, it's just me.
    Last edited by karlkatzke; 04-20-2008 at 11:05 PM.

  25. #25
    Join Date
    Nov 2006
    Location
    Long Island, NY
    Posts
    287
    The two of you should(ve) called each other. Would(ve) save(d) a LOT of typing on both parts
    Robert @ SmoothVPS
    www.SmoothVPS.com - Since 2008

  26. #26
    Personally, I don't care what datacenter my particular provider calls their own, as long as it is a fit for my clientele. I will hold my provider's feet to the fire, not their subcontractors.

    If you experienced unacceptable outages, why didn't you move?

  27. #27
    Join Date
    Nov 2007
    Location
    CA
    Posts
    9
    I’ve had several systems with Future Host for over a year and have been affected by the issues mentioned in this thread. While my systems are only used for development (so it has never been a real issue for me), I can attest to the power problems at SoftLayer. This morning I had my system moved to the new datacenter and I’m happy that they got out of SoftLayer. Future Host truly shows their professionalism by taking the appropriate steps to ensure our uptime.

    A quick search pulls up this thread. The thread clearly shows the major power outage at SoftLayer last summer.

    I found an e-mail that was forward to us by Future Host shortly after the power outage. I assume SoftLayer’s CEO wrote it. I think it was the second outage listed that caused the downtime for us, which was around three hours. If memory serves me correctly, there was also a power issue at SoftLayer a couple of months ago that caused one of Future Hosts’ DNS servers to go down.

    One of my systems was also affected by the SAN problem at SoftLayer, which resulted in several hours of downtime and degraded performance.

    This is the e-mail that Future Host sent me:

    First and foremost, I want to apologize for the overall failure to deliver service as promised. The last 90 days have been extremely frustrating and I assure you that we are doing everything possible to make certain service returns to normal. The only way I know how to give any insight into the situation is to start from the beginning and explain in detail how the systems work, what happened to those systems, and what we are doing to ensure they do not suffer from interruptions in the future.

    The Datacenter Infrastructure

    SoftLayer currently has 4 datacenter units (we call them server rooms) and they are all serviced by different stand alone systems. Here is a general breakdown.

    SRO1 (server room 1) – this is the original SoftLayer datacenter and is serviced by utility service “A” - has its own pair of UPS units and its own generator. Inside server room one, there are several thousand customer servers and the SoftLayer Internal Equipment (core routers & servers).

    SR02 (server room 2) – this was our first expansion datacenter and is serviced by utility service “B” - has its own pair of UPS units and its own generator. Server room two houses several thousand customer servers.

    SR03 (server room 3) – this was the second expansion datacenter and is serviced by utility service “B” - which shares utility, UPS and generator with SR02. Server room three houses a couple thousand customer servers.

    SR04 (server room 4) – this is our most recent expansion datacenter and is serviced by utility service “C" - has its own UPS units and its own generators. Server room 4 currently houses several thousand live servers and it about 50% capacity. We expect SR04 to be full by the end of the year.

    SR05 (server room 5) – currently under constructions and will be our next expansion space beginning in Q4 2007. It is serviced by utility service “D”, will have its own UPS units and its own generators. Server room 5 will house roughly 15,000 servers which is our estimated growth for 2008 in Dallas.

    **Caveat** We have pulled power from different rooms into SR01 to dual home all internal equipment and routers so internal systems reside on separate utilities, UPS units, and generators. Some overlap exists from room to room for customer equipment, but the percentage is extremely minute.

    What happened and why did power fail?

    First Outage: SR04

    SR04 suffered from two power “resets” in the months of March/April that basically reset the entire datacenter in 30 seconds or less. The problem was due to a faulty UPS unit that would fail to go into bypass mode. The unit was new and tested prior to installation in January, but the defect was not found until after it was live. This defect bounced power twice in 30 days. Bouncing power sounds relatively harmless until you realize it means several thousand servers. When this happens, about 70% of the servers return as normal within minutes (typical reboot), 20% fail to return to normal service and need some hand holding (failed service restarts, improper loading, configs not saved, etc). 10% need an FSK or disk recovery assistance and take longer to bring live and unfortunately, a handful will suffer from hardware failure or corrupt data.

    Second Outage: SR02 & SR03

    SR02 and SR03 suffered from a power “outage” on April 14. This outage was due to a complete failure of a 2500 AMP breaker that services the datacenter floor. To give you some insight, this breaker is very large (think small refrigerator), weighs about 800lbs, and has a guaranteed life of 15 years. When the breaker was replaced, it was shipped off for review and manufacturer found the breaker to be below their standard metrics for the load at which it failed. The breaker worked fine at low loads and failed as the load increased. The manufacturer confirmed failure of the breaker at 80% load.

    Third Outage: SR01

    SR01 suffered from a power “outage” this past Sunday due to a faulty fire panel. All of our systems are “smart” and tied into the fire systems. The fire system is designed to go into alarm (during a fire), notify personnel and then eventually shut down power to protect human life. The faulty fire panel shorted and immediately shut down power to SR01 in random succession (not the normal course of action).

    Now – What are we doing about it?

    First, I want to assure you that all of SoftLayer’s systems are professionally designed and installed by world class companies that specialize in datacenter construction. Even our contractors have admitted to a little “Murphy’s Law” due to the type of issues across three independent systems. We have spent millions on redundant systems that have failed to deliver and we fully intend to resolve the issues.

    SR04 – the UPS has been completely replaced (one of the power maintenance windows last month). The new unit was tested under full load with a portable truck mounted generator prior to installation. Because of our dense installations and load levels, this will be the standard protocol going forward on all UPS, generators, and PDUs. Most companies ramp from 0% to 80% utilization over years, SoftLayer does it in months.

    SR02 – SR03 – the original electrical design lead to a single point of failure in the 2500AMP breaker. The breaker was replaced that night and has subsequently been removed completely (it is now a straight pass thru from the transformer). The rationale behind the design was due to a City of Dallas ordinance that required a single EPO (emergency power off) switch to the datacenter floor. This ordinance was modified late last year to have an EPO per UPS unit that services “portions of the datacenter.” For those that don’t know, an EPO button is the BIG RED BUTTON that powers everything off in case of emergency. It is only used to save human life. This design was corrected this month during a power maintenance window (no service interruptions occurred). SR01 has been modified as well and SR04 was built after the Ordinance was changed so design was not affected.

    SR01 – the faulty fire panel has been replaced and checked and double checked (and triple checked by me). In my twelve year career, I have witnessed faulty UPS units, generators that fail to start, faulty breakers, and other types of issues that you implement preventative maintenance and procedures to mitigate but I have never seen a faulty fire panel cause a power outage. This type of issue is so remote; I can’t even fathom the statistical possibility of it occurring again.

    We have spent the last 48 hours going through all the datacenters with a fine tooth comb. I brought in an electrical engineering firm that is going over every single aspect of the datacenters for the next two weeks. To give you an idea of how serious I am about mitigating any future issues; the engineering firm will be performing a myriad of tasks including the following.

    1. A breaker by breaker review of all power loads (compare their report to our internal report)
    2. A breaker by breaker review of breaker temperature (compare their report to our internal report)
    3. A manufacturer review of all breakers by serial number (to detect potential recalls)
    4. A complete preventative maintenance on all datacenter PDUs (power distribution units)
    5. A complete preventative maintenance on all UPS units
    6. A complete preventative maintenance on all generator units
    7. A comprehensive design, installation, and PM overview (a second opinion)

    What we have learned.

    1. Communicate with our clients. We have implemented several new systems (ex. Emergency Notification System) and will be implementing many more in the next month or so. We realize failure to communicate during these times is the most frightening part. If you have ideas, please send them my way. Many of you have given us great ideas that have been or will be implemented soon.

    2. Process and Procedure: increase the processes and procedures that affect internal systems. A human error in SR01 lead to plugging a rack full of dual power internal servers (including our www boxes) into a single source (SR01 power). Less than one foot away was a redundant circuit from SR02 and SR04 that would have maintained power. Future deployments will require multiple personnel to check off.

    3. Frequent Preventative Maintenance. SoftLayer currently follows industry guidelines for PM on all systems. Due to the recent activities, those will be escalated to a monthly review of all systems by outside contractors. I have concluded that due to our explosive growth and load levels, we will have to maintain a much higher standard than the average datacenter operator.

    4. Training, Training, and more Training. Improving the personnel training for instances such as these. You always train hoping they never occur, but use these instances as real world examples of how to respond.

    5. NO MORE OUTAGES!!! - We hear you loud and clear and we fully intend to do everything in our power to make certain nothing else occurs. No matter what is cost, no matter what we have to do, no matter what we must upgrade, replace or modify – you have my word – we will do whatever it takes to make it right.

    If anyone has any questions, please feel free to ask.
    Last edited by The Critic; 04-20-2008 at 11:41 PM.

  28. #28
    Join Date
    Nov 2006
    Location
    College Station, TX
    Posts
    185
    Quote Originally Posted by turbowarp View Post
    Personally, I don't care what datacenter my particular provider calls their own, as long as it is a fit for my clientele. I will hold my provider's feet to the fire, not their subcontractors.

    If you experienced unacceptable outages, why didn't you move?
    Simply put -- I've never gone longer than 5 minutes in an urgent situation without a reply from FutureHosting, and they've even jumped on some non-urgent stuff that they didn't have to do for me as an unmanaged customer. I'll happily deal with some downtime that isn't their fault if they're good about supporting me before, during, and after it.

    Of course, first sign that it was negligence on their part I'll jump ship like anyone else would. I'm not a hosting provider, but I do host a number of my own projects out of the VPS I rent and those projects provide me with essential income... but I haven't seen even the tiniest bit of negligence out of them, and I would recognize it if I saw it. I'm a linux sysadmin by trade.

  29. #29
    Join Date
    Jul 2006
    Posts
    285
    karlkatzke, you're just wasting your time arguing with linux-tech. Some people can't see past their own world and it's just a waste of time trying to reason with them. Plenty of other people know about Softlayer problems, and it's not just "global" outages. It's a lot of niggly issues combined with how they are sometimes dealt with/not dealt with.

    Anyway, this might have nothing to so with Softlayer's quality of service. It's very possible that it's just more economical to colo your own hardware in a VPS hosting environment. The reason this was done may or may not come out in the future, but what matters is that Futurehosting does take good care of it's customers and Vik really does care about the quality of his service.

    So whatever the reason for this decision, I'm sure that Vik made it with his customer's best interest in mind.

  30. #30
    Join Date
    Nov 2006
    Location
    College Station, TX
    Posts
    185
    I may be wasting my time, but I enjoy the traditional internet sport of troll-baiting. Someday, 200 years in the future, they'll put two computers in an arena and install an old copy of PHPBB and let two people fight to the death...

  31. #31
    Quote Originally Posted by karlkatzke View Post
    I may be wasting my time, but I enjoy the traditional internet sport of troll-baiting. Someday, 200 years in the future, they'll put two computers in an arena and install an old copy of PHPBB and let two people fight to the death...
    OT: 200 years? It's actually a 'sport' on some forums right now, they even award points and keep a scoreboard, though they're mostly populated by 13 year-olds .
    478east
    High Bandwidth Servers
    Custom Hosting Solutions

  32. #32
    Join Date
    Jan 2006
    Location
    Jersey
    Posts
    2,965
    Quote Originally Posted by layer0 View Post
    SoftLayer has been using FCP for quite some time too, and whether it is actually beneficial to the customer is debatable and dependent on how it is configured--but that's just my opinion of course.
    Oh...I wasn't aware they were using FCP, they don't advertise it as much though, weird...heh. If I were using FCP I would have it on every ad banner or advert post I make on WHT, lol.
    Email: info ///at/// honelive.com

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •