Page 1 of 7 1234 ... LastLast
Results 1 to 25 of 156
  1. #1
    Join Date
    Feb 2003
    Posts
    44

    Orange Fiber / Alpha Red - Down?

    It appears I posted this at the same time as AlphaRed posted an update - see http://www.webhostingtalk.com/showthread.php?t=549369
    Last edited by MartHUK; 09-23-2006 at 05:08 AM.

  2. #2

    AlphaRed Inc/Orange Fiber Outage - Transfer switch failure notice

    Alpha Red Inc is currently experiencing a primary transfer switch failure (as diagnosed at this time), which is causing utility feed to our datacenter to have ceased.
    We are aware of this issue, and are currently working on resolving it as soon as possible.

    Please be patient, as we do realize that your uptime is important, and we will have this rectified as soon as humanly possible.


    If you have any questions, you can reach me on my cell phone at: 713-962-5791

  3. #3
    Join Date
    Feb 2003
    Posts
    44
    Thanks for the update, much appreciated.

  4. #4
    Thanks for the info. Any idea how long this will take to resolve? I mean is this going to be a few more minutes, or are we talking hours?

    Please keep us informed. Thanks.

  5. #5
    Hello,

    I have an update on situation:

    The transformer room for our building, where our transfer switch is located is flooded.
    From what I've been able to gather, is that the building's fire suppression system has malfunctioned and is pumping water into the underground transformer room, and has caused our equipment, as well as our utility provider's equipment Centerpoint, to automatically shutdown to avoid damage.

    The Houston Fire Department is at the scene now attempting to shut off water flow and pump the water out, so that Centerpoint can begin work on restoring power.

    I will update further as I can.

    Please be sure we are working on this as fast as possible - it is 4AM on a Saturday, and we are unflinching in making sure this is getting resolved ASAP.


    Thanks

  6. #6
    Join Date
    Jul 2005
    Posts
    76
    nevermind, thanks for keeping us updated

  7. #7
    So, does this mean all the servers lost power? Does that mean even after this is resolved I'm going to have to wait for my servers to run fsck to make sure the HD's didn't die?

  8. #8
    Yes, all servers have lost power, a situation like this was never even imagined.
    Your server may or may not have to FSCK when it comes back online.
    Our UPS system's have built-in power conditioners which should've kept any current spikes from reaching server's, I don't expect that many failed HDD's.

  9. #9
    The damage sounds pretty huge. Do you have any idea yet about how much longer this will be?

  10. #10
    I don't have an ETA.
    I have learned that the entire city power grid that we are on is now down as well.

    We are working on alternative means of bringing power backup - we arn't going to wait on the building, Centerpoint or the fire department at this point.

    We are actively pursuing alternative power means, including re-wiring our feed away from our transfer switch direct to our generator, so that we can get power restored before the rest of the building/city grid.
    Last edited by MSV_AR; 09-23-2006 at 06:17 AM.

  11. #11
    Wow, you guys took out the city! 0_o

    I hope this can be resolved quickly, thank you for all your hard work.

  12. #12
    man, that's bad.. thanks for the update and hope this gets resolved soon!

  13. #13
    Quote Originally Posted by Abobo
    Wow, you guys took out the city! 0_o

    I hope this can be resolved quickly, thank you for all your hard work.
    Not the entire city, just the grid we are on is out. Also, we wern't responsible for the issue, though we are directly affected.

    We are all working on this as fast as possible guys - don't forget that this happened around 4AM CST on a Saturday, it is now 5.30AM CST on Sat. and we are trying to get all sorts of people to answer their phone.

    I am having good luck getting the neccesary people on the ball though, so I am hopeful that we will be up shortly. (relatively speaking, of course)
    Plans have already been set in motion, and our own electricians are on their way.

  14. #14
    Guys, I am heading downtown to the datacenter - the entire city grid is still out, so I don't know what type of access I will have, I am bringing my laptop with VerizonWireless internet, so I should be online once I get there.

    Please call me if you have any questions - though we are doing what we can to make sure we are back up ASAP.

    I will take pics while I am there, there is flooding like 20 feet high.

  15. #15

    More Contact Info

    I'd also like to mention that you can reach Tech Support directly at 713-589-3238. They are offsite but will be current with the situation and, once power is restored, they can escalate any server problems directly to the onsite staff. They are also available on ICQ. You may also reach me on ICQ or gtalk.

    Alphatech01: 235-435-304
    Alphatech02: 224-215-392

    --
    MattJB
    Alpha Red, INC
    ICQ: 133076358
    Gtalk: alpha.red.admin

  16. #16
    Join Date
    Jan 2004
    Location
    Sydney
    Posts
    148
    Just wanted to say that as an OF customer you guys seem to really be on top of this one - good job on the updates!

  17. #17
    Join Date
    Oct 2002
    Posts
    277
    From what I see (in this thread) , this will take days to fix......
    Last edited by marmoset01; 09-23-2006 at 08:36 AM.

  18. #18
    Join Date
    Sep 2005
    Posts
    551
    Thanks for the update MSV_AR

  19. #19
    Join Date
    Jul 2005
    Posts
    76
    Quote Originally Posted by yegorpb
    From what I hear, this will take days to fix......
    Where did you hear that? I doubt this is the case, but if it is please someone from OF confirm it, I cant afford days of downtime on 6+ servers.

    If not, is there any ETA or just a rough estimate?

  20. #20
    Join Date
    May 2003
    Location
    Moon
    Posts
    1,177
    Quote Originally Posted by yegorpb
    From what I hear, this will take days to fix......

    are u sure

  21. #21
    I just received word from the workers onsite. The situation is very close to being resolved. At this point we're looking at 30 minutes to 1.5 hours at the maximum, from what I've been told. I'm going in now, the techs are still available at the numbers above.

  22. #22
    Join Date
    Oct 2002
    Posts
    277
    Quote Originally Posted by yegorpb
    From what I hear, this will take days to fix......
    By "hear" I mean my speculation from the posts Ive seen in this thread.

    Flooded equipment......water 20 feet high..... that seems pretty bad.

  23. #23
    Quote Originally Posted by yegorpb
    From what I hear, this will take days to fix......
    This is absolutely not true.

  24. #24
    Join Date
    Oct 2002
    Posts
    277
    Quote Originally Posted by MattJB
    I just received word from the workers onsite. The situation is very close to being resolved. At this point we're looking at 30 minutes to 1.5 hours at the maximum, from what I've been told. I'm going in now, the techs are still available at the numbers above.
    I hope you are right.

  25. #25
    Quote Originally Posted by yegorpb
    By "hear" I mean my speculation from the posts Ive seen in this thread.

    Flooded equipment......water 20 feet high..... that seems pretty bad.
    Yes the damage to the buildling equipment is extensive and will take several more hours to fix. However we're not relying on building power, the problem is our connection to the generator which will be fixed very soon. Once we are on the generator everything will be back to normal.

Page 1 of 7 1234 ... LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •