Results 1 to 37 of 37
  1. #1

    Rackshack Fiber Cut ?

    Just received this from RS but only 1 of our 4 servers is affected.
    Are all their servers in the same building, and how did they damage multiple lines?




    ------------------------
    We have had multiple gigE fiber lines cut and are working to bring these
    back up as quickly as possible. We should have this resolved no later than
    2 more hours. We apologize for this outage but unforunately it was outside
    our control.
    ------------------------

    Gordon
    Formerly: Managing Director, Hostroute.com Ltd & Marketing Director, Ultraspeed UK Ltd
    View my Professional Profile: www.gordonhudson.com

  2. #2
    Join Date
    May 2002
    Location
    England , United Kingdom
    Posts
    203
    looks like ....

    cogent and time warner line has been cut
    : UTADMIN : Carl :

  3. #3
    Join Date
    Aug 2001
    Location
    Houston, Texas
    Posts
    695
    There are updates in our forum and in our chat room.

    We lost two linear Gig-es that are provided over Tw transit. We have now managed to turn up some dormant OC-3 circuits temporarily to alleviate the loss.

    All servers, save 8 high bandwidth customers, are back in business and two of those we delinquent. So, 6 real customers affected at this point. We expect those 6 to be back in business in the next 30 minutes.

    Updates will be posted on the RS forums.
    Robert Marsh
    Head Surfer

  4. #4
    Thank you very much for a update HeadSurfer, I still think you need a revolving status page. That is UPDATED at all times.

    See the attached gif for ideas!

    Heck, I would keep it updated for you.
    Attached Thumbnails Attached Thumbnails di.gif  
    L. James Prevo - President/Owner
    Prevo Network, LLC - http://www.prevo.net
    Est. 1999 - Month to Month Billing!!

  5. #5
    The server is back on line but not before I received this
    from RS support:


    ------------------------
    7/12/02 - lpate
    we found out that the reason why server was unpluged was per report for
    over charge bandwidth, cust has been unplugged, cust needs to pay balance
    before it is re-pluged, please make sure that server is unpluged, lp..
    ------------------------

    Rackshack had not billed me for any overage or attempted to charge my credit card for it.

    What sort of shambles is this company?

    I have never heard of a dedicated server being unplugged for bandwidth overage when no bill has been sent and no attempt made to obtain the money owed.

    I am going to write to Rackshack and ask for an apology.
    What they have done today is completely unforgivable by anyones standards.

    Gordon
    Formerly: Managing Director, Hostroute.com Ltd & Marketing Director, Ultraspeed UK Ltd
    View my Professional Profile: www.gordonhudson.com

  6. #6
    Join Date
    Jul 2000
    Location
    Colorado Springs, CO
    Posts
    2,280
    Originally posted by ljprevo
    Thank you very much for a update HeadSurfer, I still think you need a revolving status page. That is UPDATED at all times.

    See the attached gif for ideas!

    Heck, I would keep it updated for you.
    I beleive its plastered all over that companies section you pulled that from that you can not publicly distribute that type of thing.
    Greg Landis | Founder Jaguarpc - Keeping websites happy since 1998
    Managed IT Solutions - Business hosting | Virtual Private Servers | Cloud VPS Hosting | Dedicated servers | Backup service
    Follow us @ Facebook.com/Jaguarpc | Twitter: @JaguarPC | (888)-338-5261 | sales @ jaguarpc.com

  7. #7
    Join Date
    Nov 2000
    Posts
    374
    Originally posted by Jag


    I beleive its plastered all over that companies section you pulled that from that you can not publicly distribute that type of thing.
    I was thinking the same thing... and in addition, why block out the name if you're going to name the attachment a commonly used abbreviation for the company?

  8. #8
    Join Date
    Dec 2001
    Location
    Darmstadt, Germany
    Posts
    1,096
    so if the board is down, we know rackshack is down too
    In just two days, tomorrow will be yesterday.

  9. #9
    Join Date
    Jul 2000
    Location
    Colorado Springs, CO
    Posts
    2,280
    Originally posted by JG


    I was thinking the same thing... and in addition, why block out the name if you're going to name the attachment a commonly used abbreviation for the company?
    good catch, I didnt even see that at first.... you could be asking for trouble with that post.
    Greg Landis | Founder Jaguarpc - Keeping websites happy since 1998
    Managed IT Solutions - Business hosting | Virtual Private Servers | Cloud VPS Hosting | Dedicated servers | Backup service
    Follow us @ Facebook.com/Jaguarpc | Twitter: @JaguarPC | (888)-338-5261 | sales @ jaguarpc.com

  10. #10
    Join Date
    Jul 2001
    Location
    Toronto, ON.
    Posts
    312
    hehe. that's rackshack for you.

    i had a sorta similar issue with them (well, not really). basically, i got a server from them. when i first logged in, it had already been up and online for a few weeks. anyways, a few weeks later my server gets taken offline because it was supposedly flooding their network. so yeah, i was accused of being a poor sys. admin. when the box had infact been sitting online unpatched with all services exposed for weeks before i had even touched it. gotta love it.

  11. #11
    Join Date
    May 2001
    Posts
    682
    Originally posted by GordonH
    The server is back on line but not before I received this
    from RS support:





    Rackshack had not billed me for any overage or attempted to charge my credit card for it.

    What sort of shambles is this company?

    I have never heard of a dedicated server being unplugged for bandwidth overage when no bill has been sent and no attempt made to obtain the money owed.

    I am going to write to Rackshack and ask for an apology.
    What they have done today is completely unforgivable by anyones standards.

    Gordon
    When has anyone been billed for overages? Last time I heard HS talk about overages he said they haven't billed anyone for overages because of overall everyone having a low average....

  12. #12
    Join Date
    Nov 2000
    Posts
    374
    Originally posted by rastoma


    When has anyone been billed for overages? Last time I heard HS talk about overages he said they haven't billed anyone for overages because of overall everyone having a low average....
    Check out this thread on Rackshack's forum.
    http://forum.rackshack.net/showthrea...&threadid=8081

  13. #13
    Join Date
    Aug 2001
    Location
    Houston, Texas
    Posts
    695
    We have selected the top "over users" for billing for June. For July, billed in August, we will be billing all overages.

    There have been too many people take advantage of us recently on this issue.

    One gentleman , who was pushing about a 20 Mb stream, we could only collect about 1/3 of what he owed. Another has refused to pay.

    Earlier today there was some confusion as to why about 10 servers were "null routed/unplugged" during the height of the fiber cut where we lost two of our Gig-e circuits. We had to make a very hard decision about how to provide the best possible service to the most people during this time when we lost a significant part of our network due to a garbage truck driver.

    Out of 7,000 servers, we null-routed or unplugged the 10 Mb servers for a short time as well as our top ten bandwidth users. By taking these 160 or so customers out of service, we were able to route the balance of our customers.

    As TW brought up the first emergency OC-3, we re-enabled the 10 MB servers. When the second emergency OC-3 came up, we re-routed the 10 "over users" back into the network.

    There was obviously some confusion about what was done for what reason during this hectic time and some bad information did get distributed to a few of the customers involved. I am truely sorry for this miscommunication. I also shared these sentiemnts with the customers in our chat room, where I was available just about constantly, as well.

    Two of our Gig-e circuit loops have first generation/early adopter technology running them in a linear fashion. All new Gig-e loops run in a redundant/non-linear setup. The old loops run on Lux-n gear while the new loops run on Extreme boxes. We are in the middle of a project to move the older Gig-es to the new Extreme non-linear configurations to reduce the likelyhood of a cut affecting us in this manner again.

    It is unfortunate when any customer has an unavailability of their servers for any period of time. In this instance, we were hit with a catastrophic loss of connectivity, implemented emergency plans, and continued to work with our providers to make the best of a bad situation. While a large portion of our network experienced some latency and some very brief periods of packet loss, only about 160 servers lost total connectivity for a period during the fiber cut.

    The miscommunication was unfortunate and I certainly offer my apology to all of those affected. This fiber cut just happened to come at the exact same time that we were going through and billing/collecting for the bandwidth overcharges. For customers in our "top 10" list, and located in the United States, I made personal calls to discuss their overages. During the period affected by the cut, I was in out chat room talking with our customers as well as posting updates to our forum on a regular basis.

    Robert
    Robert Marsh
    Head Surfer

  14. #14
    Join Date
    Nov 2001
    Location
    California
    Posts
    1,991
    Headsurfer, it isn't right for you to just take their servers off the network without notice either.

  15. #15
    Join Date
    May 2001
    Location
    Dayton, Ohio
    Posts
    4,962
    Yeah.. I'd be upset if I paid 400/month for a 10mbit deal, and the owner of the datacenter decide to not turn my port back on, while everyone else is up and running...
    -Mat Sumpter
    Director, Product Engagement
    Penton Media

  16. #16
    Robert

    We had an overage of about 150GB.
    I am not disputing it even though the MRTG and Ensim stats do not support it.

    But taking aserver offline for a 150GB overage with no notification, not even an e-mail?
    And when I phoned up, nobody in customer service knew why the server was off line.

    Its a small amount of money to start taking servers off line for
    under any circumstances.

    Even in the emergency situation I would have expected my server to be treated the same as anybody elses.

    This month so far it has used 200GB so its hardly one of your biggest.

    Its simply bad business practice to pull servers with no notification or explanation.

    Make sure this does not happen again.

    Gordon
    Formerly: Managing Director, Hostroute.com Ltd & Marketing Director, Ultraspeed UK Ltd
    View my Professional Profile: www.gordonhudson.com

  17. #17
    Join Date
    May 2001
    Posts
    682
    Originally posted by JG


    Check out this thread on Rackshack's forum.
    http://forum.rackshack.net/showthrea...&threadid=8081

    yeah.. i read that after i posted


    GOOD!!! RS needs to bill for overages... that's the point of have a LIMIT

  18. #18
    Originally posted by Jag


    I beleive its plastered all over that companies section you pulled that from that you can not publicly distribute that type of thing.
    Try again, the only area that states what can't be displayed is the actual network issues, which I did not post.
    L. James Prevo - President/Owner
    Prevo Network, LLC - http://www.prevo.net
    Est. 1999 - Month to Month Billing!!

  19. #19
    Join Date
    Aug 2001
    Location
    United kingdom
    Posts
    1,003
    It's about time that Rackshack starts charging for overages, as it's not fair for customers who only use 50Gb/month or around there. For those small bandwidth users, it's means we get slightly slower speeds as the other large bandwidth users hog the bandwidth lines. Overall, it's going to affect Rackshack as a company, as if they don't get enough money, then they'll be the next company who files for bankruptcy... (even though we know that they're a profitable company, as Robert Marsh has said several times).

    We're one of those users who use small amounts of bandwidth, and we've been happy with their service for the 6 months that we've been with them.

    So to those people who use a LOT of bandwidth... no more free bandwidth!

    Alan
    Alan Ho
    Former Systems Administrator

  20. #20
    Where did anyone state in this thread that customers should not be charged for bandwidth overage?

    Gordon
    Formerly: Managing Director, Hostroute.com Ltd & Marketing Director, Ultraspeed UK Ltd
    View my Professional Profile: www.gordonhudson.com

  21. #21
    Join Date
    Aug 2001
    Location
    United kingdom
    Posts
    1,003
    GordonH, may I ask who you were replying to?

    I don't think it says anywhere that people should not be charged for bandwidth overage, but Rackshack have not been charging for it for the past 6 months or more, which everyone or most people know and so have taken advantage of it. But now that Robert Marsh has finally got the system working and have taken action... it just makes his company stronger.

    Alan
    Alan Ho
    Former Systems Administrator

  22. #22
    Join Date
    Apr 2002
    Location
    Somewhere
    Posts
    17
    Good luck Robert getting everything back. I myself think it was a very gutsy decision turning off the 10MB/S Ports, and your 10 most bandwith hungry clients, just so the rest of the network could remain operational. I would have done the same if i was in the same situation. Good work, and i hope everything continues to go well.
    Microsoft's new advertising slogan -
    "Buy a Microsoft Product, Help In The Downfall Of The Internet"

  23. #23
    I give up.

    People seem to have taken this thread off in a weird direction.

    I did my bandwidth analysis and I was 33% over
    Its hardly the end of the world and it does not make us one of their 10 highest bandwidth users, especially as this month we won't be over.

    We are buying bandwidth from them and we have every right to expect they will deliver it on demand.

    Just unplugging a server without attempting to charge for the nadwidth is completely unacceptable and all you people who are suggesting it was a great business decision did not have to deal with the 100 customers whose sites were down as a result of Rackshacks incompetence.

    On a general note:
    I really despair of this board these days.
    It seems to be full of people trying to make out they are more ethical or better at customer service than other hosts.
    It used to be a place where people could help each other but those days seem to be over.
    Its all about self promotion and trying to recruit customers these days.

    Gordon
    Formerly: Managing Director, Hostroute.com Ltd & Marketing Director, Ultraspeed UK Ltd
    View my Professional Profile: www.gordonhudson.com

  24. #24
    Join Date
    Aug 2001
    Location
    United kingdom
    Posts
    1,003
    I think both sides are right here... we're right as Rackshack do need to sort out the large amounts of bandwidth that is not being paid for and people are using like it's free. While Gordon is still correct, as Rackshack did not try to get payment off them, as I'm sure you would have paid it if they did ask for it to keep your server online.

    Robert was going on the stats he had for last month, so even if you were not going to go over your limit this month, he thought you were, without looking at the current stats, which was not good of him. And to conclude... I'm on your side Gordon, with this case .

    Alan
    Alan Ho
    Former Systems Administrator

  25. #25
    Join Date
    Aug 2001
    Location
    Houston, Texas
    Posts
    695
    I am not sure that you guys still understand. Today's issue is not related to a partucular customer being over teh included amount or not. Gordon is a good and valued customer and I can certainly understand his frustration. I would hate to think what I would have written if I had been in his situation as I tend to be extremely hot headded.

    It was a terrible, horrible, no good, very bad day (sure felt like a full moon and Friday the 13th all rolled into one) in my book and I am sure in Gordon's as well as well as anyone on "the list".

    I was left with a very hard decision to make. I take full responsibility for it. In the end, I take full responsibility for the entire situation. That's my job .. take the credit when all is well and take the blame when all is "less than well".

    Under the same circumstances, given a catastrophic event (such as losing 2 Gig-e lines and a 3rd running at only 50%), I would have to take the same actions. Granted, I woudl hope that the communication could have improved.

    Monday will bring a little Monday morning "quarterbacking" as we try to understand what TRUE risks are out there, how they impact us, etc. It give us a great time to re-evaluate our disaster plans and take a harder look at our disaster recovery plan and what backup procedures and mechanisms we have.

    Our gigantic size has really crept up on us over the last 5 months and we have a much larger responsibility because of it. I just hate to thing about the number of actual domains hosted on our servers as most of them are sold to resellers. (could it be 65,000 (10 per server), 195,000 (30 per server), ???)

    Thanks again to everyone for you understanding during a very difficult afternoon.

    Robert
    Robert Marsh
    Head Surfer

  26. #26
    Join Date
    Nov 2001
    Location
    California
    Posts
    1,991
    Crap happens I guess, but if it happened to me, I would like to know what's going on before it happens. Thank goodness, I'm probably in the last 10, of lowest bandwidth usage...

  27. #27
    Join Date
    Jul 2001
    Location
    /dev/null
    Posts
    1,219
    Originally posted by GordonH
    On a general note:
    I really despair of this board these days.
    It seems to be full of people trying to make out they are more ethical or better at customer service than other hosts.
    It used to be a place where people could help each other but those days seem to be over.
    Its all about self promotion and trying to recruit customers these days.
    Gordon
    Well said!

  28. #28
    Join Date
    Aug 2001
    Location
    United kingdom
    Posts
    1,003
    Well there you go... a reply from the man himself, Robert Marsh, the 'HeadSurfer' . Thanks for the personal post there Robert, it realy helps when a CEO replies to a problem regarding his own company.

    Alan
    Alan Ho
    Former Systems Administrator

  29. #29
    Join Date
    Aug 2001
    Location
    Matrix
    Posts
    2,469
    Originally posted by headsurfer

    I was left with a very hard decision to make. I take full responsibility for it. In the end, I take full responsibility for the entire situation. That's my job .. take the credit when all is well and take the blame when all is "less than well".

    Robert
    Robert Marsh Rocks (RMR)
    CPHosting - Web Hosting Experts Since 1998.
    United States | Europe | Singapore | Australia
    Visit Us! www.cphosting.com

  30. #30
    Join Date
    Dec 2001
    Location
    Darmstadt, Germany
    Posts
    1,096
    Originally posted by mahinder


    Robert Marsh Rocks (RMR)
    RMR! lol
    In just two days, tomorrow will be yesterday.

  31. #31
    Thank you Robert
    I do hope you find a better way to handle billing situations
    and emergencies like this.


    One tip: if you get the right phone company you can call long diatance to here very cheaply.
    I am sure you know that already.
    We pay the same to phone Houston as we do to phone London
    (and I am in the UK)

    We call US customers all the time from here without spending a lot of money.

    Gordon
    Formerly: Managing Director, Hostroute.com Ltd & Marketing Director, Ultraspeed UK Ltd
    View my Professional Profile: www.gordonhudson.com

  32. #32
    Join Date
    Aug 2001
    Location
    United kingdom
    Posts
    1,003
    Gordon, which telephone company are you using? Just wanted to know out of interest since I'm located in the UK aswell. (I'm happy with the setup now, but just wondering .)

    Alan
    Alan Ho
    Former Systems Administrator

  33. #33
    I have a Onetel account
    3p per minute to the US and Canada.

    Normal BT rate to London is 3.5p daytime.

    It makes dealing with suppliers and customers in the US a lot easier when the calls are so cheap.

    I even phone the owners of stolen credit cards!

    Gordon
    Formerly: Managing Director, Hostroute.com Ltd & Marketing Director, Ultraspeed UK Ltd
    View my Professional Profile: www.gordonhudson.com

  34. #34
    Join Date
    Aug 2001
    Location
    United kingdom
    Posts
    1,003
    Ah, yes, One.Tel was the first company I thought of when you mentioned finding a phone company in your other post . They provide really good prices.

    Alan
    Alan Ho
    Former Systems Administrator

  35. #35
    Join Date
    Apr 2002
    Location
    AU
    Posts
    1,048
    one.tel in australia collapsed

  36. #36
    Its not the same company.
    Onetel.net.uk is part of Centrica PLC
    Formerly: Managing Director, Hostroute.com Ltd & Marketing Director, Ultraspeed UK Ltd
    View my Professional Profile: www.gordonhudson.com

  37. #37
    Join Date
    Apr 2002
    Location
    AU
    Posts
    1,048
    one.tel in australia was imo the best phone company. They only collapsed because their billing system was fubared and didn't bill for most phone calls.

    they are back and rebranded as teleone

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •