Results 1 to 13 of 13
  1. #1

    Crystaltech Down? [MERGED]

    I can't get any of my sites or their domain to load...

  2. #2
    Join Date
    Jun 2005
    Location
    Secaucus, NJ
    Posts
    52
    12:25 PM - New Jersey - I can't connect to my servers for past 2 hours.

  3. #3
    Looks like their site is loading now... maybe just a matter of time?

  4. #4
    Join Date
    Mar 2004
    Location
    Minnesota, USA
    Posts
    24
    Has anyone heard anything from Crystal Tech on this?

    All of my accounts with them are down, their website is down, every support channel is jammed... and its going on 3 hours now.

    This one seems seems like a real doozy.

  5. #5
    Their DNS went sour and their phones are busy.... be patient

    checkdns.net / quickcheck domain

  6. #6

    Crystaltech downtime?

    Hey Folks,

    I have a website with Crystaltech and not only is my site down, but Crystaltech.com is also offline. Does anyone know what is going on?

    I cannot get through to any technical support by phone, and all their email servers are down. This is pretty much the height of incompetence as far as I am concerned...

    Anyone have suggestions or the same experience? My guess is that someone over there probably tripped over a power cable or something...

    DK
    Last edited by ldcdc; 11-27-2006 at 02:47 PM. Reason: subscribe

  7. #7
    Yup down for me also. website and email.

  8. #8
    Join Date
    Oct 2002
    Location
    EU - east side
    Posts
    21,920
    It's down for me as well.

    http://www.alertra.com/spotcheck.php seems to reach it though, slowly, but it does apparently.

    Code:
    Time (US/Eastern) 	Checked From 	Result 	KBytes 	Secs 	Kbps
    11/27/2006 13:48:14	Frankfurt GERMANY	OK	36.0	12.45	23.65
    11/27/2006 13:48:12	Orlando USA	OK	36.0	11.18	26.34
    11/27/2006 13:48:10	Chicago USA	OK	36.0	9.00	32.73
    11/27/2006 13:48:17	Oklahoma City USA	OK	36.0	16.10	18.29
    11/27/2006 13:48:19	London UK	OK	36.0	17.98	16.39
    11/27/2006 13:48:10	Atlanta USA	OK	36.0	9.62	30.63

  9. #9

    Exclamation Crystaltech down...

    They have been down for over 2 hours for me and they are still down.... Also the main CrystalTech site is down as well....

  10. #10

    Talking

    My site and Crystaltech.com just came back online.... Lets see how long it stays up this time.

  11. #11
    Join Date
    Mar 2004
    Location
    Minnesota, USA
    Posts
    24
    My accounts came up for a short time, but they crashed back offline again. Does anyone have any word yet on the cause there?

  12. #12
    Many of my sites were down for a couple hours yesterday... but seemed to be fine after they came back up. Here is the information CrystalTech sent out about the incident:

    /////////////////////////////////////////
    We wanted to provide you with some more detail surrounding the server issues that occurred earlier today.

    This morning, we experienced a large Distributed Denial of Service attack (DDoS) on our main network. The attack was against the shared segment of our network and our Domain Name Servers (DNS) specifically. CrystalTech blocks this kind of malicious activity on a regular basis as our Network and Server Operations Teams are well equipped and trained to keep it from affecting the services we provide. However, this was a large scale and targeted DDoS attack on our DNS which caused the 3 – 4 hour window of intermittent availability you may have experienced.

    Given the scope of this issue, it was resolved as efficiently as possible. Our Server and Network teams reduced traffic into our network by blocking thousands of offending IP addresses which were causing websites, email and shared databases to become unavailable. Then, traffic to our DNS servers was rerouted to mitigate the attack and to reduce any further activity.
    /////////////////////////////////////////

    I have had good luck with CT hosting over the past couple of years... and this is the first time I have ever had down time during normal business hours for more than a couple minutes.

  13. #13
    Join Date
    Mar 2004
    Location
    Minnesota, USA
    Posts
    24
    Well a few days later, now that the dust has settled I thought I'd put in a brief follow-up.

    Although the length, timing and scope of the service disruption was disappointing CT performed admirably to respond to the situation as it progressed, given the available resources.

    In my experience, you can only truly appreciate the nuances network security and stability if you've been forced to respond quickly to a terrible situation. I think they earned a few battle scars and a renewed sense of vigilance that will help them a lot in the future.

    ROATcom posted a partial transcript of the customer service email sent out by CT after the incident. Here is the full text of that email:

    Dear CrystalTech Customer,

    We wanted to provide you with some more detail surrounding the server issues that occurred earlier today.

    This morning, we experienced a large Distributed Denial of Service attack (DDoS) on our main network. The attack was against the shared segment of our network and our Domain Name Servers (DNS) specifically. CrystalTech blocks this kind of malicious activity on a regular basis as our Network and Server Operations Teams are well equipped and trained to keep it from affecting the services we provide. However, this was a large scale and targeted DDoS attack on our DNS which caused the 3 – 4 hour window of intermittent availability you may have experienced.

    Given the scope of this issue, it was resolved as efficiently as possible. Our Server and Network teams reduced traffic into our network by blocking thousands of offending IP addresses which were causing websites, email and shared databases to become unavailable. Then, traffic to our DNS servers was rerouted to mitigate the attack and to reduce any further activity.

    While we take every effort to resolve network issues as quickly as possible, this was a very different and unique circumstance which required more time to resolve. While we are not satisfied with the length of time our shared network was effected, it easily could have taken longer to contain if not for the skill set of our Support, Network and Server Teams and the equipment and technology utilized within our current infrastructure.

    Should you notice any further issues or discrepancies with how the service is performing, please let us know right away. Thank you for your patience while this issue was resolved, and we sincerely apologize for any inconvenience this may have caused.

    CrystalTech
    www.crystaltech.com

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •