Results 1 to 20 of 20
  1. #1
    Join Date
    Nov 2000
    Location
    localhost
    Posts
    3,771

    Linode London - Completely down

    Connectivity Issue
    Investigating - We are aware of connectivity issues to Linodes in our London datacenter and are currently investigating.
    Dec 27, 05:37 UTC

    Disappointing it took nearly 25 minutes to update the status page.

  2. #2
    Join Date
    Mar 2004
    Location
    UK
    Posts
    265
    Offline for 1 hour and counting...

  3. #3
    Join Date
    May 2014
    Location
    UK / USA (California)
    Posts
    105
    Same problem here. Linode became an absolute crap in the last 3 days. All their locations are down all day.
    Useless service.

  4. #4
    Join Date
    Apr 2000
    Location
    Brisbane, Australia
    Posts
    2,602
    my Linode Fremont, CA servers all fine.. touch wood

    maybe they are using holiday period for some maintenance ?

  5. #5
    Join Date
    Mar 2014
    Posts
    292
    My Dallas Linodes were down twice in the last two days totaling around 25 minutes


    Sent from my mobile device.

  6. #6
    Join Date
    Nov 2000
    Location
    localhost
    Posts
    3,771
    @eva2000 its not planned maint

    It's been down due to a DDOS for ~3.5hrs now. Traceroutes stop in Osaka Japan at the moment. It almost seems like they turn it back on every ~30 minutes and you see 95% packet loss for a few minutes, then they stop announcing again,as if they are simply waiting for it to fade out. First major 3hr+ outage in London Linode I recall in 5 years

  7. #7
    Looks like another DDoS: http://status.linode.com/

    This meant a 5:08am wakeup call for me

    To be fair to Linode though it's a double-bank-holiday weekend, the night shift must have been a skeleton. These things happen and it takes time to get engineers physically on location and it appears they were all-hands-on-deck by 6am.

  8. #8
    Appears to be a complete blackout in London. Every hour IPs come back online, and then they go offline again. It's as though they have been null routed to prevent packet loss to other customers in the Telecity facility.

  9. #9
    Join Date
    Nov 2000
    Location
    localhost
    Posts
    3,771
    Unfortunately with Linode you never know, they are super secret in post-mortems about all failures, I doubt we'll understand more than "its was a big DDOS" which makes it really difficult to evaluate future risk.

    Agreed, yes thankfully it is morning still and double bank holiday weekend, however if a DDOS takes out the whole of London network (which it practically has - practically completely out = partial outage) I worry about next week and the week after. At their size of network they should definitely have already reached out to people like ArborNetworks etc..

  10. #10
    Join Date
    Apr 2000
    Location
    Brisbane, Australia
    Posts
    2,602
    ah DDOSes ! nasty indeed

    maybe Linode 2016 will roll out DDOS protection

  11. #11
    Linode are being hit hard again. London is down. Looks like uptime may slip into 98% this month... what a nightmare.

  12. #12
    Join Date
    May 2014
    Location
    UK / USA (California)
    Posts
    105
    What should we expect when after another day of complete downtimes, they said yesterday "no other attacks occurred in the last hour so we consider the problem as resolved". This was on their status page.
    Per their opinion a problem is solved with it stops on its own. From their announcement it was pretty obvious that they resolve the problems with the attacks by just waiting the attacks to stop. And voila the attacks are back today for sixth day.

  13. #13
    Another day, another Linode outage. My phone now has hundreds of unread SMS outage noifications... this is getting tiresome.

  14. #14
    Atlanta appears to be under attack now, this is pretty ridiculous.

  15. #15
    Here's their first big public update:


    In progress - An update from Linode about the recent DDoS attacks
    Dec 31, 22:45 UTC
    Scheduled - I’d like to share some updates about the recent DDoS attacks.

    I am one of several network engineers at Linode who have been working around the clock on DDoS mitigation. While things are stable, I would like to take a moment to publicly address the large and frequent DDoS attacks that we have been receiving since Christmas Day.

    It has become evident in the past two days that a bad actor is purchasing large amounts of botnet capacity in an attempt to significantly damage Linode’s business. The following is a partial list of attacks we have received in no particular order:

    - Multiple volumetric attacks simultaneously directed toward all of our authoritative nameservers, causing DNS hosting outages
    - Multiple volumetric attacks simultaneously directed toward all of our public-facing websites, causing Linode Manager outages
    - Layer 7 (“400 bad request”) attacks toward our web and application servers, causing Linode Manager outages
    - Large volumetric attacks toward our colocation provider’s upstream interconnection points, overwhelming the router control planes and causing significant congestion/packet loss
    - Large volumetric attacks toward Linode network infrastructure, overwhelming the router control planes and causing significant congestion/packet loss

    All of these attacks have occurred multiple times. Over the course of the last week, we have seen over 30 attacks of significant duration and impact. As we have found ways to mitigate these attacks, the vectors used inevitably change.

    As of this afternoon, we have mostly hardened ourselves against the above attack vectors, but we expect more to come. We are working extremely closely with all of our technical partners, including our network equipment vendors and our colocation providers, to prevent future attacks.

    Once these attacks stop, we plan to share a complete technical explanation about what has been happening. Additionally, we will be announcing the details of an ongoing project to significantly improve our internet connectivity and resiliency.

    We would like to apologize for the lack of detail in some of our recent status-page updates. Please know that we are dedicating all resources from multiple departments to stopping these attacks. We acknowledge the amount of downtime we’ve been experiencing is completely unacceptable, and we appreciate the understanding and support we have received over the past several days. We will share more information as our investigation continues.

    Alex Forster
    Network Engineer at Linode

  16. #16
    Join Date
    Jan 2006
    Location
    Western New York
    Posts
    89
    Atlanta has been under attack for 8 hours plus now... sites I use there are completely down. Read a ton today and looks Linode has had some major attacks over the years. Is there something they can be doing but not?
    Last edited by Frank Broughton; 01-01-2016 at 09:08 PM.

  17. #17
    Too bad Linode does not provide DDoS protection, even as addon.

  18. #18
    Join Date
    Jan 2006
    Location
    Western New York
    Posts
    89
    Quote Originally Posted by StackNetwork View Post
    Too bad Linode does not provide DDoS protection, even as addon.
    Linode is the one being attacked....

  19. #19
    Quote Originally Posted by Frank Broughton View Post
    Linode is the one being attacked....
    I know that, i mean they do not even have protection for themselves.

  20. #20
    Same problem

Similar Threads

  1. Networkx Completely Down
    By DATARTIM in forum Providers and Network Outages and Updates
    Replies: 2
    Last Post: 09-12-2006, 09:14 PM
  2. Rackspace - London 2 - Down!
    By AH-99P in forum Providers and Network Outages and Updates
    Replies: 4
    Last Post: 07-18-2006, 07:26 AM
  3. Server At PixelFX Completely Down!!
    By chiehkai in forum Providers and Network Outages and Updates
    Replies: 8
    Last Post: 07-05-2006, 08:41 PM
  4. a2hosting.com - completely down?
    By harriska2 in forum Providers and Network Outages and Updates
    Replies: 3
    Last Post: 06-01-2006, 12:11 PM
  5. Pair.com completely down?!
    By fluffybunny888 in forum Providers and Network Outages and Updates
    Replies: 8
    Last Post: 04-01-2006, 10:15 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •