Results 1 to 38 of 38
  1. #1
    Join Date
    Mar 2008
    Location
    /usr/bin/kvm
    Posts
    248

    * Hetzner Got Hacked

    Dear Client

    At the end of last week, Hetzner technicians discovered a "backdoor" in one
    of our internal monitoring systems (Nagios).

    An investigation was launched immediately and showed that the administration
    interface for dedicated root servers (Robot) had also been affected. Current
    findings would suggest that fragments of our client database had been copied
    externally.

    As a result, we currently have to consider the client data stored in our Robot
    as compromised.

    To our knowledge, the malicious program that we have discovered is as yet
    unknown and has never appeared before.

    The malicious code used in the "backdoor" exclusively infects the RAM. First
    analysis suggests that the malicious code directly infiltrates running Apache
    and sshd processes. Here, the infection neither modifies the binaries of the
    service which has been compromised, nor does it restart the service which has
    been affected.

    The standard techniques used for analysis such as the examination of checksum
    or tools such as "rkhunter" are therefore not able to track down the malicious
    code.

    We have commissioned an external security company with a detailed analysis of
    the incident to support our in-house administrators. At this stage, analysis
    of the incident has not yet been completed.

    The access passwords for your Robot client account are stored in our database
    as Hash (SHA256) with salt. As a precaution, we recommend that you change your
    client passwords in the Robot.

    With credit cards, only the last three digits of the card number, the card type
    and the expiry date are saved in our systems. All other card data is saved
    solely by our payment service provider and referenced via a pseudo card number.
    Therefore, as far as we are aware, credit card data has not been compromised.

    Hetzner technicians are permanently working on localising and preventing possible
    security vulnerabilities as well as ensuring that our systems and infrastructure
    are kept as safe as possible. Data security is a very high priority for us. To
    expedite clarification further, we have reported this incident to the data
    security authority concerned.

    Furthermore, we are in contact with the Federal Criminal Police Office (BKA) in
    regard to this incident.

    Naturally, we shall inform you of new developments immediately.

    We very much regret this incident and thank you for your understanding and
    trust in us.

    A special FAQs page has been set up at
    http://wiki.hetzner.de/index.php/Security_Issue/en to assist you with further
    enquiries.

    Kind regards

    Martin Hetzner
    - Founder of Backupsy, VPSDime, Winity

  2. #2
    Join Date
    Aug 2007
    Location
    Belgium
    Posts
    4,183
    Let's hope they can find a solution soon. It's not Hetzner that's hacked, it's an issue with Nagios
    » www.InstantDedicated.com - Online in no time
    » Dedicated Servers in [EU] Netherlands with DAILY support, also on weekends
    » DDOS Protected network - 100% Money Back if it doesn't work for you
    » Streaming / IPTV allowed | Up to 10 Gbit ports | 100% Network Uptime

  3. #3
    Join Date
    Nov 2011
    Location
    Calgary, Alberta, Canada
    Posts
    699
    Thank god I didn't go with them. Considering they needed government issued identification to verify I am who I am and now that information could've been in the hands of some hacker...
    Little Apps
    Open Source Software

  4. #4
    Join Date
    Mar 2009
    Posts
    389
    And now the entire hetzner network is down!
    EDIT : Back up now, was down for a minute.

  5. #5
    Join Date
    Jan 2011
    Location
    Varna, Bulgaria
    Posts
    1,270
    Again? When / what was the previous one?

  6. #6
    Join Date
    Mar 2008
    Location
    /usr/bin/kvm
    Posts
    248
    Quote Originally Posted by 24x7group View Post
    Let's hope they can find a solution soon. It's not Hetzner that's hacked, it's an issue with Nagios
    Yes, Nagios got hacked, not Hetzner. All user info is safe.
    - Founder of Backupsy, VPSDime, Winity

  7. #7
    Join Date
    Mar 2003
    Location
    California USA
    Posts
    13,294
    The malicious code used in the "backdoor" exclusively infects the RAM. First
    analysis suggests that the malicious code directly infiltrates running Apache
    and sshd processes. Here, the infection neither modifies the binaries of the
    service which has been compromised, nor does it restart the service which has
    been affected.
    This is a fairly poor explanation.
    Steven Ciaburri | Industry's Best Server Management - Rack911.com
    Software Auditing - 400+ Vulnerabilities Found - Quote @ https://www.RACK911Labs.com
    Fully Managed Dedicated Servers (Las Vegas, New York City, & Amsterdam) (AS62710)
    FreeBSD & Linux Server Management, Security Auditing, Server Optimization, PCI Compliance

  8. #8
    Join Date
    Aug 2011
    Location
    Denmark
    Posts
    108
    Honestly, I do not find the content of the above message alarming as such. Simply update your password and you should be fine.

    However if you had them access any of your server(s) in recent times, I would check those servers as well just in case.
    The malicious code used in the "backdoor" exclusively infects the RAM. First
    analysis suggests that the malicious code directly infiltrates running Apache
    and sshd processes. Here, the infection neither modifies the binaries of the
    service which has been compromised, nor does it restart the service which has
    been affected.
    Just my 0.02 - I would love to hear what others have to say though.
    "Learn from yesterday, live for today, hope for tomorrow. The important thing is not to stop questioning." - Albert Einstein

  9. #9
    Ouch. But they atleast don't try to cover it up and most releavant details was given before they get asked about. Some people should take this as example when things go wrong...

  10. #10
    Join Date
    Mar 2009
    Posts
    389
    Quote Originally Posted by rds100 View Post
    Again? When / what was the previous one?
    Last year..

  11. #11
    Join Date
    Aug 2011
    Location
    Denmark
    Posts
    108
    Quote Originally Posted by softshop011 View Post
    Last year..
    Link to this? I am curious and would like to know more.
    "Learn from yesterday, live for today, hope for tomorrow. The important thing is not to stop questioning." - Albert Einstein

  12. #12
    Join Date
    Aug 2011
    Location
    Denmark
    Posts
    108
    "Learn from yesterday, live for today, hope for tomorrow. The important thing is not to stop questioning." - Albert Einstein

  13. #13
    Join Date
    Mar 2009
    Posts
    389
    Quote Originally Posted by MrEliasen View Post
    Link to this? I am curious and would like to know more.
    Looking for the link, thorough analysis of the compromise was posted by German IT specialist.

    Tobias Huch wrote about the compromise/data breach :
    http://www.golem.de/1110/86916.html
    http://www.netzwelt.de/news/88855-in...r-hetzner.html

    There's also a thread on WHT
    http://www.webhostingtalk.com/showthread.php?t=1088324
    Last edited by softshop011; 06-06-2013 at 01:22 PM.

  14. #14
    Join Date
    Aug 2011
    Location
    Denmark
    Posts
    108
    Quote Originally Posted by softshop011 View Post
    Looking for the link, thorough analysis of the compromise was posted by German IT specialist.

    Tobias Huch wrote about the compromise/data breach :
    http://www.golem.de/1110/86916.html
    http://www.netzwelt.de/news/88855-in...r-hetzner.html

    There's also a thread on WHT
    http://www.webhostingtalk.com/showthread.php?t=1088324
    Appreciated, thanks!
    "Learn from yesterday, live for today, hope for tomorrow. The important thing is not to stop questioning." - Albert Einstein

  15. #15
    Quote Originally Posted by softshop011 View Post
    And now the entire hetzner network is down!
    EDIT : Back up now, was down for a minute.
    Yes, even I noticed the same.

  16. #16
    Join Date
    Mar 2006
    Location
    Servers
    Posts
    1,588
    Seems their monitoring is sending now some fake alarms of port 80 down ...
    QHoster.com - Web Hosting with DDoS Protection | Shared & Reseller in Europe/North America
    Linux/Windows RDP VPS 13 Locations : UK, US (5 states), Mexico, Canada, Bulgaria, Lithuania,
    Italy, France, Germany,Netherlands, Switzerland, Rissia, Singapore | OpenVPN/PPTP Enabled
    INSTANT | PayPal, Skrill, Payza, Bitcoin, WebMoney, Perfect Money, Ukash, CashU, paysafecard

  17. #17
    Join Date
    Sep 2008
    Location
    Seattle, WA
    Posts
    1,268
    Quote Originally Posted by serverian View Post
    Yes, Nagios got hacked, not Hetzner. All user info is safe.
    With credit cards, only the last three digits of the card number, the card type
    and the expiry date are saved in our systems. All other card data is saved
    solely by our payment service provider and referenced via a pseudo card number.
    Therefore, as far as we are aware, credit card data has not been compromised.
    Last 3 digits and exp of credit cards compromised, I would say that is user info and probably not all of it.

    edit. I could be mistaken in this, they didn't state clearly if this is just "informational" or this info is compromised.
    Last edited by StealthyHosting; 06-06-2013 at 01:34 PM.
    █ Brian Kearney, Stealthy Hosting Inc. Seattle, WA [AS54931] Skype: StealthyHosting
    Affordable Dedicated Servers
    Remote Hands Colocation

    █ Email: [email protected] Phone: 253-880-1233

  18. #18
    Join Date
    Mar 2008
    Location
    /usr/bin/kvm
    Posts
    248
    Quote Originally Posted by StealthyHosting View Post
    Last 3 digits and exp of credit cards compromised, I would say that is user info and probably not all of it.
    That was sarcasm
    - Founder of Backupsy, VPSDime, Winity

  19. #19
    Join Date
    Nov 2006
    Location
    search.php?do=getnew
    Posts
    1,238
    Threads merged, here's being hopeful for no new ones springing up.

  20. #20
    Join Date
    May 2003
    Location
    Scotland
    Posts
    3,728
    I don't see any unusual activity directed towards my Hetzner equipment at the moment, so hopefully they did not get too much.

    Good to see they came straight out with it though and did not try to hide anything, kudos for that.

  21. #21
    Join Date
    Jun 2012
    Posts
    399
    Quote Originally Posted by serverian View Post
    Yes, Nagios got hacked, not Hetzner. All user info is safe.
    Really?

    An investigation was launched immediately and showed that the administration
    interface for dedicated root servers (Robot) had also been affected. Current
    findings would suggest that fragments of our client database had been copied
    externally.

  22. #22
    Join Date
    Jan 2008
    Location
    Portugal
    Posts
    995
    Lol jeez, I hope Hetzner will get their security issues sorted!

    Sad to see this.
    Miguel Ângelo - Marketing/PR Bachelor - Need Marketing/IT consultation? Contact me for a quote.
    BackupInstance - Backup Servers and Advanced Solutions/Clustering/High-End Server Configurations

  23. #23
    Join Date
    Dec 2010
    Location
    127.0.0.1
    Posts
    5,387
    Well at least they aren't hiding it like when Godaddy's DNS was being ddosed and they denied it blamed it on hardware failure.

    Anyway I hope their customers server's haven't been touched.
    Licensecart - We only sell High Quality licenses
    Visit us @ Licensecart.com ~ sales(➾)licensecart.com ~ webhostbundle.com
    WHT isn't what it used to be… power changes people :]

  24. #24
    I found this part most interesting:

    " First analysis suggests that the malicious code directly infiltrates running Apache and sshd processes"

    There was this issue with hacked linux root accounts this year:

    http://www.webhostingtalk.com/showthread.php?p=8702404

  25. #25
    Join Date
    May 2013
    Location
    Florida
    Posts
    418
    I agree. I like that they came right out and admitted everything without trying to make excuses or cover anything up
    Webhostpython.com - Reliable Shared, Reseller, and KVM VPS Hosting Services.
    Dual Octa Core Xeon E5 Servers. RAID10 Storage. Enterprise DDOS Protection. Pure SSD Plans
    24/7 Support | Live Chat | In-House Support Staff | 1-800-929-9061 | Dallas, TX

  26. #26
    Join Date
    Mar 2004
    Posts
    695
    On 1st June i got my credit card cancelled by the bank, the explanation i got on monday was that some company i was purchasing from was compromised.

    But on 1st June i also got a notification from 100tb about a security problem.

    Both companies had a security issue last week, both companies say credit card data wasn't compromised but a warning was sent to banks to block cards.

    I think one of the companies is not telling the truth.

    Has your credit card been blocked or cancelled because of a security issue these days? are you a customer of Hetzner or 100tb.com? (i'm customer of both)
    Mousa: [as Rambo prepares to play Afghan game 'buzkashi'] God must love crazy people.
    Rambo: [getting on horse] Why?
    Mousa: He make so many of them!

  27. #27
    Join Date
    Oct 2011
    Posts
    188
    I just want to know one thing, is our card info safe?

  28. #28
    Join Date
    Jul 2010
    Posts
    38
    Quote Originally Posted by elmister View Post
    On 1st June i got my credit card cancelled by the bank, the explanation i got on monday was that some company i was purchasing from was compromised.

    But on 1st June i also got a notification from 100tb about a security problem.

    Both companies had a security issue last week, both companies say credit card data wasn't compromised but a warning was sent to banks to block cards.

    I think one of the companies is not telling the truth.

    Has your credit card been blocked or cancelled because of a security issue these days? are you a customer of Hetzner or 100tb.com? (i'm customer of both)
    Hi,

    I'm an customer of Hetzner. My cc have not been blocked or cancelled.

  29. #29
    Join Date
    Jan 2010
    Location
    Lithuania
    Posts
    1,089
    Can anyone post a link to detailed analysis of nagios vulnerability explanation?
    Time4VPS - flexible, worry-free, fast and affordable VPS hosting in Europe.

  30. #30
    Join Date
    Mar 2003
    Location
    WebHostingTalk
    Posts
    16,960
    Quote Originally Posted by HaronMedia View Post
    Really?

    An investigation was launched immediately and showed that the administration
    interface for dedicated root servers (Robot) had also been affected. Current
    findings would suggest that fragments of our client database had been copied
    externally.
    Yes, and that is because of Nagios.
    Specially 4 You
    .
    JoneSolutions.Com ( Jones.Solutions ) is on the net 24/7 providing stable and reliable web hosting solutions and services since 2001

  31. #31
    Join Date
    Mar 2009
    Posts
    389
    Hetzner should provide more information! If the attackers had access to robot, they also had access to 'rescue' system! If they were after a specific server they had access to :

    Reboot any server in rescue more, mount hard drives, steal data, reboot back into normal mode

    The only giveaway would have been an automated email sent to the owner when a reboot request is performed via robot.

    So now to my question, did anyone receive an unauthorized robot reboot request email in the last couple of months ? (Subject : Automatischer Reset Ihres Servers #xxxxxx)

  32. #32
    Join Date
    Dec 2007
    Location
    UK
    Posts
    948
    It almost seems no website or database that is online is ever safe of being hacked. This just proves that the technologies we rely on, whether it be a billing system, or in this case a monitoring system, leave us as businesses vulnerable to attack.
    Follow me on Twitter: @conrjac

  33. #33
    Hi
    It is a threat to our database

  34. #34
    Join Date
    May 2011
    Location
    /root
    Posts
    598
    Wow, although I am very unware on how the Robot system works or how their servers are monitored etc but I believe the explanation given here is very vague. However, I find it pretty astonishing that Nagios was actually connected to the billing and client data. Thats pretty complex...
    || Tecsys Solutions LLC | Outperforming the Performers!! ||
    || Outsourced Server Management and Technical Support Solutions ||
    || Now Offering Secure Managed VPS and Dedicated Servers specially setup for Hosting Providers ||
    || https://www.24x7TechnicalSupport.net ||

  35. #35
    Join Date
    Mar 2005
    Location
    New York City
    Posts
    2,559
    Quote Originally Posted by tecsys View Post
    Wow, although I am very unware on how the Robot system works or how their servers are monitored etc but I believe the explanation given here is very vague. However, I find it pretty astonishing that Nagios was actually connected to the billing and client data. Thats pretty complex...
    That's exactly what I was thinking. I don't know how they connected the systems, but you would think an API would be in place to keep the systems isolated. One system being compromised should have never been able to affect the other. At worst, infecting Nagios should have leaked some uptime statistics...
    Matthew Rosenblatt, and I do lots of things.
    Currently a Master Electrician on Broadway.
    My company, BurstAV, specializes in A/V Systems Design and integration.
    I also own ConcertCables. We build power/data cables for the entertainment industry.

  36. #36
    Hello
    I'm wondering if Hetzner was hit by Cdorked.A. I see some similarities.

    "The malicious code used in the backdoor exclusively infects the RAM. First analysis suggests that the malicious code directly infiltrates running Apache and sshd processes"

    ESET:

    "As mentioned before, Linux/Cdorked does not write any files on the disk. Instead, it allocates around six megabytes of shared memory to keep its state and configuration information.

    Rif: http://www.welivesecurity.com/2013/0...rves-blackhole

    Regards.

  37. #37
    Join Date
    Mar 2009
    Posts
    389
    Quote Originally Posted by servermanaged View Post
    Hello
    I'm wondering if Hetzner was hit by Cdorked.A. I see some similarities.
    Nice catch, the backdoor most likely was Cdorked.A .. I wonder how they got infected in the first place and if robot/customer data was actually extracted from their db servers..

  38. #38
    Join Date
    May 2013
    Location
    USA
    Posts
    928
    Kudos for salting the password hashes, but I bet they're wishing they used a better hashing algorithm. SHA256 has long been insufficient for password protection--hashcat can do over a billion SHA256 hashes per second on a Radeon HD7970. That'll crack any password under 10 characters in not much time at all.
    ▄▀▄ Brian Harrison, Lead Engineer - Reprise Hosting (AS62838)
    ▄▀▄ Deals on cheap dedicated server hosting. IPMI included! Unmetered bandwidth.
    ▄▀▄ Website migration, 24/7/365 support, basic server setup, 15 day money back.
    ▄▀▄ Looking for DEALS on self-managed cheap VPS hosting? Visit VPSHostingDEAL.com

Similar Threads

  1. Staminus /Hetzner /Webtropia Reviews (Bad) Except hetzner
    By gmakhs in forum Dedicated Server
    Replies: 13
    Last Post: 12-03-2012, 02:18 PM
  2. Hacked? We Guarantee Your Website Will Not Be Hacked or Defaced!
    By UNIXy in forum Hosting & Network Security
    Replies: 1
    Last Post: 05-28-2012, 03:09 PM
  3. hetzner review , hetzner company or hetzner swindler
    By fun_9990 in forum Dedicated Server
    Replies: 65
    Last Post: 03-03-2012, 01:11 AM
  4. Hetzner robot apparently hacked
    By wartungsfenster in forum Providers and Network Outages and Updates
    Replies: 1
    Last Post: 10-06-2011, 12:53 PM
  5. Can my blog be hacked on shared hosting if my neighbour is hacked?
    By zobe in forum Hosting Security and Technology
    Replies: 17
    Last Post: 03-10-2011, 04:09 AM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •