Results 26 to 50 of 139
Thread: SuperMicro IPMI Security
-
10-26-2010, 10:54 AM #26
Yeah, your host will get an actual abuse complaint about you "sending spam", when in fact, it just turns out that one of the spam blacklisters listed your server because it didn't like your hostname or reverse dns. All very confusing since they aren't just up front and tell you what's going on, and instead say "hey you've got a virus / trojan". After working things out with them they say "oh well, there wasn't any actual spam, but 99% of the time, when RDNS or your hostname looks like this, you're an infected spambot, so we just bi**h to your ISP just in case you might someday send spam".
very frustrating, especially if you can't figure out the cause of it all.IOFLOOD.com -- We Love Servers
Phoenix, AZ Dedicated Servers in under an hour
★ Ryzen 9: 7950x3D ★ Dual E5-2680v4 Xeon ★
Contact Us: sales@ioflood.com ★
-
10-26-2010, 01:09 PM #27WHT Addict
- Join Date
- Jul 2008
- Location
- Dallas, TX
- Posts
- 107
I'm quoting this post because it is the most accurate out of anything in this thread. I'm pretty sure the CBL does not list based on your rDNS.
I never knew you could SSH to the BMC before but I gave it a shot on 3 different SM motherboards.
X7DCU (No SSH responding)
X8DTU-F (Running BusyBox v1.1.3)
X8SIL-F (Running ATEN SMASH v1.00)
The IPMI interface you get is one running SMASH from ATEN. If you look on Page 87 of this PDF they have a whole guide on using the SMASH interface to manage the BMC.
I wouldn't put it past someone to possibly hijack one of the BMCs running your standard BusyBox environment but doing this on one running SMASH would prove to be more difficult.
If it was me I would just let the CBL listing expire and ignore it, unless it gets updated. You could also monitor the traffic coming out of that IP as well to see if any spam is actually being sent from it. Another option is to contact support@supermicro.com and have them login to the BMC and take a look.Last edited by Ryan G - Limestone; 10-26-2010 at 01:14 PM. Reason: more info about CBL
-
10-26-2010, 01:26 PM #28Randy
- Join Date
- Aug 2006
- Location
- Ashburn VA, San Diego CA
- Posts
- 4,615
X7DCU (No SSH responding)
X8DTU-F (Running BusyBox v1.1.3)
X8SIL-F (Running ATEN SMASH v1.00)Fast Serv Networks, LLC | AS29889 | DDOS Protected | Managed Cloud, Streaming, Dedicated Servers, Colo by-the-U
Since 2003 - Ashburn VA + San Diego CA Datacenters
-
10-26-2010, 07:13 PM #29Web Hosting Guru
- Join Date
- Nov 2005
- Posts
- 305
What we know: IP running IPMI got blacklisted (cbl provides the blacklisting date (some days ago - IP has just been running IPMI for months).
I was looking for:
1) information on getting a BASH alike shell on the BMC to audit it, but I believe it won't be possible based on Ryan's post (running X8SIE-F - Just able to get that ATEN SMASH prompt)
2) Information about the possibilities/risks of having SuperMicro's IPMI being abused by people who do not have access to it.
After looking around, I believe this may be a CBL mistake.. At first I thought that this could be some vulnerability that could be exploited to send out e-mail (like those common PHP scripts that would get all message information from $_POST without confirming privileges before - I know SuperMicro's IPMI Web Management does not work with PHP, this is just an example).
We are looking around for VPN implementations and are probably going to get all IPMI on internal IPs.
-
10-30-2010, 01:50 AM #30WHT Addict
- Join Date
- Jul 2005
- Posts
- 131
I *just* got notice from my provider tonight that one of my IPMI IPs on an x7spa-hf system spammed AOL. Same ATEN SMASH is available, but I cannot find anything that would have sourced email from the machine. Says it was sourced from ADMIN at the IP, but having a hard time imagining the password was cracked, because then I'd expect it to be changed or other users added. Going to email SM support and see if they know of anything going on. Also making sure the firmware is all up to date. Wish these mainboards had the built in IP restriction that other SM IPMI enabled mainboards get.
*edit* looks like that board was still at fw 1.34, putting it to fw 2.02 and I guess hoping that does the trick. Have an email into SM now about this issue.Last edited by Lockjaw; 10-30-2010 at 02:02 AM.
-
10-30-2010, 02:02 AM #31Web Hosting Master
- Join Date
- Aug 2003
- Location
- /dev/null
- Posts
- 2,132
Too bad. X7SPA-HF is one of those whose IP shares the main NIC (no dedicated interface). This kills any possibility of having the IPMI on a private network, unless NIC1 is set for the private net and NIC2 for the public, which will surely make some users wanting your liver.
-
10-30-2010, 02:19 AM #32WHT Addict
- Join Date
- Jul 2005
- Posts
- 131
IPMI port is off elsewhere in another vlan, and there are no "users" on the machine at the OS level, real or virtualized (gah who'd vm on an atom). That is pretty much the setup, each in separate vlans, etc. I suspect same issue with the X7SPE, but won't know until I need to order and build another machine.
and just got an out-of-office reply from SM o.O
"Hi, I am currently out of office and I will returning on 12/06/2010"
It'd be nice if it didn't forward to *all* their techs heh.
*edit* also I'm thinking if it was really sourced from IPMI, it is entirely possible that having it at shipped-out firmware and not updated might have been an issue. Going through allllll the other Atom IPMI devices, this was actually the only one still at 1.34, the rest were at 2.04 (beta I got from SM in August). Their site still only has 2.02 as officially released, so I'll just stick with my 2.04 beta and update this machine.Last edited by Lockjaw; 10-30-2010 at 02:24 AM.
-
10-30-2010, 03:46 AM #33Temporarily Suspended
- Join Date
- Jan 2004
- Posts
- 147
hello,
it does not solve the exploit, but, what about filtering outbound smtp from those IPMI devices, on the router/switch/firewall ?
example:
access-list 165 deny tcp host 1.2.3.4 any eq smtp
access-list 165 permit ip any any
int ve xx
ip access-group 165 in
et voilá... no spam from 1.2.3.4 to the world anymore.
-
10-30-2010, 07:28 AM #34Web Hosting Guru
- Join Date
- Nov 2005
- Posts
- 305
Now we all are going to agree that there is some vulnerability and that I am not the paranoid guy that thinks SuperMicro's IPMI is vulnerable.
Did you get the headers? Would you forward me (removing your ips/hostname?).
SuperMicro returned to me saying that I should configure outlook not to have those messages tagged as SPAM So I believe you won't get much help there.
These are bad news.. I was going to forget the incident and now we do have to get back planning to move these IPMI IPs to internal addresses
-
10-30-2010, 12:51 PM #35WHT Addict
- Join Date
- Jul 2005
- Posts
- 131
PM'd what I got from my provider, from AOL
I'm just going to move them to IPv6 behind something that handles RA (still wish these atoms took manual IPv6 config instead of just dhcp/ra). I've got v6 access remotely into my racks and I doubt anyone scanning does, or at least I hope they enjoy scanning a /64 looking for the hosts.
*edit* - I think my lesson here is make sure I keep better (read: keep an) inventory of what got what firmware updates, if that was really the cause. It is the only thing different I can see between the other atoms. At least the quad-core Intel systems IPMI chip/version have that IP restriction, now if only those would use IPv6 :/Last edited by Lockjaw; 10-30-2010 at 12:56 PM.
-
10-30-2010, 03:32 PM #36Web Hosting Guru
- Join Date
- Nov 2005
- Posts
- 305
Not sure if it has to do with firmware versions. We have around 30 servers with IPMI and have not upgraded their firmwares. Just one of them has been abused. Maybe the scan robot has not found the other ones yet OR the SPAM has not reached anyone who would send a notification (or SpamTrap).
Based on the headers it seems that IPMI is being used to make the socket connections to other SMTP servers. There is just "one hop" and it is the AOL one.
-
10-30-2010, 03:58 PM #37Randy
- Join Date
- Aug 2006
- Location
- Ashburn VA, San Diego CA
- Posts
- 4,615
Any way to log into these suspect IPMI and check the syslog or other log file? Netstat, ect?
Fast Serv Networks, LLC | AS29889 | DDOS Protected | Managed Cloud, Streaming, Dedicated Servers, Colo by-the-U
Since 2003 - Ashburn VA + San Diego CA Datacenters
-
10-30-2010, 05:43 PM #38Web Hosting Evangelist
- Join Date
- Sep 2004
- Location
- Cluj-Napoca, Romania
- Posts
- 504
X8SIE-LN4F-O replies on port 22: SSH-2.0-dropbear_0.52
-
10-31-2010, 12:04 AM #39Web Hosting Master
- Join Date
- Feb 2004
- Posts
- 2,197
We had a spam report from one of our Supermicro IPMI servers. Its in its own VLAN, so not possible for the IP to be hijacked. We thought it was a spoofed report, so ignored it. It seems more likely it either got brute forced, or there is a vulnerability in the IPMI units.
Will take a closer look on Monday and provide some details as to which motherboard and firmware.
-
10-31-2010, 06:43 PM #40Web Hosting Master
- Join Date
- Feb 2004
- Posts
- 2,197
We had this issue appear on a X8SIL-F motherboard, running firmware 01.29 / 2010-01-26. The SSH port is open, however can't login with the ADMIN user.
-
10-31-2010, 09:16 PM #41Randy
- Join Date
- Aug 2006
- Location
- Ashburn VA, San Diego CA
- Posts
- 4,615
Seems more likely a web interface vulnerability... unless there's a dead simple password for a common user, much less likely an SSH exploit...
Fast Serv Networks, LLC | AS29889 | DDOS Protected | Managed Cloud, Streaming, Dedicated Servers, Colo by-the-U
Since 2003 - Ashburn VA + San Diego CA Datacenters
-
10-31-2010, 09:43 PM #42Web Hosting Guru
- Join Date
- Nov 2005
- Posts
- 305
What makes me worried: Would this vulnerability allow full access to the IPMI interface? Would it allow the intruder to REINSTALL your server ? Poweroff ? Poweron ? If so, this is a really serious vulnerability and all of us should be looking for a fix asap.
-
10-31-2010, 10:56 PM #43WHT Addict
- Join Date
- Feb 2004
- Location
- Australia
- Posts
- 121
Interesting might have to keep a eye on this thread.
Cant supemicro include a built in approved IP list interface which will block all IPs apart from the IPs listed.
-
10-31-2010, 11:18 PM #44WHT Addict
- Join Date
- Jul 2008
- Location
- Dallas, TX
- Posts
- 107
The motherboards that run BusyBox you could easily edit the /etc/hosts.allow as well as probably place an .htaccess in the web directory or modify the interface. The ones running ATEN SMASH your probably a lot more limited because its not your standard Linux environment.
I agree this must be some kind of web exploit if its something.
-
11-01-2010, 11:48 PM #45WHT Addict
- Join Date
- Jul 2005
- Posts
- 131
Got a reply from SM tonight:
Our engineers are working on the issue! We will inform it to you when we get any feedback from them. Please allow them some times for review and detect the issue.
-
11-02-2010, 08:42 PM #46******* Unleaded
- Join Date
- Feb 2004
- Posts
- 3,849
It does not really matter whether it is ssh or http(s).
Anything that has a default admin account that cannot be renamed (aka root), that does not let the port be set arbitarily, and that does not rate limit retries is an incredibly naive security design. The above conditions describes many ipmi implementations.
It is beyond belief that the lessons of the past in addressing the above weaknesses are ignored in contemporary designs.
If an account name is known, the maximum password lenght is known, the port is known, and there is no rate limiting or lockout, brute forcing is trivial. Just fire up a script and let it run until success.edgedirector.com
managed dns global failover and load balance (gslb)
exactstate.com
uptime report for webhostingtalk.com
-
11-08-2010, 11:25 AM #47Web Hosting Guru
- Join Date
- Nov 2005
- Posts
- 305
Any updates?
-
11-08-2010, 11:38 AM #48Randy
- Join Date
- Aug 2006
- Location
- Ashburn VA, San Diego CA
- Posts
- 4,615
Anyone else notice a default ADMIN account with 'anonymous' login credential on the IPMI 2.0 (aka X8SIL-F)?
Fast Serv Networks, LLC | AS29889 | DDOS Protected | Managed Cloud, Streaming, Dedicated Servers, Colo by-the-U
Since 2003 - Ashburn VA + San Diego CA Datacenters
-
11-08-2010, 01:16 PM #49Web Hosting Master
- Join Date
- Jan 2008
- Location
- Jax, FL
- Posts
- 2,707
-
11-08-2010, 01:52 PM #50Web Hosting Evangelist
- Join Date
- Aug 2008
- Posts
- 536
Yes there is but you can disable access for that account.
Similar Threads
-
Supermicro IPMI Issue
By XFactorServers in forum Colocation, Data Centers, IP Space and NetworksReplies: 9Last Post: 08-23-2010, 02:29 PM -
SuperMicro 's IPMI
By Peter-SexyWing in forum Colocation, Data Centers, IP Space and NetworksReplies: 16Last Post: 07-10-2010, 04:51 PM -
supermicro ipmi installation
By phactor in forum Systems Management RequestsReplies: 5Last Post: 04-02-2010, 02:57 PM -
Supermicro IPMI
By opax in forum Colocation, Data Centers, IP Space and NetworksReplies: 6Last Post: 04-29-2009, 12:13 PM -
Supermicro IPMI
By DevelopAl in forum Colocation, Data Centers, IP Space and NetworksReplies: 14Last Post: 03-10-2006, 02:17 PM