Results 1 to 25 of 61
-
06-26-2014, 03:21 PM #1Problem Solver
- Join Date
- Mar 2003
- Location
- California USA
- Posts
- 13,681
IPMI -- public ips -- What the hell are you thinking?
<rant>
For the past year if not longer there has been a string of IPMI exploits. Hosts have been warned repeatedly to take IPMI off the public internet yet there is numerous providers who have not done this.
I started getting emails last night/today from providers warning about the newest exploit. Now mind you this is DAYS after it was announced.. Come on seriously?
What is wrong with you guys? Do you have any care or respect for your customers?
Do some searching on various forums, there is a bunch of hosting companies who have had their servers wiped clean due to this most recent exploit in the past few days.
If you offer dedicated servers, and ipmi that is still to this day publicly accessible. I will openly say you are a company no one should ever host with.. because you are flat out clueless.
Its really not that hard to limit this stuff to a VPN.
</rant>Last edited by Steven; 06-26-2014 at 03:27 PM.
-
06-26-2014, 03:38 PM #2
I liked what someone else said in the exploit thread, went something like "Get your IPMI accessible on from a VPN and be much better off, those customers that dont like this are customers you probably dont want." Which I think made a lot of sense.
-
06-26-2014, 03:41 PM #3Sam is here
- Join Date
- Mar 2010
- Posts
- 822
Thanks for your post , fortunately I was aware of the exploit before it get announced on the websites, 7 - 8 days ago , so we sent email to our colo customers and ask them to upgrade their firmware if the IPMI is accessible over public IP address . for dedicated servers , first we blocked the port 49152 on private network and started to upgrade firmwares and we get it done yesterday .
We had zero report for a problem even for colo customers ...
-
06-26-2014, 03:52 PM #4Problem Solver
- Join Date
- Mar 2003
- Location
- California USA
- Posts
- 13,681
I repeat. If ipmi is on public ips and you offer dedicated servers you are doing your customers an injustice.
I dont care if you blocked ports or whatever. There will likely be more exploits.
Its time for you to be proactive. Only a matter of time before there is a private 0 day that you can't protect against.
-
06-26-2014, 03:54 PM #5Randy
- Join Date
- Aug 2006
- Location
- Ashburn VA, San Diego CA
- Posts
- 4,615
With regards to VPN being the end-all-be-all... humor me this -- What prevents a rouge customer/signup from hijacking servers from the inside? Many if not most of the IPMI VPN solutions I've seen allow customers access to more than their own IPMI, e.g. large flat private network ready for easy mass exploit. I surely hope strong ACL's are implemented in any setup, and certainly worth investigating beyond 'oh we have a VPN, we're good'.
-
06-26-2014, 03:56 PM #6Problem Solver
- Join Date
- Mar 2003
- Location
- California USA
- Posts
- 13,681
Agreed. But there is plenty of hosts that do not even give out access yet are on public ranges.
Getting off public ranges is very important. Being public is just as easy if not easier than a vpn compromise.
Aka zmap of the internet.
You can make a home brew vpn solution that ties ipmi user and pass into vpn auth.. then you can restrict ip access... It is not that hard.
Ionity has a similar setup with their SRX works great.
Definately not rocket science to make something secure.Last edited by Steven; 06-26-2014 at 04:01 PM.
-
06-26-2014, 03:59 PM #7Sam is here
- Join Date
- Mar 2010
- Posts
- 822
-
06-26-2014, 04:01 PM #8Randy
- Join Date
- Aug 2006
- Location
- Ashburn VA, San Diego CA
- Posts
- 4,615
True...VPN will prevent drive-by attacks. However the recent mass-deletion event appears to be the targeting a specific provider. VPN is one extra step for a determined attacker, seeking to discredit their competition. I really hope that everyone takes a good look at their current setup, VPN or not.
-
06-26-2014, 04:02 PM #9Problem Solver
- Join Date
- Mar 2003
- Location
- California USA
- Posts
- 13,681
-
06-26-2014, 04:02 PM #10
-
06-26-2014, 04:04 PM #11
I have to completely disagree with your assertion that "it's not that hard". It's absolutely positively 100% WORTH THE EFFORT, but to claim it's not hard is completely wrong. It took our lead technician a lot of hours to get a vpn solution set up that he was happy with from a security and functionality standpoint. Time well spent, but I can understand why others have dragged their feet up to this point. That said, difficult or not, it absolutely must be done.
-
06-26-2014, 04:05 PM #12Randy
- Join Date
- Aug 2006
- Location
- Ashburn VA, San Diego CA
- Posts
- 4,615
-
06-26-2014, 04:10 PM #13
Given the fact that being able to access the IPMI login page is equivalent to giving someone root access on a server, after the string of exploits that were published a few months back, we decided we couldn't give clients any access to IPMI at all without something at least that secure.
-
06-26-2014, 04:13 PM #14Problem Solver
- Join Date
- Mar 2003
- Location
- California USA
- Posts
- 13,681
-
06-26-2014, 04:21 PM #15Problem Solver
- Join Date
- Mar 2003
- Location
- California USA
- Posts
- 13,681
A few months ago. Someone who hangs out in a chat room I frequent did a global scan for ipmi.
What he found was thousands of ipmi intances with admin/admin default login. Our ids has been seeing hundreds of scans per day on our tiny net blocks (/22x2). Which just recently ramped up.
What I don't think some people realize is how bad this is going to get.
-
06-26-2014, 04:44 PM #16Web Hosting Guru
- Join Date
- Jul 2011
- Posts
- 281
I'd go a step further and say that *all* of your management should be on an isolated management network that's not internet accessible without a VPN or otherwise being on-site. I don't expose hypervisors to the Internet, I don't expose internal tracking tools or out of band management or anything of the sort. Does it make certain problems (like console access for customers) a bit harder to solve? Sure. Are those problems solvable? Absolutely - it really wasn't that hard. It absolutely boggles my mind that there are people out there with their IPMI/DRACs/iLOs/etc on the public Internet. That's just... extremely lazy. I don't know how else to put it.
-
06-26-2014, 05:19 PM #17Corporate Member
- Join Date
- Aug 2004
- Location
- Kauai, Hawaii
- Posts
- 3,799
-
06-26-2014, 05:40 PM #18The VPS Specialist
- Join Date
- Aug 2003
- Location
- Edinburgh/London
- Posts
- 5,789
Semoweb too I think?
-
06-26-2014, 05:43 PM #19Problem Solver
- Join Date
- Mar 2003
- Location
- California USA
- Posts
- 13,681
-
06-26-2014, 05:47 PM #20Web Hosting Master
- Join Date
- Mar 2009
- Location
- CA
- Posts
- 9,350
-
06-26-2014, 08:21 PM #21Problem Solver
- Join Date
- Mar 2003
- Location
- California USA
- Posts
- 13,681
-
06-26-2014, 09:01 PM #22Web Hosting Master
- Join Date
- Mar 2009
- Location
- CA
- Posts
- 9,350
-
06-26-2014, 09:58 PM #23Junior Guru
- Join Date
- Dec 2013
- Posts
- 194
Here's an important question for you: Are the IPMI controllers isolated from each other as well? If you're giving someone access to the web UI (or even if you're not), it's fairly easy to gain shell access to it. If you haven't secured the IPMI controllers from each other, you could then start attacking other machines.
Granted, it requires a bit more knowledge to exploit, but it's not impossible.
-
06-26-2014, 10:03 PM #24Web Hosting Master
- Join Date
- Aug 2007
- Location
- L.A., CA
- Posts
- 3,710
Thats why any real company now a days needs 2 networks.
Public and Private.
Each client would have a public VLAN and a private VLAN.
The private VLAN would contain IPMI and any inter-server internal connectivity and isolate each client to their own private, secure LAN.
This means that you need redundant public routers (for internet access) and redundant private routers (for the private network).
Besides QuadraNet, I don't know any other larger volume host here on WHT that actually has a legitimate private network and supplies clients with both a public vlan and a private vlan as a default offering across their server range.
Feel free to pitch in names if you are 100% positive they operate private network with private vlan per customer.
-
06-26-2014, 10:34 PM #25WHT Addict
- Join Date
- Feb 2014
- Posts
- 103
Not everybody is rich and tech savvy clients to get into complicated ipmi setup, with recent events i guess one has to secure ipmi network and probably rather for those who live in countries where vpn itself is banned or the customer who is has hassle of extra step of getting around complex vpn setup or when he is outside or when he is on guest machine with no admin privilege, and need emergency access, all would filter down to support ticket system, where customer would seek help, all the reason for IPMI was to get down or save money effort on support system so that client is doing self service. and ask support for only complex matters,
there is no need to say what the hell is wrong with you keeping ipmi on public lan, because only few people have static ip even if we need to use access list, and vpn being a luxury/complex for say 50% of customers, and providers themself do not have luxury to loose that percentage of customers these days like someone said they are better off not having such customers. Provider perspective it is extra effort but only a one time setup and bit of extra management which is better than dealing with emergency situations.
Similar Threads
-
IPMI safe (secure) to use over public / WAN IP?
By john2k in forum Colocation, Data Centers, IP Space and NetworksReplies: 12Last Post: 08-16-2013, 09:56 PM -
Number of public IPs
By preciouspigsystems in forum Hosting Software and Control PanelsReplies: 5Last Post: 01-13-2013, 02:41 AM -
VM template without public IPs
By WEBCRAFT-TUCKY in forum Hosting Security and TechnologyReplies: 4Last Post: 05-08-2011, 04:11 PM -
Juniper SSG5 and Public IPs
By zynfella in forum Colocation, Data Centers, IP Space and NetworksReplies: 0Last Post: 12-19-2007, 08:59 AM -
Godaddy's thinking about going public....
By Glanhosts in forum Web Hosting LoungeReplies: 8Last Post: 07-17-2004, 10:46 PM