How does everyone balance paranoia and realism with regards to server security? Obviously the most secure computer is one that is not connected to any network at all. But when you offer network services (web, email, etc), you are exposed.

We are engaged in internal debate about various intrusion detection systems, router access control lists, user privileges, etc. One team member has suggested that getting hacked is essentially inevitable. We all want to minimize the risks of course, but does anyone else agree with this sentiment?

As a followup, what intrusion detection software do people use (we're Linux-based)?

You're running linux? Yep, getting hacked is inevitable. ;)

More seriously, you should operate on the assumption that there is always a remote root exploit to which your system is vulnerable. Because there is always a delay between when a vulnerability is discovered and when it is announced and patched, to do otherwise would be unreasonably optimistic.

What this means is that you should always set up your systems so that even if someone "breaks root" they can't cover their tracks. BSD securelevel and immutable/append-only files are your friends here.

While you can never say for certain that your system is secure, with proper security you can say for certain that nobody has broken in yet.

The LIDS kernel patch (http://www.lids.org) allows you to control the CAPs of the root user so that even if a cracker 'breaks root', he can't hurt the system too much (cannot kill certain processes or modify certain files), or cover audit trails (append-only logfiles, sending logs directly to other hosts, etc), amongst other things.

If you want the hassle of doing this, that is.

Yes, I was reading about LIDS this morning. Has anyone used it? What about network traffic monitors like snort or portsentry... anyone have experience with either of those, or another?

Finally, I am trying to put together a "security plan", and would appreciate any examples or guidelines of what to include.

Thanks.

I am using iplog, it's pretty need and you definitely get a feeling how "exposed" your machine is...

on my machine I have a couble of traceroutes, pings, port scans etc. every day....

a good start would be to turn off all services you do not need.

Here's how to STOP your servers from getting hacked.

Tell your sysadmins your new policy. If the servers get hacked, you will hack off 1 finger per incident that occurs. I find this method to be most effective! :-) J/K

Heh, yes I saw the article in Web Hosting magazine as well, hence this thread.

The author of that article asks some scary questions -- what do you do when you discover all your production servers that support hundreds of hosting customers have been infiltrated and rootkits, sniffers etc installed? Ug!

As a previous poster said, you're naive if you really think you will NEVER be hacked.

To be honest it doesnt matter what you do to protect yourself, if someone wants to hack your server they will no questions asked. THe tick is to stay low enough key that hackers dont notice you and to stay on top of as many security fixes as possible, but as I said locking your box down tighter than fort nox will not guerentee you wont be hacked..

There's a lot to be said for proactive security - reading BUGTRAQ, staying on top of patches, etc. And as another poster said, using your OS's security features can be a real boon too. FreeBSD's notion of securelevels is a major argument for it in a public server environment. Combined with proper file permissions, it can stop a lot of common things a malicious hacker does in their tracks.

If you get to an operation of any size, I'd say having a box compromised probably is inevitable. However, there are a few things to keep in mind. 90% of malicious hackers out there don't really know much of anything - they're just relying on widely-known security vulnerabilities and pre-built compromise kits. It's pretty trivial to secure against these guys; just stay on top of things. We've been fortunate to have had only one machine compromised in our few year history, but it was through a very well-known vulnerability. No damage was done, but that's not the point. That was a couple years back, and as you might imagine, it was a real wake-up call for us in terms of security policy.

Probably your best overall defense is to have a full-time security person who really gets passionate about it and knows what they're doing. Of course, a lot of us can't afford that. My security consultant - a real pro, and easily the best in the state - happens to be a friend who works for steak. ;) If you can find yourself a similar deal, you'll be in much better shape.

I don't agree that it's inevitable, only that it's possible. Surely it depends on what OS you're running, and certainly how it's configured, how it's monitored and what steps you've taken in more ways than just keeping up to date on the latest/recent exploits, etc. As Travis said, and I agree; "Probably your best overall defense is to have a full-time security person who really gets passionate about it and knows what they're doing."

Let's put it this way, you don't even need a very secure server, in most cases, if someone's there actively monitoring it, having the system do checking and implementing some good policies, etc. However, provided that you just make sure that no exploits can be done remotely or locally that can be done in such a small amount of time, and you keep an eye out, you can effectively make the chance of compromise basically nil.

That's to say, even with an insecure server, how many system crackers do you think are going to try and break in, when they can see that there's someone there, at that moment, monitoring the system, killing their process as soon as they see it, which could be almost immediately, having alerts and notices for the administrator(s) if anything is different, anyone gains any sort of access they shouldn't be, etc. As Travis also said, 90% of these people don't really know anything about it anyway.... most use known exploits and tools and then other tools to cover their tracks. There's not that many people that can realistically go through a system, intelligently trying things.

If you monitor, log and configure the system intelligently, only a very small percentage of system crackers can work with it. And, if you implement a means for most of the things left, those changes are very much diminished. Again, actively monitoring the server, I couldn't imagine too many people trying to break in while people are basically sitting there, watching, waiting. It doesn't have to be a full time 24/hour a day job to just monitor -- I'd like to see anyone stare straight at the screen for that long -- but just a well implemented policy. I don't believe it's just bound to happen, but the more user's you have on a system or network, with more or less access than others, bigger profile sites, etc. surely all contribute, but don't make it inevitable. I've seen servers never compromised, and some that are often tried to be broken into.

Just make sure you stay a step ahead of the general public, with patches, work around's, actively being there watching, logging and sending this information off site, wrappers, filters and anything to either stop or slow the process of gaining root, to be able to take immediate action to kill their processes and secure whatever means they used to gain that access.