Preface:
After reading a few posts here about being hacked I elected to compose this short guide.
I won't profess to be a security expert by any means. While I'm sure industry collegues might be able to add to this I hope to offer a good starter in the hopes that other admins don't suffer losses like I did in my early days a few years ago.
My familiarity with Nix based systems is limited to the RedHat line thus my instructions will work on such systems but I can't count that they will on other Nix flavours. (although they should for the most part.)
This is directed to system administrators, not website administrators. If you are a website administrator and you feel you or your server has been hacked I urge you to backup all that you can from your site and contact your system/network administrator with what ever evidence you have to support your concerns.
Starters:
You will need SSH and Root access to your server to complete this. Have note pad open and prepare to copy and paste entries which point to the illegal activity for later reporting. Coffee and Valum are optional.
Suspicions, have I been hacked?
Check your bandwidth usage.
As a system administrator you should have a general idea of what your using for bandwidth. Tools such as MRTG are excellent to show your bandwidth trends. Excessive usage indicates that someone has comprimised your server to distribute warez or porn which is a very common purpose for hackers.
Check your disk usage:
[root@yourserver root]# df
Your output should be similar to:
Code:
Filesystem 1k-blocks Used Available Use% Mounted on
/dev/hda3 38376872 3461780 32965608 10% /
/dev/hdb1 19750244 3937284 14809704 22% /archive
/dev/hda1 101089 9043 86827 10% /boot
none 515612 0 515612 0% /dev/shm
We're looking for unusual usage here. As the admin you should have an idea of what your using.. If your numbers looks strange try to track down larger files and see whats up.
[root@yourserver root]# find '/' -size +50000k
This will seek out files on your server larger than 50 Megs. If thats normal adjust the number accordingly. Pirate CD's will be in the 500+ Meg range where as video files of quality are 50M - 1+ Gig in size.
Alternatly, you can look for files with common file extensions such is .bin, .rar, .iso etc....
[root@yourserver root]# find '/' -iname *.rar
This would find all files on your server with the .rar extension. Take note of the odd files and thier directories.
Run Chkrootkit
I scribbled a guide on installing and running chkrootkit here on WHT:
http://www.webhostingtalk.com/showth...hreadid=228109
Google any results you get for information/solutions on the particular trojan you've been infected with.
If so far you've gotten confirmation that thigns aren't right we need to press on.
Find out whos been logging in.
[root@yourserver root]# last | more
This will output what username from what location has logged into your server when.
Determine if your logs are running.
Hackers love to shut the loggers off to help cover thier tracks.
[root@yourserver root]# ps -aux | grep syslogd
You should see a line that looks like:
Code:
root 695 0.0 0.0 1460 536 ? S Jan05 0:35 syslogd -m 0 -a /
If you don't thats likely bad. Restart it:
[root@yourserver root]# /etc/rc.d/init.d/syslog restart
Find suspicious accounts/groups
We're looking for suspicious users. gamerz, pron, adm1n, test, qwerty and the like.. Anything that looks out of place. Before editing this file check with other servers or admins to see what thiers looks like. Some users such as shutdown, games, operator and the like are are normal upon install.
[root@yourserver root]# less /etc/passwd
The 1st column are your users. (tip "q" exits a less screen)
[root@yourserver root]# less /etc/shadow
Same deal, look for funky users.
[root@yourserver root]# less /etc/group
More of the same, looking for funky groups.
Ok, so if we've found and determined that we have some funkiness going on, it's time to get rid of the users. If you've seen interesting logins in (last | more) you would have found them in these files.
[root@yourserver root]# vi /etc/passwd
Scroll down to funky user and delete the entire line. Repeat this on all 3 of the files (passwd, shadow, groups) until all traces of un-authorized users are gone.
Shutting them down
We want to see if hacker has edit our crontab to run thier programs etc..
[root@yourserver root]# crontab -e
Should be empty for the most part. If you see something in here that doesn't belong, again check with other administrators/servers to see if in fact it belongs, if not - delete the line.
Now, check the history of the users to find out what he did to your box.
[root@yourserver root]# find '/' -iname .bash_history
If your hacker wasn't thorough they may have left you this gift. Edit the file in your favorite editor and copy the entire contents to your notepad. This is what the person was doing on your box. You can compare this to the information given in last | more to put the actions to the users IP/Hostname, this information should be submitted to the authorities to persue prosecution.
Now, lets go through and remove all of the files we no longer want on our sever. Using the notes taken during our file search remove the files.
At this point we've shut out the bad guys, and remove thier files from the system and also have notes and logs of who they were.. Retribution might be on the horizon. 1 Problem though; your system is still vulnerable. The hacker did afterall get in initially so that means they or others can again as well.. You can do a security lockdown (I'll write that up another night) but that possibly won't erase all traces.
We can't be certain at this point that we've really shut this person down no more then we can ensure that your server is 100% safe. Ultimatly, your best bet is to grab backups of your websites and re-install. Take a day or 2 hoping that our shut down here has scared the hacker away from your box but no more as you are open to the next hacker in line.
After a successful re-installation/migration/restore or whatever you do take the time to really secure your box. I will write what I know of doing this in another thread but you may want to consider hiring an industry professional to do this for you if your not comfortable with it.
I hope this is helpful to some, and I'm eager to see what other administrators can add to this write up so that I can learn more as well !
Best regards and safe serving.
Ryan
)
Linear Mode
