View Full Version : extremely high cpu & memory load ... help please!!!
an hour ago cpu load was 0.03 and memory used was 36% ... but now to cpu load is 2.02 and memory (red light) 96%!!! is my server being attacked by hackers or is there any error with the control panel or what might have happened? ... i have rebooted the server and check the bandwidth usage ... traffic/bandwidth is normal and i dont have any heavy loaded script running ... what should i do now? help please ... :(
i run "ps -ax" and found out that :
10676 ? S 0:00 /usr/sbin/httpd -f /etc/httpd/conf/httpd.conf
10677 ? S 0:00 /usr/sbin/httpd -f /etc/httpd/conf/httpd.conf
10678 ? S 0:00 /usr/sbin/httpd -f /etc/httpd/conf/httpd.conf
10679 ? S 0:00 /usr/sbin/httpd -f /etc/httpd/conf/httpd.conf
10680 ? S 0:00 /usr/sbin/httpd -f /etc/httpd/conf/httpd.conf
10681 ? S 0:00 /usr/sbin/httpd -f /etc/httpd/conf/httpd.conf
10682 ? S 0:00 /usr/sbin/httpd -f /etc/httpd/conf/httpd.conf
10683 ? S 0:00 /usr/sbin/httpd -f /etc/httpd/conf/httpd.conf
10685 ? S 0:00 /usr/sbin/httpd -f /etc/httpd/conf/httpd.conf
10686 ? S 0:00 /usr/sbin/httpd -f /etc/httpd/conf/httpd.conf
10693 ? S 0:00 /usr/sbin/httpd -f /etc/httpd/conf/httpd.conf
10721 ? S 0:00 /usr/sbin/httpd -f /etc/httpd/conf/httpd.conf
10745 ? S 0:00 /usr/sbin/httpd -f /etc/httpd/conf/httpd.conf
10770 ? S 0:00 /usr/sbin/httpd -f /etc/httpd/conf/httpd.conf
10823 ? S 0:00 /usr/sbin/httpd -f /etc/httpd/conf/httpd.conf
10835 ? S 0:00 /usr/sbin/httpd -f /etc/httpd/conf/httpd.conf
10836 ? S 0:00 /usr/sbin/httpd -f /etc/httpd/conf/httpd.conf
10842 ? S 0:00 /usr/sbin/httpd -f /etc/httpd/conf/httpd.conf
10844 ? S 0:00 /usr/sbin/httpd -f /etc/httpd/conf/httpd.conf
10846 ? S 0:00 /usr/sbin/httpd -f /etc/httpd/conf/httpd.conf
10884 ? S 0:00 /usr/sbin/httpd -f /etc/httpd/conf/httpd.conf
10948 ? S 0:00 /usr/sbin/httpd -f /etc/httpd/conf/httpd.conf
10969 ? S 0:00 /usr/sbin/httpd -f /etc/httpd/conf/httpd.conf
10972 ? S 0:00 /usr/sbin/httpd -f /etc/httpd/conf/httpd.conf
10977 ? S 0:00 /usr/sbin/httpd -f /etc/httpd/conf/httpd.conf
10978 ? S 0:00 /usr/sbin/httpd -f /etc/httpd/conf/httpd.conf
10981 ? S 0:00 /usr/sbin/httpd -f /etc/httpd/conf/httpd.conf
10983 ? S 0:00 /usr/sbin/httpd -f /etc/httpd/conf/httpd.conf
11133 ? S 0:00 /usr/sbin/httpd -f /etc/httpd/conf/httpd.conf
11143 ? S 0:00 /usr/sbin/httpd -f /etc/httpd/conf/httpd.conf
11150 ? S 0:00 /usr/sbin/httpd -f /etc/httpd/conf/httpd.conf
11155 ? S 0:00 /usr/sbin/httpd -f /etc/httpd/conf/httpd.conf
11231 ? S 0:00 /usr/sbin/httpd -f /etc/httpd/conf/httpd.conf
11273 ? S 0:00 /usr/sbin/httpd -f /etc/httpd/conf/httpd.conf
11279 ? S 0:00 /usr/sbin/httpd -f /etc/httpd/conf/httpd.conf
11282 ? S 0:00 /usr/sbin/httpd -f /etc/httpd/conf/httpd.conf
11285 ? S 0:00 /usr/sbin/httpd -f /etc/httpd/conf/httpd.conf
11286 ? S 0:00 /usr/sbin/httpd -f /etc/httpd/conf/httpd.conf
11287 ? S 0:00 /usr/sbin/httpd -f /etc/httpd/conf/httpd.conf
11288 ? S 0:00 /usr/sbin/httpd -f /etc/httpd/conf/httpd.conf
11289 ? S 0:00 /usr/sbin/httpd -f /etc/httpd/conf/httpd.conf
11290 ? S 0:00 /usr/sbin/httpd -f /etc/httpd/conf/httpd.conf
11326 ? S 0:00 /usr/sbin/httpd -f /etc/httpd/conf/httpd.conf
11327 ? S 0:00 /usr/sbin/httpd -f /etc/httpd/conf/httpd.conf
11329 ? S 0:00 /usr/sbin/httpd -f /etc/httpd/conf/httpd.conf
11331 ? S 0:00 /usr/sbin/httpd -f /etc/httpd/conf/httpd.conf
11332 ? S 0:00 /usr/sbin/httpd -f /etc/httpd/conf/httpd.conf
11337 ? S 0:00 /usr/sbin/httpd -f /etc/httpd/conf/httpd.conf
11338 ? S 0:00 /usr/sbin/httpd -f /etc/httpd/conf/httpd.conf
11342 ? S 0:00 /usr/sbin/httpd -f /etc/httpd/conf/httpd.conf
11369 ? S 0:00 /usr/sbin/httpd -f /etc/httpd/conf/httpd.conf
11416 ? S 0:00 /usr/sbin/httpd -f /etc/httpd/conf/httpd.conf
11417 ? S 0:00 /usr/sbin/httpd -f /etc/httpd/conf/httpd.conf
11418 ? S 0:00 /usr/sbin/httpd -f /etc/httpd/conf/httpd.conf
11419 ? S 0:00 /usr/sbin/httpd -f /etc/httpd/conf/httpd.conf
11509 ? S 0:00 /usr/sbin/httpd -f /etc/admserv/conf/httpd.conf
11513 pts/1 R 0:00 ps -ax
(only part of the list)
is the report above normal or is it normal for httpd.conf to process so many times?
and when i run "top" httpd uses around 10% of memory :
11917 root 17 0 1132 1132 880 R 0 4.0 0.2 0:05 top
10667 httpd 11 0 6732 6732 5472 S 0 0.3 1.3 0:00 httpd
10721 httpd 1 0 6372 6372 5332 S 0 0.1 1.2 0:00 httpd
12070 httpd 1 0 6492 6492 5912 S 0 0.1 1.2 0:00 httpd
12111 httpd 1 0 6036 6036 5796 S 0 0.1 1.1 0:00 httpd
12114 httpd 1 0 6052 6052 5808 S 0 0.1 1.1 0:00 httpd
1 root 0 0 476 476 404 S 0 0.0 0.0 0:03 init
2 root 0 0 0 0 0 SW 0 0.0 0.0 0:00 kflushd
3 root 0 0 0 0 0 SW 0 0.0 0.0 0:00 kupdate
4 root 0 0 0 0 0 SW 0 0.0 0.0 0:00 kpiod
5 root 0 0 0 0 0 SW 0 0.0 0.0 0:00 kswapd
6 root -20 -20 0 0 0 SW< 0 0.0 0.0 0:00 mdrecoveryd
91 root 0 0 524 524 432 S 0 0.0 0.1 0:00 syslogd
100 root 0 0 784 784 388 S 0 0.0 0.1 0:00 klogd
717 root 0 0 540 540 456 S 0 0.0 0.1 0:00 crond
729 root 0 0 484 484 412 S 0 0.0 0.0 0:00 inetd
758 root 0 0 1652 1652 908 S 0 0.0 0.3 0:00 named
is it normal?
anything i can do to reduce the httpd load/processing?
thanks a lot in advance for helping :)
dutchie 10-22-2001, 10:01 AM Hi Noti,
It might help people if they knew how much memory is in your raq.
A increase from 36% to 96% seems less shocking to me if you've got 32Mb instead of 512.
ops sorry miss out the info ... i am using raq3 server with 512ram ... any thought what might have happened?
i installed logcheck a while ago and just got the report ... it has lots of :
Oct 22 10:45:01 www proftpd[14299]: www.mydomain.com (localhost[127.0.0.1]) - FTP session opened.
Oct 22 10:45:01 www proftpd[14299]: www.mydomain.com (localhost[127.0.0.1]) - no such user 'anonymous'
Oct 22 10:45:01 www proftpd[14299]: www.mydomain.com (localhost[127.0.0.1]) - no such user 'anonymous'
Oct 22 10:45:01 www proftpd[14299]: www.mydomain.com (localhost[127.0.0.1]) - FTP session closed.
Oct 22 10:45:06 www telnetd[14337]: ttloop: read: Broken pipe
Oct 22 11:00:00 www proftpd[15971]: www.mydomain.com (localhost[127.0.0.1]) - FTP session opened.
Oct 22 11:00:00 www proftpd[15971]: www.mydomain.com (localhost[127.0.0.1]) - no such user 'anonymous'
Oct 22 11:00:00 www proftpd[15971]: www.mydomain.com (localhost[127.0.0.1]) - no such user 'anonymous'
Oct 22 11:00:00 www proftpd[15971]: www.mydomain.com (localhost[127.0.0.1]) - FTP session closed.
Oct 22 11:00:04 www telnetd[16006]: ttloop: read: Broken pipe
i didn't log-in to the ftp but only telnet, but i got this messages since 6:00 until 11:00 (end of report) for every 15 minute ... is someone trying to hack into my server? what should i do? how can i prevent it?
jahsh 10-26-2001, 02:58 PM try running 'top' to see what is choking the most resources
cyansmoker 10-26-2001, 03:02 PM Originally posted by noti
i didn't log-in to the ftp but only telnet, but i got this messages since 6:00 until 11:00 (end of report) for every 15 minute ... is someone trying to hack into my server? what should i do? how can i prevent it?
Woops...looks like someone is trying to use your server as a repository for warez or whatnot. Well, this is not necessarily true but it happens quite often.
Regarding how many httpd processes were forked, yes it may be absolutely normal, depending on how you configured this in httpd.conf. Check the content of the file, but with 512MB memory I wouldn't worry too much even if there was like 50 httpds...beware of their time to live, though, 'cause if they were to leak memory that would be bad.
Chicken 10-26-2001, 04:26 PM Originally posted by noti
i installed logcheck a while ago and just got the report ... it has lots of :
Oct 22 10:45:01 www proftpd[14299]: www.mydomain.com (localhost[127.0.0.1]) - FTP session opened.
Oct 22 10:45:01 www proftpd[14299]: www.mydomain.com (localhost[127.0.0.1]) - no such user 'anonymous'
Oct 22 10:45:01 www proftpd[14299]: www.mydomain.com (localhost[127.0.0.1]) - no such user 'anonymous'
Oct 22 10:45:01 www proftpd[14299]: www.mydomain.com (localhost[127.0.0.1]) - FTP session closed.
Oct 22 10:45:06 www telnetd[14337]: ttloop: read: Broken pipe
Oct 22 11:00:00 www proftpd[15971]: www.mydomain.com (localhost[127.0.0.1]) - FTP session opened.
Oct 22 11:00:00 www proftpd[15971]: www.mydomain.com (localhost[127.0.0.1]) - no such user 'anonymous'
Oct 22 11:00:00 www proftpd[15971]: www.mydomain.com (localhost[127.0.0.1]) - no such user 'anonymous'
Oct 22 11:00:00 www proftpd[15971]: www.mydomain.com (localhost[127.0.0.1]) - FTP session closed.
Oct 22 11:00:04 www telnetd[16006]: ttloop: read: Broken pipe
i didn't log-in to the ftp but only telnet, but i got this messages since 6:00 until 11:00 (end of report) for every 15 minute ... is someone trying to hack into my server? what should i do? how can i prevent it?
No one is hacking in. This is the server monitoring itself (checking to see if this or that is up and running) every 15 minutes. note that the hostname/IP that these entries are from:
localhost[127.0.0.1]
thanks a lot for all the replies and help :)
well, after reading Chicken's post i am finally relieved now ... thank a lot ... :D
Anatole 10-27-2001, 01:41 PM Here are some simple step, according to my little experience during last 4 years:
- install mod_throttle you will immediately see who is eating your bandwidth, in a case you will find someone who really loads your server: return money to him and say good-bye. This person needs a dedicated server, but instead tries to save a lot by putting your machine under heavy load.
- limit number of simultanious sql connections. 8 - 16 connections is a reasonable number for a virtual host, if someone wants more - a dedicated server is waiting for him
- use following parameters in apache config for each host: RLimitCPU 8 12
RLimitMem 20000000
these nubers are unadequote in case you give frontpage ext to your users.
these will provide a maximum of 12 seconds for serving requests and not more than 20Megs of RAM for cgi.
- install mod_gzip - you will save a lot of bandwidth and decrease number of httpd requests for text documents.
cyansmoker 10-27-2001, 05:05 PM Originally posted by Anatole
- install mod_throttle you will immediately see who is eating your bandwidth,
mot_throttle seems to be a wonderful tool. However, have you yourself, or do you know anyone who installed mod_throttle with an EAPI version of Apache?
hi and thanks "Anatole"
your suggestions are really very helpful, could u also tell us whether it is a good idea to always implement these scripts before putting the servers live, and could you also suggest other ways to secure the server in diff ways.
thanks
hostchamp 08-13-2002, 02:27 PM Hi,
I have found difficulty in getting mod_throttle to run, i have RH7.2 and latest apache version. If there an rpm for mod_gzip or can i simply copy the module (.so file precompiled)?
Does mod_gzip has any downside?
Do i need to configure mod_gzip for ead virtual domain or do i need to add something to each virtual host block? i already have lots of domains hosted and would prefer some global parameter to achieve this.
Also has anyone successfully used mod_watch to count bandwidth used by each virtual domain? i did check out the website but if anyone can provide basic steps on how to setup the directives in the httpd.conf.
Thanks,
viv
Rochen 08-18-2002, 05:25 PM Originally posted by hostchamp
Does mod_gzip has any downside?
It can cause an increase on the server load and it also fills up the /tmp rather fast with .wrk files, so that will no doubt need to be auto cleaned every 2, 5, 10 minutes etc.
|