Could processing server logs keeps crashing a server? I have a dual 800Mhz processor and 512MB of ram. Should this not be powerful enough to process my logs? The server uses CPanel which has Webalizer and Analog. 2 of my larger sites have logs of about 1.7 gigs and 2 gigs.

no wonder i can't access your site :(

Yup this is really stressing me out. :( Any suggestions?

is there really a need for log files to be that big
dont you thinkits time you delete them

If your log files are that big, you really need to rotate them and process them daily. Not only is that a waste of space, but its a large hamper on resources when you go to process that large of logs. I know webalizer has a "Incremental" processing to do logs, but I am not sure how analog works.

Hi Node,

Doesn't the logs delete itself after awhile? Sorry I'm quite the amature when it comes to managing my own server.

Hi Dilhole,

How do I specify the rotate times for the logs on a RedHat linux machine? Sorry I'm a sad newbie.

I don't know who your're hosting with, but they should be able to help you sort this kind of thing out.

If not, maybe you should consider hosting w/ someone who can/does/will.

I will give them a bit more time to investigate. I hope they can find out because moving is a real pain especially with all the scripts on the sites. Due to the recent incidents, I'm watching my server very closely from now on.

<<Please email directly, or post information on the forum>>

George,

Well Burst has solved the HD space problem by adding another hard drive to /usr/ so I don't need to do that workaround. The problem right now is the fact that the logs are too big and I need to set them to rotate daily instead of weekly.

I edited the /etc/logrotate.conf to reflect this but I am not sure if this will do the trick because of CPanel.

ah so more $$$ added to your costs :(

depends of log size, daily time of stats generating, OS and stats program

petr, it's on a Burst.net Cpanel/WHT setup. Log size, up to 2 gigs. Time generated: unknown. OS:Linux Redhat. Stats prog: Webalizer & Analog.

Meanwhile I have deleted /usr/local/apache/logs/error_log because it was close to 800MB. I have also deleted all the big vhost logs. I really don't know what else I could do with my limited linux knowledge. I tried editing the /etc/logrotate.conf to rotate the logs on a daily basis but that doesn't seem to work. I think it's because of the way CPanel is setup.

Also deleted all the big logs for each vhost.

My #@*&%!@ server crashed again. This is !@$%@#! up! I've written another ticket to Burst.net. ARRRRRRRGHHHHHHH!!!!!!! I'm not having a #@%#! good day!

I had total of 2.5GB worth of log files under /usr/local/apache/domlogs. There is one single file of 1.7GB and another single file of 700MB size.... BTW, these logs files are rortated monthly ... So this 2.5GB is just for the 18 days of November. The server never crashes ... In fact, I did a test quite a while ago ... restarting my log files from zero and see how the servre load goes ... To my best recollection, it was not much different ... I think Linux is very capable of handling large files ...

Out of curiosity, can you please do this (using root, but be very careful):

type: '/sbin/hdparm /dev/hda', and give me the output

type: '/sbin/dparm -tT /dev/hda', and give me the output. This will benchmark your HDs and hence the result vary greatly on how busy (read: server load) your server is ...

You may replace /dev/hda to whatever your drive(s) is mounted on. Be careful with hdparm, don't try any other command unless you know what you are doing ... The above should be OK ...

cheers,
:beer:

Simply setup a cron to delete the logs once per week at least. Another thing is that something could be corrupting your logs. If logs are corrupted, apache wont start..and it wont tell you the reason either. Also do a du and make sure no partitions are full. The tmp directory can get loaded up, fill the partition, and raise all sorts of havoc..

Dektong, here's what I got:

root@ns1 [/usr/local/apache/domlogs]# /sbin/hdparm /dev/hda

/dev/hda:
multcount = 16 (on)
I/O support = 1 (32-bit)
unmaskirq = 1 (on)
using_dma = 1 (on)
keepsettings = 0 (off)
nowerr = 0 (off)
readonly = 0 (off)
readahead = 8 (on)
geometry = 7297/255/63, sectors = 117231408, start = 0

dparm gave me this:

root@ns1 [/usr/local/apache/domlogs]# /sbin/dparm -tT /dev/hda
bash: /sbin/dparm: No such file or directory

I guess I don't have that command installed.

Dektong, your server seems to have the same amount of traffic as mine. Could you give me your MaxClients, MaxRequestsPerChild, MinSpareServers, MaxSpareServers, StartServers and KeepAlive settings for your httpd.conf? I have a dual pentium 800 with 512MB of ram. I think this might help with my problems.

Webdude, I might have to resort to doing a cron that will delete my logs daily if I can't get logrotate on CPanel/WHM to do it. I'm pretty sure that if I do it this way, my logs on webalizer will be inconsistent though. There is enough HD space for the logs. I think the problem is that when it tries to process the logs, it crashes.

check you post over at cpanel.net's forum regarding this :)

i replied there :D

Originally posted by a1022
[B]dparm gave me this:


It should be /sbin/hdparm , sorry for the typo :D

cheers,
:beer:

Originally posted by eva2000
<<Please email directly, or post information on the forum>> doh didn't know that wasn't acceptable? i did the same for raq4 php/mysql and mod_gzip install on this forum before with no probs?

sorry :(

<<MOD NOTE: I think it was something like, contact me, or I don't know...>>