Results 1 to 6 of 6
Thread: Reducing error log size
-
09-08-2005, 02:55 PM #1Junior Guru Wannabe
- Join Date
- Oct 2002
- Posts
- 36
Reducing error log size
I'd like to reduce the amount of information that is logged in the error_log and access_log in /etc/local/apache/logs
Currently it records missing 404's, missing robots, missing favicon's, jpg, etc, etc.
This all adds up to wasted resources, huge log files, extra reboots, hard to read logs, and sexually transmitted diseases.
I've found the following info very helpful (ie this is how you do it), however I haven't had success so far.
<VirtualHost *:80>
...
# Logging --
# Filter out everything of no importance to reduce the logging load
# Check the logs periodically to make sure this filter is up-to-date **
<IfModule mod_setenvif.c>
# Filter out what we don't want to log
SetEnvIf Request_URI "\.gif$" no_log
SetEnvIf Request_URI "\.jpg$" no_log
SetEnvIf Request_URI "\.png$" no_log
SetEnvIf Request_URI "\.css$" no_log
SetEnvIf Request_URI "\.js$" no_log
SetEnvIf Request_URI "\.ico$" no_log
SetEnvIf Request_URI "css.css" no_log
SetEnvIf Request_URI "robots.txt" no_log
</IfModule>
# Log what we need for awstats
LogFormat "%h %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" mylog
CustomLog /var/log/httpd/mysite-access_log mylog env=!no_log
ErrorLog /var/log/httpd/mysite-error_log
...
</VirtualHost>
Here are my particulars:
OS: 2.4.20-021stab028.17.777-enterprise
Apache: 1.3.33
Cpanel 10.2.0-RELEASE 82
So I'd like to ensure the logging is fixed for each domain I create.. is it possible? Or do I need to tweak each entry in the httpd.conf?
Does ALL of the above need to be in each virtual container? Or one statement in the main part of the conf, and then just the CustomLog line per container?
Thanks for any help!
-
09-08-2005, 03:18 PM #2Web Hosting Master
- Join Date
- Dec 2002
- Location
- The Shadows
- Posts
- 2,925
The above only works for the access log. If you want to filter the error log, you will need to pipe it to a program and have that do the logging. Otherwise, you are SOL.
Dan Sheppard ~ Freelance whatever
-
09-08-2005, 03:26 PM #3Junior Guru Wannabe
- Join Date
- Oct 2002
- Posts
- 36
I'll take your word for it, but now I'm confused...
This is typical of my access_log:
"127.0.0.1 - - [08/Sep/2005:12:08:51 -0700] "GET / HTTP/1.0" 200 2973
127.0.0.1 - - [08/Sep/2005:12:10:09 -0700] "GET /whm-server-status HTTP/1.0" 200 15594
127.0.0.1 - - [08/Sep/2005:12:15:05 -0700] "GET /whm-server-status HTTP/1.0" 200 15241
127.0.0.1 - - [08/Sep/2005:12:17:13 -0700] "GET / HTTP/1.0" 200 2973
127.0.0.1 - - [08/Sep/2005:12:20:06 -0700] "GET /whm-server-status HTTP/1.0" 200 15442"
There is no mention of any type of file.. that only happens in the error_log. So why would you filter out certain file types if they are not even mentioned in the access_log?
-
09-08-2005, 03:29 PM #4Web Hosting Master
- Join Date
- Dec 2002
- Location
- The Shadows
- Posts
- 2,925
/whm-server-status is a "file", sort of. It is a alias to a internal module. Since this is a cPanel box, your actual domain access logs are here:
/usr/local/apache/domlogs/
You will most likely find, in the domain you filtered out images, that if someone requests a image, it doesn't appear in the access log.
The error_log shows errors about files that don't exist.Dan Sheppard ~ Freelance whatever
-
09-08-2005, 04:05 PM #5Junior Guru Wannabe
- Join Date
- Oct 2002
- Posts
- 36
Thanks, that explains a lot!
Now that I know where to look and what to look for I should be able to get the .conf setup right. I didn't think it was working before (no matter what I did) because i couldn't see any positive results. ie i was looking at the wrong file.
A futher question (to solve the original question). Is there a common method/program that people use to pipe the error_log to that filters out useless data, etc? And would piping this file effect the hosted domain error logs, or are theirs separate?
-
09-08-2005, 04:13 PM #6Web Hosting Master
- Join Date
- Dec 2002
- Location
- The Shadows
- Posts
- 2,925
I don't know of anything, but then again, I am a custom software kinda guy
As far as affecting hosted domains, yes it would.Dan Sheppard ~ Freelance whatever