Web Hosting Talk







View Full Version : Custom Apache logs (specifically formmail logging)


2Grumpy
01-19-2004, 02:37 PM
I dunno about you other guys, but formmail abuse is our #1 cause of spam complaints. So today I finally did a little research and came up with this little jewel for logging ANY and ALL calls to "formmail" (case irrelevant) to a formmail.log in apache (/var/log/httpd/formmail.log on most systems):

LogFormat "%h %l %u %t \"%v %r\" %>s %b" formmaillog
SetEnvIf Request_URI "[F,f][O,o][R,r][M,m][M,m][A,a][I,i][L,l]" formmail
CustomLog logs/formmail.log formmaillog env=formmail

Here's an output from the formmail.log:

24.214.X.X - - [19/Jan/2004:12:24:56 -0600] "dixiesys.com GET /ForMMail.pl HTTP/1.1" 404 286

The most helpful thing is the %v in the LogFormat because this logs _virtual host_ information too so you can quickly see what vhost is getting the request.

Now all I need is a script to monitor formmail.log and alert me on a certain condition, maybe a 5 minute check and alert if more than say, 10 new lines are in the log (obviously this would need tweaking in the real world probably) so that you can catch the formmail crap before it results in XX thousand emails and several spamcop complaints! Actually I'll probably change the log location to something like /var/www/html/logs/formmail.log and then I can run a central script on a remote server that checks all my servers every few minutes by just loading this url (like www.servername.com/logs/formmail.log) so I only have to run one script rather than 54 scripts once on each server!

This is a first draft of this log rule if anyone improves it reply here and share the improvements :)

I spend way too much time chasing down formmail abuses :(

<edit>signature removed</edit>

Dan L
01-20-2004, 06:18 PM
Originally posted by Dixiesys I spend way too much time chasing down formmail abuses :(

Nah, this is pretty helpful. :)

Akash
01-24-2004, 12:56 AM
i tried this and it doesnt work :( my formmail.log is still empty

I stuck the

LogFormat "%h %l %u %t \"%v %r\" %>s %b" formmaillog
SetEnvIf Request_URI "[F,f][O,o][R,r][M,m][M,m][A,a][I,i][L,l]" formmail
CustomLog logs/formmail.log formmaillog env=formmail
right above the access log section in my httpd.conf.....is that not where it goes?


I am using the /cgi-sys/formmail.cgi and .pl that i got on my cpanel system...

<edit>signature removed</edit>

choon
01-24-2004, 01:16 AM
If you are using Cpanel and from the CustomLog you have specified... which location you are trying to view?
Is it /usr/local/apache/logs/formmail.log?

Akash
01-24-2004, 01:17 AM
Yeah - the log is there - but the log is empty (0 Lines)

<edit>signature removed</edit>

choon
01-24-2004, 01:19 AM
I assume you have tried http://yoursite/formmail.pl ?

Akash
01-24-2004, 01:22 AM
I'm using http://mysite/cgi-sys/formmail.pl (and formmail.cgi) - this is the formmail clone script cpanel comes wiht by default - does it not log the scripts available like this systemwide?

<edit>signature removed</edit>

2Grumpy
01-24-2004, 01:29 AM
Try just accessing www.blah.com/formmail.php (whether it exists or not is irrelevant it'll still log the hit).

I just tried www.something.com/asdf/asdf/asdf/formmail.php and it was logged.

<edit>signature removed</edit>

Akash
01-24-2004, 01:33 AM
still 0 lines.

yes i restarted httpd, and i know its creating the log file, i deleted it, restarted httpd, accessed formmail again, and the log file is created again - but with 0 lines.

<edit>signature removed</edit>

choon
01-24-2004, 01:37 AM
I wonder do you have mod_setenvif ? Is it default to have it?

2Grumpy
01-24-2004, 01:39 AM
Hmm I've tried this on DA and Ensim servers so far and it worked but that's not a very large test bed :)

<edit>signature removed</edit>

Akash
01-24-2004, 01:42 AM
choon - i dont see it in the rpm list in WHM. How would I check to see if i have it? and If i dont have it (probably dont) How do i install it (would make another great tutorial ;))

<edit>signature removed</edit>

choon
01-24-2004, 01:57 AM
I will test it and get back to you guys... it is compiled as default. Use this command to check as it is compiled as DSO:
cat /usr/local/apache/conf/httpd.conf \
| grep mod_setenvif

Dixiesys, have you try with SetEnvIfNoCase instead of SetEnvIf so that your regular expression matching is performed in a case-insensitive manner? I mean instead of using:
SetEnvIf Request_URI "[F,f][O,o][R,r][M,m][M,m][A,a][I,i][L,l]" formmail
you can try this:
SetEnvIfNoCase Request_URI "FoRmMaIl" formmail

Akash
01-24-2004, 02:12 AM
^^ NoCase still doesn't work for me

<edit>signature removed</edit>

choon
01-24-2004, 03:45 AM
Ok, I have it worked and here are my steps:

Step 1: Locate the line in your httpd.conf:
LogFormat "%{User-agent}i" agent
And add the following line after the above line:
LogFormat "%h %l %u %t \"%v %r\" %>s %b" formmaillog

Step 2: Locate the line in httpd.conf:
<IfModule mod_setenvif.c>
And add the following after the above line:
# for formmail
SetEnvIfNoCase Request_URI "formmail" formmail

Step 3: Enable each VirtualHost logging
Add the following line in those VirtualHost you want to enable logging:
CustomLog /usr/local/apache/logs/formmail.log formmaillog env=formmail

My suggestion will be change the logged file to something more useful like using a directory just for logging this formmail:
mkdir /usr/local/apache/logs/formmaillog

Then within each VirtualHost...
CustomLog /usr/local/apache/logs/formmaillog/yourdomain formmaillog env=formmail
Or below (which you can skip Step 1 if you are using this):
CustomLog /usr/local/apache/logs/formmaillog/yourdomain combined env=formmail

However I won't know by adding this within each VirtualHost will affect WHM/Cpanel or not :(

Akash
01-25-2004, 04:08 PM
Yep, it looks like the only way to get it to work on a cpanel box is to follow choons instructions above.

Does anyone know of a way to auto-insert this into each virtualhost entry when cpanel makes a new account?

<edit>signature removed</edit>

Andrew
01-26-2004, 03:32 PM
edit the <virtualhost> settings in /scripts/wwwacct

then chattr +i /scripts/wwwacct

<edit>signature removed</edit>