Is anyone using a cron job to copy log files from the folder they get rotated out of to one inside their own account?

I'd like to be able to set up a daily cron job so that I don't miss much data when the rotation is done at the end of the month.

I know that the logs are stored in /usr/local/apache/domlogs.

If someone is copying log files to another directory using cron I would appreciate knowing how it is done.

Regards/

Here are a few quick bash files I just hacked that can manually "rotate" a users raw access logs. I should emphasize hacked as they aren't pretty but get the job done. Note that if you implement this your control panel stats may not report correctly.

The account-username variable should be changed to the system username you want the logs stored under and change account-domain to the domain name you want to "rotate" the logs on.

Keep in mind this hasn't been tested and you should verify that this will function on your own system. Use at your own risk, there is no warranty or guarantee that this will work.


===================================================
Directory Structure - Run at midnight ET every day
===================================================

This script creates the directory structure within your account.

#!/bin/sh

user=`account-username`

year=`/bin/date +%Y`
month=`/bin/date +%m`
week=`/bin/date +%m`
day=`/bin/date +%d`

/bin/mkdir -p /home/$user/rawlogs/$year
/bin/mkdir -p /home/$user/rawlogs/$year/$month
/bin/mkdir -p /home/$user/rawlogs/$year/$month/$week
/bin/mkdir -p /home/$user/rawlogs/$year/$month/$week/$day

==================================================
Log Copying - Run every 30 minutes
==================================================

This script copies the domain access log file from there original locations into the structure created above.
Once the copy is done the script will then zero the log file.

#!/bin/sh

user=`account-username`
domain=`account-domain`

year=`/bin/date +%Y`
month=`/bin/date +%m`
week=`/bin/date +%m`
day=`/bin/date +%d`

logtimestamp=`/bin/date +%T`

/bin/cp -ap /etc/httpd/domlogs/$domain /home/$user/rawlogs/$year/$month/$week/$day/$logtimestamp
/bin/echo -n > /etc/httpd/domlogs/$domain

===================================================
Consolidation script - Run Daily before midnight ET
===================================================

This script consolidates the new log files created every 30 minutes.
The log files created every 30 minutes are copied into a larger file labeled by the week.
This also creates another larger log file for the month as well.

#!/bin/sh

user=`account-username`

year=`/bin/date +%Y`
month=`/bin/date +%m`
week=`/bin/date +%m`
day=`/bin/date +%d`

/bin/touch /home/$user/rawlogs/$year/$month/$week/$week
/bin/cat /home/$user/rawlogs/$year/$month/$week/$day/* >> /home/$user/rawlogs/$year/$month/$week/$week

/bin/touch /home/$user/rawlogs/$year/$month/$month
/bin/cat /home/$user/rawlogs/$year/$month/$week/* >> /bin/touch /home/$user/rawlogs/$year/$month/$month

=====================================================
Cleanup script - Run weekly before midnight Sunday ET
=====================================================

This script removes the day directories from the past week.
This will help reduce the size of your log file archive.

#!/bin/sh

user=`account-username`

year=`/bin/date +%Y`
month=`/bin/date +%m`
week=`/bin/date +%m`
day=`/bin/date +%d`


/usr/bin/find /home/$user/rawlogs/$year/$month/$week/ -type d -exec rm -rf {} \;

Hi Serial.

I appreciate the hacks mentioned above, now i have my self confused. One of my customer has many a domains, whose logs he needs to be mailed to him daily through CRON job,

So could you tell me how do i do it for all his domain..which of the above hacks should i proceed with..so that only his account is tampered with ..leaving the rest untouched ?

I hope i was clear in my query...appreciate your help

Thanks

You will still need to cron all of the bash files above for each domain name you want to manually rotate log files for. The bash script below will mail the accumulated stats each day for the given week.

Be sure to change a few of the variables below (account-username, email address and any paths which may be incorrect to the binaries listed at the top.)


#!/bin/sh

tar=`/bin/tar`
uuencode=`/usr/bin/uuencode`
mail=`/bin/mail`
user=`account-username`

backupdir=`/home/$user/rawlogs/`

year=`/bin/date +%Y`
month=`/bin/date +%m`
week=`/bin/date +%m`
day=`/bin/date +%d`

subject="Stats Backup"
mailto="user@domain.com"

$tar zcvf ${backupdir}/stats-email.tar.gz /home/$user/rawlogs/$year/$month/$week/$week
$uuencode ${backupdir}/stats-email.tar.gz stats-email.tar.gz > ${backupdir}/stats-email.tar.gz.uu
$mail -s "$subject" $mailto < ${backupdir}/stats-email.tar.gz.uu

rm -f ${backupdir}/stats-email.tar.*

exit

A very nice tool to mail files as a mime encoded attachment from within a shell is mpack (http://ftp.andrew.cmu.edu/pub/mpack/)!

just to be mentioned ... ;)


Michael