MRTG: Switch Ports [Archive] - Web Hosting Talk

lars

09-27-2001, 02:53 PM

I've written some scripts to trim MRTG log files into new log files for 95th percentil reporting. (Use these at your own risk....these will require tweaking they are not perfect. I wrote these 2 years ago, so please back off with the comments on how my style sucks, I was a confused individual at the time)

First, the trim script MRTGtrim2.pl

#!/usr/local/bin/perl

# Program: MRTGtrim2

# Author: Lars Patterson

# Date: 09/27/1999

## Description: MRTGtrim takes standard MRTG format log files and converts them to logfiles

## to be used with a 95th percentile billing system. MRTG logfiles only store

## about 600 lines of 5 minute samples covering a 50 hour time span. To correctly bill the customer

## for the 95th percentile of usage a month, we must have a months worth of 5 minute samples. To

## address this, MRTGtrim recreates logfiles after MRTG is executed via cron, capturing the forth

## line of the MRTG generated logfile and appending it to a new logfile compatible with

## my 95th percentile billing system.

#

## Usage: MRTGtrim -i <mrtg logfile> -o <new logfile>

$numargs = @ARGV;

if ( $numargs == 4 && @ARGV[0] =~ /\-i/ && @ARGV[2] =~ /\-o/ ) {

$infile = @ARGV[1];

$outfile = @ARGV[3];

if ( -r $infile && -w $outfile ) {

mrtgtrim($infile,$outfile);

} else {

printerror("ERROR: The mrtg logfile is not readable or does not exist OR the new logfile is not writeable or does not exist");

exit;

}

} else {

printerror();

exit;

}

sub mrtgtrim {

my $infile = shift(@_);

my $outfile = shift(@_);

my $linecounter=0;

open(INFILE, $infile);

open(OUTFILE, $outfile);

while(<INFILE>) {

$linecounter++;

next unless ( $linecounter == 5 );

@line_entry = split(/\s+/);

$intimestamp = shift(@line_entry);

$avgin = shift(@line_entry);

$avgout = shift(@line_entry);

}

while(<OUTFILE>) {

@line_entry = split(/\s+/);

my $outtimestamp = shift(@line_entry);

if ( $outtimestamp == $intimestamp ) {

printerror("ERROR: Duplicate entry in output logfile");

die "Duplicate Timestamps\n";

}

}

close(OUTFILE);

open(OUTFILE, ">> $outfile");

print OUTFILE "$intimestamp $avgin $avgout\n";

close(INFILE);

close(OUTFILE);

}

sub printerror {

print "Usage: MRTGtrim -i <mrtg logfile> -o <new logfile>\n";

my $numargs = @_;

if ( $numargs > 0 ) {

my $errormsg = shift(@_);

print "$errormsg\n";

}

}

Put something like this in your crontab:

/home/noc/run/MRTGtrim2.pl -i /home/noc/htdocs/admin/MRTG/router.7.log -o /home/noc/log/router.7.log

This will generate over time logs to be used with the script I wrote below to calculate 95th percentile usage.

And here is the script to calculate 95th percentil usage for a one month period 95th.pl:

Again, please use at your own risk as I am not a perl expert. Use with the above log file router.7.log.

#!/usr/local/bin/perl

# Program: 95th

# Author: Lars Patterson

# Date: 09/27/1999

# description: computes the 95th percentile of usage for a 30 day billing cycle

# usage: 95th.pl -f <logfile> -d <ending date>

# ending date format is YYYYMMDD

use Time::Local;

sub printerror {

print "Usage: 95th.pl -f <logfile> -d <ending date>\n";

my $numargs = @_;

if ( $numargs > 0 ) {

my $errormsg = shift(@_);

print "$errormsg\n";

}

}

sub syntax {

print "DEBUG #2\n";

$numargs = @_;

if ( $numargs == 4 && @_[0] =~ /\-f/ && @_[2] =~ /\-d/ ) {

$infile = @_[1];

$datestruct{'stop'}->{'local'} = @_[3];

} else {

printerror("ERROR: incorrect number of args, or incorrect syntax");

exit;

}

unless ( -r $infile ) {

printerror("ERROR: cannot read $infile");

exit;

}

($sec, $min, $hour, $mday, $mon, $year, $wday, $yday, $isdst) = localtime(time);

$yearstring = $year + 1900;

$monstring = $mon + 1;

$mdaystring = $mday;

if ( $monstring < 10 ) {

$monstring = 0 . $monstring;

}

if ( $mdaystring < 10 ) {

$mdaystring = 0 . $mdaystring;

}

$datestruct{'today'}->{'local'} = $yearstring . $monstring . $mdaystring;

if ( $datestruct{'stop'}->{'local'} > $datestruct{'today'}->{'local'} ) {

printerror("ERROR: $datestruct{'stop'}->{'local'} is in the future");

exit

} elsif ( $datestruct{'stop'}->{'local'} <= 0 ) {

printerror("ERROR: $datestruct{'stop'}->{'local'} is the epoch, or before the epoch");

exit;

}

}

sub billingstart {

# finds the start date of the billing cycle (1 month prior to the entered date)

@datearray = split(//, $datestruct{'stop'}->{'local'});

$yearstring = @datearray[0] . @datearray[1] . @datearray[2] . @datearray[3];

$monstring = @datearray[4] . @datearray[5];

$daystring = @datearray[6] . $datearray[7];

# translate YYYYMMDD into Unix time

$yearstring = $yearstring - 1900;

# convert month into Unix time format

$monstring = $monstring - 1;

# subtract one month, and if beginning of year, subtract one year

if ( $monstring == 0 ) {

$monstring = 11;

$yearstring = $yearstring - 1;

} else {

$monstring = $monstring - 1;

}

# convert to Unix time

$datestruct{'start'}->{'unix'} = timelocal(0,0,0,$daystring,$monstring,$yearstring);

print "year = $yearstring\tmonth = $monstring\tday = $daystring\n";

print "Billing start date is $datestruct{'start'}->{'unix'}\n";

}

sub billingend {

# finds the end date of the billing cycle (unix time of the stop date)

@datearray = split(//, $datestruct{'stop'}->{'local'});

$yearstring = @datearray[0] . @datearray[1] . @datearray[2] . @datearray[3];

$monstring = @datearray[4] . @datearray[5];

$daystring = @datearray[6] . $datearray[7];

# translate YYYYMMDD into Unix time format

$yearstring = $yearstring - 1900;

# convert month into Unix time format

$monstring = $monstring - 1;

# convert to Unix time

$datestruct{'stop'}->{'unix'} = timelocal(0,0,0,$daystring,$monstring,$yearstring);

print "year = $yearstring\tmonth = $monstring\tday = $daystring\n";

print "Billing end date is $datestruct{'stop'}->{'unix'}\n";

}

sub loadlog {

# opens the logfile and reads between $billstart and $billend time stamps

@avgin_array = ();

@avgout_array = ();

$numpolls = 0;

open(LOGFILE, $infile);

while(<LOGFILE>) {

@line_entry = split(/\s+/); # split the line

$linestamp = @line_entry[0]; # get the timestamp for the line

next if ( $linestamp < $datestruct{'start'}->{'unix'} ); # remove dates before billing period

next if ( $linestamp > $datestruct{'stop'}->{'unix'} ); # remove dates after billing period

# Relevant lines to examine

$numpolls++;

$avgin = @line_entry[1] * 8;

$avgout = @line_entry[2] * 8;

push(@avgin_array,$avgin);

push(@avgout_array,$avgout);

#print;

}

#print "Total number of polls: $numpolls\n";

close(LOGFILE);

}

sub bynumber {

$a <=> $b;

}

sub main {

$counter = 0;

$ninetyfifth = $numpolls - int( .95 * $numpolls );

# sort both the arrays from smallest to largest

@sorted_avgin = sort bynumber @avgin_array;

@sorted_avgout = sort bynumber @avgout_array;

while ( $counter < $ninetyfifth ) {

$tempin = pop(@sorted_avgin);

$tempout = pop(@sorted_avgout);

$counter++;

print "IN = $tempin\tOUT = $tempout\n";

}

$in = pop(@sorted_avgin);

$out = pop(@sorted_avgout);

print "Number of polls: $numpolls\n";

print "95th percent values are: IN = $in bps\tOUT = $out bps\n";

}

sub input {

syntax(@ARGV);

&billingstart();

&billingend();

&loadlog();

&main();

}

&input();

Hope this helps:cool: