Web Hosting Talk : Web Hosting Main Forums : Hosting Security and Technology : How is bandwidth measured via apache? Is there any software that can calculate that?

[wheimeng]

As mentioned, I am requested to build a custom system that does not use any control panel but calculates bandwidth used via httpd (apache)

Any idea how is it measured? Is there any good software that does that?

[layer0]

VNStat?

cd /tmp
wget http://humdi.net/vnstat/vnstat-1.4.tar.gz
tar xzf vnstat-1.4.tar.gz
cd vnstat-1.4/
make
make install
vnstat -u -i venet0

If you get an error after that, no worries.

Thanks,

[wheimeng]

This seems to be total usage. I need a bandwidth calculator that works on individual website like cpanel does. Individual bandwidth usage.

[ozarkhost]

Closest thing out there is mod_throttle which seems to be what your looking for.

[EXOWorks]

Yes.. mod_throttle is the best choice. But you can write something on your own, that would run on daily or hourly basis and make use of the access logs to count the bandwidth usage of a particular site.

It would be something like this:
- Read the access_log file of the site.
- Get all access files.
- Make an array and store all the filenames, in the memory. Make sure that you are able to count the instances of each file so you can multiply later on. We could count the size for each instance but this would be waste of server resources.
- Start counting the file sizes.
- Store the total size you got, somewhere.
- Delete the access log file.

[SROHost]

I disagree that mod_throttle is the best choice because it adds more overhead than stat collection alone (unless you plan to run it already of course).

There are more than a few Apache modules which can do the back end of this (mod_accounting, mod_watch and mod_log_sql are a few more), but if your Apache install is already logging for each VirtualHost, adding a separate accounting module might increase the server's work load unneccesarily. If load is potentially an issue, the best solution may be to replace Apache's internal logging with one of the above modules or write a custom script as Mike_R suggests to parse Apache log files and collect stats on a schedule. The advantage to the latter is that you can schedule your stat collection at times when the server is normally under minimal load.

Since most third-party Apache log parsers are designed for full stats and not just bandwidth, we wrote our own. In concept this is fairly simple: open each log file, read the 7th field of each line (assuming CLF that's bytes) and update a database table for each domain. In practice, however you will want to do things to integrate it into your system and keep it efficient. We read the log file names from an account database, save file position pointers and date information for each log (don't have to delete logs every time the script runs, but never count the same line twice), exclude our own IPs from the collected stats and integrate http, ftp and email counters in one script.

If you don't want to do all that, I would try mod_log_sql or mod_accounting.

[The_Overl]

If you do decide to parse log files, don't use the file size, the size of the http responce is already in the log file (usually). For dynamically created content, file size and actual amount transferred wont match anyways, so the file size is irrelevant.