Results 1 to 25 of 32
Thread: how is this possible ...
-
04-27-2004, 12:44 PM #1Junior Guru Wannabe
- Join Date
- Apr 2004
- Location
- Houston, TX
- Posts
- 80
how is this possible ...
I have a T1 running into my small hosting facility, so the total possible bandwidth should be around 487GB / month.
I wrote a script to add up the web traffic based on my Apache web logs. When I add up the size of the files served out by my webserver, from the Apache logs, *several* sites on my webserver are serving out a total of over 400GB / month.
That can't be right.
I've tried narrowing it down to only log entries which resulting in a result code of 200, which according to W3C means the file was successfully transferred, but it still adds up to over 400GB / month for each of several busy sites on the server.
I thought maybe it was my log parsing script not working right, so I saved a log and FTP'd it over to a Windows system, opened it in Excel, sorted it by result code, added up the lines with a result code of 200 ... and still got a total of over 400GB in a single month.
Am I missing something? How can my webserver be serving out over 1200GB per month when my T1 would be completely saturated 24 / 7 at 487GB / month?
Besides ... the line gets busy occsaionally during the day but there's long periods of time with little or no traffic. And most T1's actually can only handle about 350GB per month after errors and retransmits and stuff get factored in.
Someone please point me to a apache logs 101 or something ... I *must* be misunderstanding some very basic concept here ...http://www.dynamis.net
A small domain host focusing on personalized service.
-
04-27-2004, 07:38 PM #2Web Hosting Guru
- Join Date
- Jun 2003
- Posts
- 299
maybe gzip is installed?
Pro-net-hosting.com - www.pro-net-hosting.com
Pro hosting solutions since april 2003.
-
04-27-2004, 10:04 PM #3WHT Addict
- Join Date
- Apr 2004
- Location
- Bulgaria
- Posts
- 159
if I'm not mistaken a T1 line has a capacity of 10mpbs. At full utilization this would add up to over 3000GB per month
Solid-Hosting.net - affordable multiple domain hosting solutions
Reliability and Customer Service are our focus!
-
04-27-2004, 10:08 PM #4Web Hosting Master
- Join Date
- Jul 2003
- Location
- Satyr, Chrisalya, Canada
- Posts
- 1,901
a T1 is 1.5mbps.
--
-
04-27-2004, 10:34 PM #5Junior Guru Wannabe
- Join Date
- Apr 2004
- Location
- Houston, TX
- Posts
- 80
Originally posted by heavypredato
maybe gzip is installed?
AddEncoding x-compress Z
AddEncoding x-gzip gz
The comments for these lines indicate they allow some browser to uncompress information on the fly. However it's unclear whether they mean uncompressing gzip (.gz) documents on the fly or uncompressing compressed html files and such on the fly.
Tomhttp://www.dynamis.net
A small domain host focusing on personalized service.
-
04-28-2004, 09:55 AM #6Junior Guru
- Join Date
- Sep 2003
- Location
- UK
- Posts
- 184
are you sure that you are working the math out correctly? 400GB for one site is pretty high (depening if they host any rich media or not).
400GB would be 419430400 Bytes, are you sure its not 400MB?
-
04-28-2004, 10:33 AM #7Web Hosting Master
- Join Date
- Nov 2002
- Posts
- 737
I would be more concerned it was 400 GigaBits, perhaps you the logs are recording bits and not bytes, and you are forgetting to divide by 8.
Just a thought.LipWeb.Net
"Less Lip More Service"
Providing Quality Hosting at Fair Prices
AIM: LipWebNet MSN: daniel[at]lipweb.net [color=red]
-
04-28-2004, 10:34 AM #8Junior Guru
- Join Date
- Sep 2003
- Location
- UK
- Posts
- 184
Yeah I did think about that, surely apache logs in bytes though? I have not used apache before so have no idea lol.
-
04-28-2004, 01:30 PM #9WHT Addict
- Join Date
- Apr 2004
- Location
- Phoenix, Arizona
- Posts
- 151
Do the logs make the distinction between what was really served and what was checked, but not served because of the cache on the client side?
just a thought.
Danhttp://www.IwantFUI.com
If you could host a new kind of content from your old-fashioned web servers
and make new money from your customers and differentiate your business all at the same time... could you afford not to try? See the new site
-
04-28-2004, 07:28 PM #10Junior Guru Wannabe
- Join Date
- Apr 2004
- Location
- Houston, TX
- Posts
- 80
FOUND THE PROBLEM ... IT WAS ME ...
Delirium, thanks .... I have found the problem and it was me.
The total number of bytes for the 1st 9 days of the month was:
496,060,118
That's 484,433KB
or 473MB
or .46 GB
I was skipping the KB step in my computation.
419,430,400 Bytes is
409,600 KBytes is
400 MBytes is
.39 GBytes
Guess that's what I get for working the problem at 2am ...
Thanks a bunch guys for all the input !!!! WHT rocks!!!!
Tomhttp://www.dynamis.net
A small domain host focusing on personalized service.
-
04-28-2004, 07:32 PM #11Junior Guru Wannabe
- Join Date
- Apr 2004
- Location
- Houston, TX
- Posts
- 80
Originally posted by delirium
are you sure that you are working the math out correctly? 400GB for one site is pretty high (depening if they host any rich media or not).
400GB would be 419430400 Bytes, are you sure its not 400MB?
429,496,729,600 bytes.http://www.dynamis.net
A small domain host focusing on personalized service.
-
04-28-2004, 08:31 PM #12Junior Guru
- Join Date
- Sep 2003
- Location
- UK
- Posts
- 184
Originally posted by dynamis
Well actually it seems that per my new discovery of the true nature of gigabytes , 400GB is actually
429,496,729,600 bytes.
Sorry you are correct with your math above (I needed to multiply by another 1024),
In future you can use google to work it out open google and type
400 gigabytes in bytes
as your query - Google will return 400 gigabytes = 429 496 729 600 bytes
obviously you could run this query the other way round
-
04-28-2004, 08:39 PM #13Junior Guru Wannabe
- Join Date
- Apr 2003
- Posts
- 40
Re: how is this possible ...
Originally posted by dynamis
I have a T1 running into my small hosting facility, so the total possible bandwidth should be around 487GB / month.
I wrote a script to add up the web traffic based on my Apache web logs. When I add up the size of the files served out by my webserver, from the Apache logs, *several* sites on my webserver are serving out a total of over 400GB / month.
That can't be right.
I've tried narrowing it down to only log entries which resulting in a result code of 200, which according to W3C means the file was successfully transferred, but it still adds up to over 400GB / month for each of several busy sites on the server.
I thought maybe it was my log parsing script not working right, so I saved a log and FTP'd it over to a Windows system, opened it in Excel, sorted it by result code, added up the lines with a result code of 200 ... and still got a total of over 400GB in a single month.
Am I missing something? How can my webserver be serving out over 1200GB per month when my T1 would be completely saturated 24 / 7 at 487GB / month?
Besides ... the line gets busy occsaionally during the day but there's long periods of time with little or no traffic. And most T1's actually can only handle about 350GB per month after errors and retransmits and stuff get factored in.
Someone please point me to a apache logs 101 or something ... I *must* be misunderstanding some very basic concept here ...
-
04-28-2004, 10:30 PM #14Junior Guru Wannabe
- Join Date
- Apr 2004
- Location
- Houston, TX
- Posts
- 80
It's $523 per month including local loop.
Tomhttp://www.dynamis.net
A small domain host focusing on personalized service.
-
04-28-2004, 11:02 PM #15Junior Guru Wannabe
- Join Date
- Apr 2003
- Posts
- 40
Originally posted by dynamis
It's $523 per month including local loop.
Tom
-
04-29-2004, 02:08 PM #16
I've found this site to be very helpful when dealing with bandwidth and conversion from bytes to gb, mb to gb or anything of the like. If you have to do this in the future, you might want to head over there, save yourself a LOT of math and legwork
Tom Whiting, WHMCS Guru extraordinaire
Linux problems? WHMCS Problems? Give me a shout
Check out my WHMCS Addons
-
04-29-2004, 02:12 PM #17Junior Guru Wannabe
- Join Date
- Apr 2004
- Location
- Kansas City, MO
- Posts
- 53
That's an extremely handy site.
Take care,
Brad Birmingham
http://www.bluevirtual.com
-
04-29-2004, 02:35 PM #18WHT Addict
- Join Date
- Dec 2002
- Posts
- 100
Originally posted by noimad1
Wow, that is pricey. Do you primarily just do hosting with that? How many servers/clients are you able to run on that?.
-
04-29-2004, 02:47 PM #19
yeah, I meant to say something about that one myself.
I called a local cable (also doing inet and t1) provider recently and the prices they quoted me made what you pay look like nothing. IIRC they wanted around $1.5k / month and that DIDN'T include telco or building charges.
$500 seems to be about the average right now though.Tom Whiting, WHMCS Guru extraordinaire
Linux problems? WHMCS Problems? Give me a shout
Check out my WHMCS Addons
-
04-29-2004, 02:57 PM #20Junior Guru Wannabe
- Join Date
- Apr 2003
- Posts
- 40
Originally posted by wolfstream
yeah, I meant to say something about that one myself.
I called a local cable (also doing inet and t1) provider recently and the prices they quoted me made what you pay look like nothing. IIRC they wanted around $1.5k / month and that DIDN'T include telco or building charges.
$500 seems to be about the average right now though.
It is certainly handy to be able to have complete control of the server/hardware, but is it completly worth it as a hosting company?
Do you have UPS/Generator backup at your facility?
Please don't take this e-mail the wrong way. I am not trying to flame anyone for using their own data connection, but I am just trying to see the advantages. Or see if this is something I might want to look into myself.
Our servers use around 100GB of bandwith each....so I could only get 4 servers max off that T1. But then again, how well does it handle bursting etc. If all 4 servers were running at full capacity, I would have to assume the dataconnection would slow rapidly?
Then, at the same price about $500-600 I could get 4 dedicated servers in a datacenter that has the larger dataconnection, UPS and Generator support. However, you lose the advantage of using your own equipment (unless you colo)....
Just trying to compare the pro's and con's of each.
Thanks,
Damion
-
04-29-2004, 03:25 PM #21WHT Addict
- Join Date
- Dec 2002
- Posts
- 100
call me old fashion, but I like to be able to look into the server room and see the servers.
-
04-29-2004, 03:28 PM #22WHT Addict
- Join Date
- Dec 2002
- Posts
- 100
....we do point to points...and we can do a T for 550 or so that are on the same CO as we are.
-
04-29-2004, 03:30 PM #23Junior Guru Wannabe
- Join Date
- Apr 2003
- Posts
- 40
Originally posted by human39
....we do point to points...and we can do a T for 550 or so that are on the same CO as we are.
-
04-29-2004, 03:41 PM #24WHT Addict
- Join Date
- Dec 2002
- Posts
- 100
actually we have burstable OC lines.
that price I posted were for our customers that get T1's through us.
-
04-29-2004, 03:42 PM #25Junior Guru Wannabe
- Join Date
- Apr 2003
- Posts
- 40
Originally posted by human39
actually we have burstable OC lines.
that price I posted were for our customers that get T1's through us.
Do you normally just call up your local phone company get get it through them?