Page 1 of 2 12 LastLast
Results 1 to 25 of 32
  1. #1
    Join Date
    Apr 2004
    Location
    Houston, TX
    Posts
    80

    how is this possible ...

    I have a T1 running into my small hosting facility, so the total possible bandwidth should be around 487GB / month.

    I wrote a script to add up the web traffic based on my Apache web logs. When I add up the size of the files served out by my webserver, from the Apache logs, *several* sites on my webserver are serving out a total of over 400GB / month.

    That can't be right.

    I've tried narrowing it down to only log entries which resulting in a result code of 200, which according to W3C means the file was successfully transferred, but it still adds up to over 400GB / month for each of several busy sites on the server.

    I thought maybe it was my log parsing script not working right, so I saved a log and FTP'd it over to a Windows system, opened it in Excel, sorted it by result code, added up the lines with a result code of 200 ... and still got a total of over 400GB in a single month.

    Am I missing something? How can my webserver be serving out over 1200GB per month when my T1 would be completely saturated 24 / 7 at 487GB / month?

    Besides ... the line gets busy occsaionally during the day but there's long periods of time with little or no traffic. And most T1's actually can only handle about 350GB per month after errors and retransmits and stuff get factored in.

    Someone please point me to a apache logs 101 or something ... I *must* be misunderstanding some very basic concept here ...
    http://www.dynamis.net
    A small domain host focusing on personalized service.

  2. #2
    maybe gzip is installed?
    Pro-net-hosting.com - www.pro-net-hosting.com
    Pro hosting solutions since april 2003.

  3. #3
    Join Date
    Apr 2004
    Location
    Bulgaria
    Posts
    159
    if I'm not mistaken a T1 line has a capacity of 10mpbs. At full utilization this would add up to over 3000GB per month
    Solid-Hosting.net - affordable multiple domain hosting solutions
    Reliability and Customer Service are our focus!

  4. #4
    Join Date
    Jul 2003
    Location
    Satyr, Chrisalya, Canada
    Posts
    1,901
    a T1 is 1.5mbps.
    --

  5. #5
    Join Date
    Apr 2004
    Location
    Houston, TX
    Posts
    80
    Originally posted by heavypredato
    maybe gzip is installed?
    I used a site which said it would check that, and it came back negative. However I'm not sure if it is correct. How can I check for sure? I did not see mod_gzip in the module list, but I found another entry

    AddEncoding x-compress Z
    AddEncoding x-gzip gz

    The comments for these lines indicate they allow some browser to uncompress information on the fly. However it's unclear whether they mean uncompressing gzip (.gz) documents on the fly or uncompressing compressed html files and such on the fly.

    Tom
    http://www.dynamis.net
    A small domain host focusing on personalized service.

  6. #6
    Join Date
    Sep 2003
    Location
    UK
    Posts
    184
    are you sure that you are working the math out correctly? 400GB for one site is pretty high (depening if they host any rich media or not).

    400GB would be 419430400 Bytes, are you sure its not 400MB?

  7. #7
    I would be more concerned it was 400 GigaBits, perhaps you the logs are recording bits and not bytes, and you are forgetting to divide by 8.

    Just a thought.
    LipWeb.Net
    "Less Lip More Service"
    Providing Quality Hosting at Fair Prices
    AIM: LipWebNet MSN: daniel[at]lipweb.net [color=red]

  8. #8
    Join Date
    Sep 2003
    Location
    UK
    Posts
    184
    Yeah I did think about that, surely apache logs in bytes though? I have not used apache before so have no idea lol.

  9. #9
    Join Date
    Apr 2004
    Location
    Phoenix, Arizona
    Posts
    151
    Do the logs make the distinction between what was really served and what was checked, but not served because of the cache on the client side?


    just a thought.


    Dan
    http://www.IwantFUI.com
    If you could host a new kind of content from your old-fashioned web servers
    and make new money from your customers and differentiate your business all at the same time... could you afford not to try? See the new site

  10. #10
    Join Date
    Apr 2004
    Location
    Houston, TX
    Posts
    80

    * FOUND THE PROBLEM ... IT WAS ME ...

    Delirium, thanks .... I have found the problem and it was me.

    The total number of bytes for the 1st 9 days of the month was:
    496,060,118

    That's 484,433KB
    or 473MB
    or .46 GB

    I was skipping the KB step in my computation.

    419,430,400 Bytes is
    409,600 KBytes is
    400 MBytes is
    .39 GBytes

    Guess that's what I get for working the problem at 2am ...

    Thanks a bunch guys for all the input !!!! WHT rocks!!!!

    Tom
    http://www.dynamis.net
    A small domain host focusing on personalized service.

  11. #11
    Join Date
    Apr 2004
    Location
    Houston, TX
    Posts
    80
    Originally posted by delirium
    are you sure that you are working the math out correctly? 400GB for one site is pretty high (depening if they host any rich media or not).

    400GB would be 419430400 Bytes, are you sure its not 400MB?
    Well actually it seems that per my new discovery of the true nature of gigabytes , 400GB is actually
    429,496,729,600 bytes.
    http://www.dynamis.net
    A small domain host focusing on personalized service.

  12. #12
    Join Date
    Sep 2003
    Location
    UK
    Posts
    184
    Originally posted by dynamis
    Well actually it seems that per my new discovery of the true nature of gigabytes , 400GB is actually
    429,496,729,600 bytes.
    Glad you found where all those gigs were going hehehe

    Sorry you are correct with your math above (I needed to multiply by another 1024),

    In future you can use google to work it out open google and type

    400 gigabytes in bytes

    as your query - Google will return 400 gigabytes = 429 496 729 600 bytes

    obviously you could run this query the other way round

  13. #13

    Re: how is this possible ...

    Originally posted by dynamis
    I have a T1 running into my small hosting facility, so the total possible bandwidth should be around 487GB / month.

    I wrote a script to add up the web traffic based on my Apache web logs. When I add up the size of the files served out by my webserver, from the Apache logs, *several* sites on my webserver are serving out a total of over 400GB / month.

    That can't be right.

    I've tried narrowing it down to only log entries which resulting in a result code of 200, which according to W3C means the file was successfully transferred, but it still adds up to over 400GB / month for each of several busy sites on the server.

    I thought maybe it was my log parsing script not working right, so I saved a log and FTP'd it over to a Windows system, opened it in Excel, sorted it by result code, added up the lines with a result code of 200 ... and still got a total of over 400GB in a single month.

    Am I missing something? How can my webserver be serving out over 1200GB per month when my T1 would be completely saturated 24 / 7 at 487GB / month?

    Besides ... the line gets busy occsaionally during the day but there's long periods of time with little or no traffic. And most T1's actually can only handle about 350GB per month after errors and retransmits and stuff get factored in.

    Someone please point me to a apache logs 101 or something ... I *must* be misunderstanding some very basic concept here ...
    Kind of OT, but do you mind if I ask how much you pay for that T1?

  14. #14
    Join Date
    Apr 2004
    Location
    Houston, TX
    Posts
    80
    It's $523 per month including local loop.

    Tom
    http://www.dynamis.net
    A small domain host focusing on personalized service.

  15. #15
    Originally posted by dynamis
    It's $523 per month including local loop.

    Tom
    Wow, that is pricey. Do you primarily just do hosting with that? How many servers/clients are you able to run on that?.

  16. #16
    Join Date
    Sep 2002
    Location
    Top Secret
    Posts
    14,135
    I've found this site to be very helpful when dealing with bandwidth and conversion from bytes to gb, mb to gb or anything of the like. If you have to do this in the future, you might want to head over there, save yourself a LOT of math and legwork
    Tom Whiting, WHMCS Guru extraordinaire
    Linux problems? WHMCS Problems? Give me a shout
    Check out my WHMCS Addons

  17. #17
    Join Date
    Apr 2004
    Location
    Kansas City, MO
    Posts
    53
    That's an extremely handy site.
    Take care,
    Brad Birmingham
    http://www.bluevirtual.com

  18. #18
    Originally posted by noimad1
    Wow, that is pricey. Do you primarily just do hosting with that? How many servers/clients are you able to run on that?.
    here in western PA, thats not pricey at all. Anything less that that I would be too worried about the company going under in a year.

  19. #19
    Join Date
    Sep 2002
    Location
    Top Secret
    Posts
    14,135
    yeah, I meant to say something about that one myself.
    I called a local cable (also doing inet and t1) provider recently and the prices they quoted me made what you pay look like nothing. IIRC they wanted around $1.5k / month and that DIDN'T include telco or building charges.
    $500 seems to be about the average right now though.
    Tom Whiting, WHMCS Guru extraordinaire
    Linux problems? WHMCS Problems? Give me a shout
    Check out my WHMCS Addons

  20. #20
    Originally posted by wolfstream
    yeah, I meant to say something about that one myself.
    I called a local cable (also doing inet and t1) provider recently and the prices they quoted me made what you pay look like nothing. IIRC they wanted around $1.5k / month and that DIDN'T include telco or building charges.
    $500 seems to be about the average right now though.
    I guess I just say pricy compared to just getting a dedicated server. Which is why I guess I asked how many servers with decent loads, you could run off that connection.

    It is certainly handy to be able to have complete control of the server/hardware, but is it completly worth it as a hosting company?

    Do you have UPS/Generator backup at your facility?

    Please don't take this e-mail the wrong way. I am not trying to flame anyone for using their own data connection, but I am just trying to see the advantages. Or see if this is something I might want to look into myself.

    Our servers use around 100GB of bandwith each....so I could only get 4 servers max off that T1. But then again, how well does it handle bursting etc. If all 4 servers were running at full capacity, I would have to assume the dataconnection would slow rapidly?

    Then, at the same price about $500-600 I could get 4 dedicated servers in a datacenter that has the larger dataconnection, UPS and Generator support. However, you lose the advantage of using your own equipment (unless you colo)....

    Just trying to compare the pro's and con's of each.

    Thanks,
    Damion

  21. #21
    call me old fashion, but I like to be able to look into the server room and see the servers.

  22. #22
    ....we do point to points...and we can do a T for 550 or so that are on the same CO as we are.

  23. #23
    Originally posted by human39
    ....we do point to points...and we can do a T for 550 or so that are on the same CO as we are.
    Do you also run a T1 connection? How many servers are you running on it?

  24. #24
    actually we have burstable OC lines.

    that price I posted were for our customers that get T1's through us.

  25. #25
    Originally posted by human39
    actually we have burstable OC lines.

    that price I posted were for our customers that get T1's through us.
    I see. So what kind of price do youapy ofr the burstable OC lines?

    Do you normally just call up your local phone company get get it through them?

Page 1 of 2 12 LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •