Anyone measure how much transfer they actually get with 95th percentile measurements? Assuming you get 315GB for each 1Mbps. We're charged just for outgoing (like most DC's do) and are seeing about 65% of possible.
We're using 2.4Mbps (95th measurement) on a link and project 475 GB usage for 30 days. In reality outgoing projects 378 GB for the 2.4, which is 50%. Our traffic is typical of most hosting companies - up during the day, down at night, lower on weekends..Crazy that w/ 95th percentile you really only get 50%!

Usually average levels are around 2/3 of the 95th percentile levels.

Originally posted by marksy
Anyone measure how much transfer they actually get with 95th percentile measurements? Assuming you get 315GB for each 1Mbps. We're charged just for outgoing (like most DC's do) and are seeing about 65% of possible.
We're using 2.4Mbps (95th measurement) on a link and project 475 GB usage for 30 days. In reality outgoing projects 378 GB for the 2.4, which is 50%. Our traffic is typical of most hosting companies - up during the day, down at night, lower on weekends..Crazy that w/ 95th percentile you really only get 50%!

Which is one of the reasons I dislike 95th percentile. I understand its role in smaller businesses who do not lease whole GigE's, etc, but for companies who already utalize most of their line...

Hi:

Comparing 95th percentile is almost like comparing Miles/Kilometers Per Hour to Gallons or Liters they are very differant in terms of how measurements are taken.

Trying to determine the "Actual" GB's based on your 95th reading is like trying to figure out "exactly" how much gas you would use in a car if it travelled 65 Miles/Kilometers Per Hour for 30 days --> almost impossible with all the differant variations that could take place.

Your best and most accurate measurement would be calculating GB's direclty from the MRTG log files, this in the end will assist in calculating a final tally that should be right on target for you.

Aaron Phillips

The best you can do is compare what you pay to what you get. Bear in mind that the majority of backbone providers out there bill their customers with 95% billing. This all came from the fact that originally, networks were pipes. When bursting became a "hot idea" people had to figure out how to bill it to justify the cost of their pipes/infrastructure.

Billing on actual usage is not "natural" for most providers. In one way or another, they must recover their costs which in most cases are either full-pipe or 95% usage-based costs. Some providers will quote both Mbps and GB/month for you, but in the end, convert it to see what you are paying.

Our measurements are snmp 5 minute intervals - not mrtg but essentially the same. So the transfer is pretty accurate.

Our 95th is generally 80% more than our average usage and our actual usage (We measure all 3).

If your 95% rates are killing you, ask your provider to set a cap and figure a round about number for the max you want to spike.

This has helped a lot in our situation.

For a newbie, could someone explain the methodology of 95th percent billing?

There are plenty of posts concerning 95th and how it works, i'd repost it again but good examples exist.. search the forum for posts by jamesc (James from NAC), he explained it best.

(ok fine I know, people are lazy so here is a SHORT version)

every 5 minutes samples are taken (for example, and this is how its usually done)

here are the outputs:

1 - 10
2 - 15
3 - 20
4 - 17
5 - 50
6 - 234
7 - 8
8 - 4
9 - 10
10 - 123
11 - 46
12 - 172
13 - 123
14 - 21
15 - 5
16 - 83
17 - 63
18 - 47
19 - 37
20 - 13

as you can see, we have 20 samples..

sort them, highest to lowest, remove top 5% (what is 5% of 20, 1)

so you remove the 1st sample (being the highest, we sorted it remember?)

so 234 is removed, now what is the next highest? 172.. so 172 is your # to be billed on....

ofcourse this would be bandwidth and not just #'s, but hopefully you get the point..

The easiest way to understand it is by looking at a graph, eg. http://traffic.datacolo.com/switch1/14all.cgi?log=datacolo-sw1&png=daily

See that pale blue line? That's the 95th percentile. Out of the 400 five-minute samples shown, 5% (ie, 20 of them) are above that line, while the rest are below. 95th percentile is often referred to as "peak sustained" bandwidth -- you can see that isolated spikes are filtered out, but the 95th percentile line reflects the highest level which is sustained for a significant time.

That's why the 95th percentile level is so useful -- when a large ISP is aggregating the traffic from all its customers, their individual spikes will even out, but the peak sustained levels will add up (because almost all web sites have the same overall daily trends).

oops, double post

So based on Sprynex's example. Assuming those numbers were bandwidth, and not just random numbers. Your actual bandwidth usage is 1101 (added em all together). While being billed at 95%, your bandwidth usage is 3440 (172 * 20).

Not totally sure if this is right. But if so..I don't understand how that's fair. Your being billed for 3x the amount of bandwidth your actually using. (again I know these are just random numbers..but for the sake of an example, that's what I'll use).

Oh well. *shrugs*

kfreak, it's perfectly fair, because you pay a lower rate when your bandwidth is measured based on the 95th percentile level. For example, an ISP might offer the option of paying $150/Mbps of 95th percentile level, or $250/Mbps of average level.

Ahh, yea, I guess I get that.

Btw, holy crap, nice sig (althought canada sucks..jk) :-)

Originally posted by cperciva
Usually average levels are around 2/3 of the 95th percentile levels.

This sounds like a decent rule of thumb, in December our 95% in one data center was 9.8 mbits and we transferred right at 2200 gigs.

No actual usage would not be that.. actual usage would be all #'s added up / $count

My example was poor for actualy "data" because I was getting bored and just randomly hitting keys ... if you were to take a look at more bouncy traffic.. like lets say two of those measurements was 5000, you would see how 95th could hurt you.

As a lower end customer (meaning, not a datacenter) I would stay away from 95th whenever possible.. while it may be fair to an isp, it's not good to start out with it, as it generally leads to higher usage bills.

With the 95% rates you can decrease your usage and beat your ISP in its own game by using compression algorithms such as mod_gzip and without affecting your server's performance.

No matter what method you're billed on mod_gzip can decrease yur usage, but that still doesn't change the fact that your 95th would work out more than average or actual usage.

I don't see where it really matters. A competent host who truly understands the different billing mechanisms will have appropriate rates for each, i.e. the 95th rate might be X, the average billing rate might be Y, and the GB transfer rate might be Z, where X<Y<Z.

If X=Y=Z, then of course you'd want to go with the most advantageous billing method, in this example per GB pricing.

Brandon

for example - if you get a burstable pipe from us on 95% you will pay about 125 per mbps. if you want to buy on actual trasfer - you will pay about 1.00 per gig. Do the math. I think 95% is still cheaper by far for higher bandwidth users.

Originally posted by cbtrussell
I don't see where it really matters. A competent host who truly understands the different billing mechanisms will have appropriate rates for each, i.e. the 95th rate might be X, the average billing rate might be Y, and the GB transfer rate might be Z, where X<Y<Z.

If X=Y=Z, then of course you'd want to go with the most advantageous billing method, in this example per GB pricing.

Brandon

95th is good for sites with established traffic patterns. I personally don't think it would be good for some web hosts with erratic traffic patterns.