95% billing is *not* a rip off. I do not understand where people get that idea. I've heard that "if you burst to 2 MBPS one day a month then you're going to pay a lot more", but that's not even true. Considering a month is aproximately 30 days 1 day in each month is aproximately 3.4 percent. 95% discards the first *five* percent. Unless you get DoSed multiple times in a month, or you just use up more bandwidth than alotted to you most of the time you're going to be fine. (also: Most DoS attacks don't last for a full day, if so... I'd reccomend taking the ip down for a little while).

Most people seem to see 95% billing as a rip off, but I regard it as the exact oposite. When a person buys burstable bandwidth they pay for being able to get a little bit more ban for a while. Let's say a person wants to download a big file directly off of a server for a while: On a burstable connection they're going to get a much faster download, and most probably their download won't go for more than 5-10 minutes. On the "X GB/Month" deal the user will pay for whatever they downloaded, if they download some huge files they're going to pay for that in lower bandwidth for the rest of the month.

I can assure you that when people bill 95% they're not trying to rip you off. 95% billing is used so that a company can provide a burstable connection instead of a dedicated one where they would have to take some percent off the top of bandwidth and so that the user can't just rip off the company (if a user has X amount of gigabytes and are pay 50% then what stops them from *inciting* DoS attacks, etc?)

In a month's samples 95% shaves off a lot more than expected, and it gives both the user and the company some flexibility.

I'm probably wrong on half this information, but would like to be corrected so I could learn why users half qualms with 95%.

You're quite right. I think what it comes down to is that people don't understand it, so they are scared away.

Hmm, maybe this would help... people out there using 95th percentile billing, could you post numbers/pictures/URLs here to show people your monthly average and 95th percentile bandwidth utilization? (Complete MRTG graphs would also be helpful too, I expect).

Percentile bandwidth can be looked on in a purely statistical method. Percentiles are based on mean averages and a fundemntal idea is that extreme #s greatly effect mean averages.

For example, lets say your site uses 100 megs of bandwidth a day, but one day you get featured on cnet and it goes up:

So here's your bandwidth usage by day:

100 100 100 100 100 100 100
100 100 100 100 100 100 100
4000 3500 2000 1500 1000 500 250
100 100 100 100 100 100 100 100 100

Normally you'd use 100mb a day x 30 days = 3000 mb and an average of 100mb dialy

But this month you use 13450 mb for an average of 448mb... minus the top 5%

In short.. your paying for your extremely high dialy average even though your not using that much bandwidht. The irs works teh same, you pay estimated taxes on the asusmption that your income is constant all year long but if you're like me and make a good deal in just a few months, things get screwy.

paying for percentile bandwidth is fine if your site is constant with no real jumps. However, if you expect any fluxation you're far better off paying for true bandwidth, even if it smore expensive by the meg. I for one dont like the idea of paying for an average usage of 400 megs if I really only use 100 megs a day with a few short spikes. That doesnt make sense and is why I buy my bandwidth in the gig, not the %.

To see just what I mean check out rachshack's bandwidth ussage: http://www.rackshack.net/aboutus/networks.asp

look at connection #2 and #4 and you'll see serious changes. #2 is below 78M/s almost all month but for a few days it spikes... in fact it spikes alot, why would they want to pay for an average that includes that big spike? as you cna see its over 5%.... that means the fee would go up and up...

Thats why using percintile is a problem.. any spike over 5% (and many spikes are) hurts your pocket a lot where as paying for actual ussage doesnt.

Originally posted by TedS
To see just what I mean check out rachshack's bandwidth ussage: http://www.rackshack.net/aboutus/networks.asp

That's not traffic spiking -- that's rackshack changing their routing. You'll note that the dramatic changes there are mirrored by equal and opposite changes in the bandwidth utilization on their Cogent line.

Ok... thats a bad example then.

However, many sites do spike. One of my clients gets a steady flow of around 10,000 hits a day but hes on tv every now and then and when that happens, he goe sup to about 5 times that for a day or two. That means he pays for an overly inflated average simply because he signed up for 95% bandwidth pricing (which is why he didnt).

For many sites it works but for anyone converned about spikes its a bad idea.

Ted, you have to remember thou, any large provider who deals with their own leased lines is billed by the 95% method, its either that or get a full capped line (DS3, Oc3, Oc12 etc) Which they have to over buy to allow for your bursting.

I think you have to be aware of your traffic patterns and choose the best method for your situation. One is necessarily better or worse per se, but in some cases, one would most certainly be better.

In most cases, 95th percentile works out to be 1.5 X what average billing would be.

Originally posted by cperciva
You're quite right. I think what it comes down to is that people don't understand it, so they are scared away.

Hmm, maybe this would help... people out there using 95th percentile billing, could you post numbers/pictures/URLs here to show people your monthly average and 95th percentile bandwidth utilization? (Complete MRTG graphs would also be helpful too, I expect).


Not really representative though as this is about half the traffic this server normally sees

Bandwidth is bandwidth... we pay for BANDWIDTH, data that's transferred in and out of our servers. So 'actual' measurement is the more realistic type of measurement to use, since we're paying for EXACTLY what we use during each month.

Alan

Originally posted by RackMy.com
In most cases, 95th percentile works out to be 1.5 X what average billing would be.

Could you please elaborate on this?

Originally posted by Walter
Could you please elaborate on this?

Most links have traffic patterns such that on average 200GB is transferred for every 95th percentile Mbps-month. (Once you get about 2-5 Mbps, the issue isn't "spiky" bandwidth utilization; the issue is the daily and weekly periodic variation).

Bandwidth is bandwidth... we pay for BANDWIDTH, data that's transferred in and out of our servers. So 'actual' measurement is the more realistic type of measurement to use, since we're paying for EXACTLY what we use during each month.

I'll have to disagree there.

Lets say you transfer a constant 300 gig a month, thats a little less than 1mbps.... but lets say in that time, each day you bursted to 2.5mbps for just a short time. Your using the capacity of a 2.5mbps pipe but yet only paying for a 1mbps pipe.....so how exactly is it more realistic? Actual is only realistic in capped line situations where you have X bandwidth and there's no way you can go over X bandwidth. So if your buying 300gb a month or 1mbps you'd be capped @ 1mbps.

I'll have to disagree there.

Lets say you transfer a constant 300 gig a month, thats a little less than 1mbps.... but lets say in that time, each day you bursted to 2.5mbps for just a short time. Your using the capacity of a 2.5mbps pipe but yet only paying for a 1mbps pipe.....so how exactly is it more realistic? Actual is only realistic in capped line situations where you have X bandwidth and there's no way you can go over X bandwidth. So if your buying 300gb a month or 1mbps you'd be capped @ 1mbps.

No.

I'm talking about places that have bustable bandwidth. They've got the lines, they've got the bandwidth available for their clients to use. So what you pay for is how much bandwidth, in terms of data transfer YOU have used. Obviously, you can only burst to a certain point (the full capacity of the line). So you shouldn't compare it to capped lines. I was talking about DATA TRANSFER, not about the line capacity, you pay for how much data you transfer.

Alan

Here .. take a look at a Monthly Graph for a server

you can see the Actuall Bandwidth Usage versus the 95th percentile .. I honestly thing 95% is allot more fair to everybody, since you as the customer knows the bandwidth will allway be avail, and the NOC can cover it's expenses .. anyway .. you judge

PS - Last month on the graph does correspond to dec 1 - jan 1

Dumb question - but which graph/stats are usually used for percentile pricing (ie, 5 minute average, hourly average, what?) Because a monthly MRTG graph shows two hourly average right?

:)

Carlos, what program did you use to generate that?

Is there no one who can actually explain this where it's understood?

We're like 15 messages into this thread and no one has yet exactly descriped how it works for sure.

I feel dumb :eek:

http://www.svwh.net/products/95percentile.shtml

The above explains it fairly well.

This is my understanding:


95% Bandwidth:

Most companies gather statistics of how much bandwidth you use, they then chop off the top 5% of those statistics. You pay for "the 95%" of what you use.

GB Bandwidth:

You pay for what you use. Many companies allot you a "maximum" amount of gigabytes/month. You pay for what you use, and no more or less.

Bandwidth is generally monitered by SNMP (I believe?). Most companies give you 5 min MRTG stats.

so basically, they chop off the top 5% of the 5 minute interval tests, and you pay for the biggest burst left after that?

What I'm trying to figure out if I'd be better to buy a .5 mbit connection which uses 95% or if I'm truly cheaper off using the flat rate per gig.

Man guess I need to enable snmp on my web server and do the 5 minutes myself since Rackshack doesn't do 5 minute graphs except for 1 day on your stats page.

if you're using around 150 GB/month I think your best option may be to go with flat bandwidth. I generally recomend 95% billing to people who are using 2 MBPS+ as your usage becomes more predictable.

If you looked at your MRTG stats you would probably find out which one is better for you, but if you have site that gets a lot of "night hits from latenight surfers" etc. a set allotment of Gigabytes would be a better idea.

Originally posted by Revelation
if you have site that gets a lot of "night hits from latenight surfers"

Translation: If you have a pr0n site ;)

Here is my graph for the last month - traffic generated by my servers. You can see, that traffic at night and during weekends
is 3 - 4 times lower that during business hours, so in a case with 95% - i would have to pay a lot more (cause spikes are much longer than 5%) than with simple per/GB accounting.

However, we in russia, pay only for incoming international traffic and local traffic in russia for most ISPs is free, due to peering agreements.

I think that situation is similiar in USA, it is easy for large operators to arrange free traffic exchanges, but they do not provide this traffic free for their customers. Am i right?

Originally posted by Planet Z
Carlos, what program did you use to generate that?

I can answer that for him... it's MRTG + RRDTool + 14all.cgi + some extra code I hacked in for him.

Originally posted by cperciva


I can answer that for him... it's MRTG + RRDTool + 14all.cgi + some extra code I hacked in for him.

Yep :)

We had CPercival code that for us, plus some he was working on already

Originally posted by cperciva


14all.cgi + some extra code I hacked in for him.


greetings,

can you send me a copy of it as well?

you got mail, Thanks


elijaH :)

Originally posted by elijah



greetings,

can you send me a copy of it as well?

you got mail, Thanks


elijaH :)

err .. that costed $$ to have it made

good evening Carlos,

i just got an email from CPerciva.
we'll be paying for it to use it commercially and have no probs with it.

thanks.


elijaH :)

Hear hear for a good discussion.

I believe a key point missing in this isn't just about bursting versus actual bandwidth usage.

The issue is about QoS (quality of service).

We bill by the 95th percentile because we are guaranteeing that the bandwidth will be available to our clients.

When you pay per gb of transfer, there are no service guarantees about quality.

To use an analogy that hopefully clarifies:

When you rent a car, they charge you for the number of miles driven (bandwidth transferred). At the end of the rental, they look at the odometer and you pay up.

However, this says nothing about the QUALITY of your drive. If you are stuck in traffic the whole time, and average 20 mph, and it takes 8 hours to go somewhere that is only one hour away, that's a lot different driving experience than cruising along the highway at 55 or 85 mph.

Charging by 95th percentile is equivalent to charging you not by miles driven, but by your average speed - miles per hour. If you sign-up for 60 mph, you are guaranteed that you can ALWAYS drive this fast, and a little faster (5% bursting), but you will NEVER slow down to 5 mph some of the time.

I.E. you are guaranteed a QoS (quality of service) level of pure bandwidth at least equal to what you are willing to pay for.

95th percentile is like renting a private commuter lane on the freeway -- no matter how many other people are jamming the other lanes going to the beach, you can speed along at 80 mph while they are stuck going 20 or 30 mph.

The whole point of 95th percentile is that the provider is NOT oversubscribing their links - they are guaranteeing the bandwidth will always be there for you. Since they are in effect RESERVING this bandwidth for you whether you use it or not, you pay whether you use it all or not, based on how much you want reserved.

Funny thing - 95th is used by backbone providers, tier 1 colo, and boutique data center/colo companies that care about quality and have writen QoS policies. While GB of transfer is used by hosting resellers and others that mostly compete only on price.

Corrollary: It usually takes hardware and expertise to measure and report on 95th percentile and traffic shapers or expensive switches/router to control it. It only takes a web log to count bytes transferred. I suspect a portion of the hosting resellers don't do 95th because they don't have the capability to accurately measure or provision the bandwidth.

spiv - that's an excellent explaination. Thanks. :) But what I want to know is this... can 95% pricing let me get ahead of all the other at the McDonnalds drive-thru? :D

Hey spiv,

I hope I am not being rude, but that makes no sense at all. So what you are saying is that a host that offers bandwidth by actual usage do not off a quality service and 95th percentile hosts do?

Corrollary: It usually takes hardware and expertise to measure and report on 95th percentile and traffic shapers or expensive switches/router to control it. It only takes a web log to count bytes transferred. I suspect a portion of the hosting resellers don't do 95th because they don't have the capability to accurately measure or provision the bandwidth. A lot of hosts bill by the total transfered because customers seem to understand it better than 95th percentile billing. How many times do I see in the forum someone asking for someone to explain 95th percentile billing. See what I mean?

Also, it's actually easier to calculate 95th percentile using MRTG and/or RRDTool (which I would say 95% of hosts use) than it is to get actual measurements.

Sort of --

What I said is that when billing for total transfer usage, that is the only thing being billed - how much data has moved through the network during a 30 day billing period.

There is no quality statement, no mention of performance, you guarantee of how LONG each bit of data will take nor promise of latency, delay, etc.

I did not say that every company billing only by data transfer has a lousy network or infrastructure, only that the billing for the customer does not include any QoS promise or measurement.

In contrast, 95th percentile is a quality/usage measurement. It is a promise by the provider to guarantee a level of bandwidth and quality for a specific price.

Again - why do you think virtually all wholesale (and backbone) Tier 1 bandwidth is sold via 95th percentile? Because savvy buyers want to know what they are buying and without QoS and bandwidth minimum guarantees, they know they have no recourse.

With 95th, the measurements are out there and they can "keep their supplier honest".

The reason people here don't understand it -- well, I believe most of the people here are entrepreneurial small-business resellers or first-time co-location customers that have very little prior experience with the innerds of the data center, the core backbone, BGP routing, etc.

They are savvy business people, but unfortunately often don't know what to ask for. Afterall, most posters here are obsessed with saving every last buck but don't seem to care much about quality of service, backbone redundancy, data center reliability, etc.

If you view everything as a commodity paperclip, then, yeah, sure, some fancy algebraic formula that attempts to be fair to both buyer and seller while providing an objective QoS measure is just too much "rocket science".

The argument about which method "costs more" or "overcharges" the customer is totally bogus.

The important question is -- does the customer want full bandwidth capability and pay fairly for it, or does the customer want to get by with the least bandwidth and cost they need and hope their customers don't notice degraded service during those periods when they exceed their bandwidth and/or their provider can't give them full bursting.

Remember the airlines -- when travel was busy, overbooking (selling more bandwidth than exists) was a big problem and many people were affected by being bumped off flights or not getting the flights they want on short notice.

Now that travel is off quite a bit, we hear very little about it.

Same with data centers -- if you never burst and your bandwidth usage isn't growing, then you will usually get quality bandwidth and the issue of 95th versus straight usage will be insignificant both in cost and quality.

How do you want to run your business? Planning for success and growth, or arbitraging every last penny trying to make a profit by "cutting corners" and hoping your clients don't notice?

You are still not making any sense:

There is no quality statement, no mention of performance, you guarantee of how LONG each bit of data will take nor promise of latency, delay, etc. What does this have to do with 95th percentile.

In contrast, 95th percentile is a quality/usage measurement. It is a promise by the provider to guarantee a level of bandwidth and quality for a specific price.How is it verses an average or actual measurement?

95th percentile is just another way to measure/bill for bandwidth and has nothing to do with the quality.

Most tier 1 providers *do* use 95% because it's really impossible to bill in "Gigabytes used". People would be going nuts if they were placed on a 5 MBPS line with 15 other people, and told that they can "use" 300 GB/Month. (1 MBPS is aprox 320 GB/month).

When a person buys in 95% they buy in MBPS, and generally are alloted that much bandwidth + a little bit from their upstream link (Verio, Ardent, UUnet, etc.). When they buy in GB they generally are getting oversold bandwidth where if each person used most maximum alloted amount of gigabytes the network would run at 100% latency. If they all tried to use their maximum the network would just become crap :)


From Verio(95%) -


Network nodes on Verio's Backbone Network create a highly redundant system for measuring and reporting on the health of the Verio Backbone network for our customers.

As described below, Verio's Backbone Network Service Level Agreements (SLAs) provide commitments based upon goals in three key areas. Click on the following links to view current performance measurements with respect to the Verio Backbone SLAs:

Availability: Network connections from the Verio data center to the Internet will be available to customers free of Network Outages 100% of the time.*
Packet Loss: The average monthly Packet Loss over network connections from the Verio data center to the Internet will not exceed .3%. *
Latency: The average monthly Latency over network connections from the Verio data center to the Internet will not exceed 60 milliseconds round-trip.*
* The foregoing commitments are subject to certain definitions and limitations set forth in the applicable Verio Backbone Network SLA. For complete details of the Verio Backbone Network SLAs, go to Verio Backbone SLA Contract Terms.

From Rackshack (Gigs/month)-

Don't have one :) Rackshack won't garuntee anything, and most people who bill in gigs/month won't either as their bandwidth is oversold and generally not very redundant.

From VDI (Gigs/month)-

Couldn't find one either :P

Only the 95% is garunteed to be there. Not oversold, redundant, it's your choice, but IMO 95% = more reliable.