Web Hosting Talk : Web Hosting Main Forums : Colocation and Data Centers : Feedback Requested - 95th Percentile Explained

[jsw6]

I feel like we go through an explanation of 95th percentile billing about once a week on the co-lo forum, and while referencing old posts is usually time saving, it seems like folks still need more detailed answers from time to time.

I've spent the past couple of hours writing what I feel like is a fairly thorough, detailed explanation of how 95th percentile billing is done and how it compares to per-GB or mean port utilization billing. 95th Percentile Explained. I would appreciate feedback. Don't bother clicking around too much on the site. There are lots of broken links and this is the only thing that is really ready for public consumption.

[mgphoto]

Very nice job Jeff.

[Dennis Nugent]

commendable effort


suggestions for clarification on "How is my 95th percentile rate calculated?"
The 95th in the name of this billing model has specific meaning. In order to calculate the (percentile) traffic rate for which you will be billed, your ISP sorts (in descending order) the (5 minute) samples taken during your billing period, then ignores the highest five percent of those samples. (The next sample is the 95th percentile traffic rate)
The top 432 samples are not relavent to your bill. This means that, in a 30 day billing period, you can burst for 36 hours.


----------------------------------------------------------------
and two on units:

$50/Mbit--> $50/mbps 11th row from the bottom

$88/Mbit --> $88/mbps 10th row from the bottom

[cabalstudios]

nicely done Jeff

[cbtrussell]

Great! It's about time someone gave us something newer than the old Sean Adams standby

I think you've done a great job! Here are a few constructive comments, I hope you will take them in the spirit with which they are intended.

- I would move the detailed examples to the bottom of the page. The 324GB question and related topics are much more relevant to your core audience, and I'd hate to see some of them give up before they get there.

- "How many gigabytes are in a megabit/sec?"
Consider going the other way as well, i.e. "My host says I transferred X GB last month. What is the 95th percentile equivalent?" I would emphasize that there is no reliable way to make this determination without a corresponding value for throughput during the same period.

- "To determine the number of gigabytes transferred you simply multiply the mean utilization, represented on the example graph at 3.11 Mbit/sec, by the time span of the billing period."
Consider explaining exactly what mean utilization is. I think this is particular important given most out-of-the-box metering software configurations display 'average' values.

- "That 324GB per Mbit/sec figure is roughly accurate for a 31 day calendar month of mean utilization,"
I think it would be beneficial to emphasize you're referring to 31 days of *continuous 1mbps* mean utilization, being sure to tie back to the previous definition of mean utilization.

- "which tells us that the example traffic is about 56% efficient vs a 100% efficient traffic profile, where the mean utilization is equal to the 95th percentile utilization"
Maybe it's just me, but I think it's easy to miss you're actually defining 100% efficient here.

- "My host offers per-GB and 95th percentile billing. Which is better?"
The entire answer hinges on your one example, no doubt appropriately chosen for this specific application . While I agree your analysis & result is orders of magnitude better than the oft-quoted 324 per 1mbps reference, in the interest of full disclosure it would probably be a good idea to clarify that the 190GB cited could be considerably higher or lower based on the reader's specific circumstances. That of course would then impact the subsequent ratio results of 0.56 and 1.76, respectively. Not a big deal of course, I would just hate for those values to become ingrained as defacto standards without going further to clarify that they are based on a specific (albeit typical) observation.

- "Example Customer Equivilant Costs"
Equivalent

Again, great job. Thanks for taking the time to post this since it's sure to be referenced at least once a week on this forum alone

Brandon

[ThinIce]

Thanks very much for rolling your exisiting posts up into this guide and to anyone who makes contributions.

For those of us laboriously studying the industry before deciding whether to join it, peer reviewed / community produced guides like this are worth their weight in gold.

[Mfjp]

This needs to be put a Sticky!

[jsw6]

I have incorporated a number of the suggestions posted in this thread into a revised version of the document. I did decide not to rearrange the example to the bottom since a lot of my explanation depends on assumptions provided by the sample data set. I realize some folks may not read that far, though. That is why I included a Contents link at the top, as well as hyperlinks to the commonly asked questions and associated answers.

Several people asked why I use the term mean utilization so often, and do not refer to average utilization. The word average is somewhat broad, and is correct for three different types of statistical calculation -- mean, median, and mode. Most people incorrectly believe that average is actually defined as mean is, but that is not always the case. When money is on the line, I prefer to be precise.

Keep the feedback coming!

Originally posted by cbtrussell
The entire answer hinges on your one example, no doubt appropriately chosen for this specific application

Actually, I selected that data set at random from a group of about 50 customer ports with mean utilization exceeding 2Mb/s. It's amazing how often web hosting traffic comes incredibly close to the profile I represent to be "typical of web hosting traffic."

Hosting / datacenter companies, on the other hand, will find that with a diverse customer base, they'll have a better mean to 95th utilization ratio. This is especially true when you start to pick up clients who serve content to end-users outside the US. There is economy of scale there.

I would love to have folks email me other RRD files for specific applications. I do not have access to data for a large set of game hosting companies, for example, so any example I selected might be skewed by customer base growth, game outages, marketing to players in a specific time zone, etc. If I were to accumulate a larger data set for game servers, I'd be happy to produce a second set of example data for that popular, growing field.

[TGM-Admin]

Great breakdown....

[Unknown_User]

Ok well I read the first few lines, and I have wondered about this for a while, so I am going to read over this in the evening, I have sent the URL to my self in an e-mail, thankyou for this document it looks fantastic.

Jeff, as it seems you have a few minutes spare Maybe you could do a similar document on Bandwidth:

Carrier
Transit
Peering

I know MANY people will find this beneficial - my self included

DislexiK

[Tom O'Connor]

A very good job indeed.

[TBergman]

Great job as usual Jeff. Im still waiting on you for your index of help tutorials. Maybe you should sell them off in a suite DemoDemo style. It would probably be of great asset to a few companies I can name off the top of my head.

Regards,

Troy

[jsw6]

I've never heard of DemoDemo. I didn't really intend to make money off that. I just thought it'd be easier to put together a thorough document so I can stop posting about that topic once a week

[lumbyjj]

Outstanding job Jeff, as usual you have provided us with enough knowledge to soak our brains to the fullest!