Page 1 of 2 12 LastLast
Results 1 to 25 of 26
  1. #1
    Join Date
    Apr 2003
    Location
    San Jose, CA.
    Posts
    1,624

    95th percentile billing polling interval.

    Why do seemingly most companies do 95th percentile billing based on a 5 minute polling interval?

    Do you think there would be any complaints about basing 95th percentile billing on a 1 minute polling interval?

    Thanks.

  2. #2
    Join Date
    Nov 2002
    Posts
    2,780
    Alot of carriers have defined their polling interval as 5 minutes in their contract.

    1 minute interval will give a more accurate usage level, but it is also harder on the system when there are a large number of devices being monitored.
    http://Ethr.net jay@ethr.net
    West Coast AT&T / Level3 / Savvis Bandwidth, Colocation, Dedicated Server, Managed IP Service, Hardware Load Balancing Service, Transport Service, 365 Main St, SFO / 200 Paul Ave, SFO / PAIX, PAO / Market Post Tower, 55 S. Market, SJC / 11 Great Oaks, Equinix, SJC

  3. #3
    Join Date
    Apr 2001
    Location
    United Kingdom
    Posts
    1,651
    5 minutes seems to be the standard, however we use 1 minute for our customers.
    Coreix™ | your solution™ www.coreix.net - 08000226734 - ISO27001 Certified
    Dedicated Servers - Colocation - Secure Cages - Enterprise & Managed Solutions - Clustering - LoadBalancing - High Availability - Hosted Exchange 2007
    NEW Tier III London DC with power sourced from 100% renewable sources, N+N UPS, N+1 Chillers and CRAC units, Mist Fire Suppression, Biometric and Man Trap security to all data areas

  4. #4
    Join Date
    Apr 2004
    Location
    London
    Posts
    390
    On another note, anyone know off hand if it's easy to modify cacti to use the same interval (i.e. 5 mins) for all graphs (daily, weekly, monthly etc)?

    Kris
    Kris - nitrohosting.net
    Premium Dell Windows/Linux dedicated servers in London
    Great value shared hosting & ADSL.
    PM/Email for no-obligation quotes.

  5. #5
    Join Date
    Oct 2004
    Location
    Nevada
    Posts
    887
    'Do you think there would be any complaints about basing 95th percentile billing on a 1 minute polling interval?'

    Not as long as you still tossed out the top 5% of the data when calculating the invoice

  6. #6
    Join Date
    Jan 2004
    Location
    North Yorkshire, UK
    Posts
    4,164
    We're using 30s, monitoring about 200 switch ports on a single Xeon 3.0 w/512mb, it's coping fine.

    It does annoy me when providers use the 5 minute model because small traffic spikes can look like they lasted a lot longer than they did and cost you more money.

    Dan
    █ Dan Kitchen | Technical Director | Razorblue
    █ ddi: (+44) (0)1748 900 680 | e: dkitchen@razorblue.com
    █ UK Intensive Managed Hosting, Clusters and Colocation.
    █ HP Servers, Cisco/Juniper Powered BGP Network (AS15692).

  7. #7
    Join Date
    Aug 2002
    Location
    UK
    Posts
    1,040
    Isn't it worse with 1 minute intervals? If you have a big burst for (say) 30 seconds, this is averaged out over 5 minutes normally, so you pay for a lower amount of bandwidth. With 1 minute intervals, it would be averaged out over 1 minute.

    Yeah, it's more accurate, but it's also more expensive

    I don't pretend to be an expert on this though.
    Robin Balen
    Gyron Internet Ltd - http://gyron.net/
    UK colocation, managed hosting and connectivity services with 100% uptime SLAs

  8. #8
    Join Date
    Aug 2002
    Location
    Seattle
    Posts
    5,525
    Quote Originally Posted by robinbalen
    Isn't it worse with 1 minute intervals? If you have a big burst for (say) 30 seconds, this is averaged out over 5 minutes normally, so you pay for a lower amount of bandwidth. With 1 minute intervals, it would be averaged out over 1 minute.

    Yeah, it's more accurate, but it's also more expensive

    I don't pretend to be an expert on this though.
    I would say you're correct.

  9. #9
    Join Date
    Jan 2004
    Location
    North Yorkshire, UK
    Posts
    4,164
    Quote Originally Posted by robinbalen
    Isn't it worse with 1 minute intervals? If you have a big burst for (say) 30 seconds, this is averaged out over 5 minutes normally, so you pay for a lower amount of bandwidth. With 1 minute intervals, it would be averaged out over 1 minute.

    Yeah, it's more accurate, but it's also more expensive

    I don't pretend to be an expert on this though.
    As far as I see it, a measurement of current throughput is taken from the switch every 5 minutes, and it is assumed that the current throughput is sustained for a whole 5 minute period, which in the case of a 30 second burst would amount to you being billed for 5 minutes of usage rather than 30 seconds ...

    I could be wrong though.

    Dan
    █ Dan Kitchen | Technical Director | Razorblue
    █ ddi: (+44) (0)1748 900 680 | e: dkitchen@razorblue.com
    █ UK Intensive Managed Hosting, Clusters and Colocation.
    █ HP Servers, Cisco/Juniper Powered BGP Network (AS15692).

  10. #10
    Join Date
    Nov 2005
    Location
    Minneapolis, MN
    Posts
    1,648
    Quote Originally Posted by RazorBlue - Dan
    As far as I see it, a measurement of current throughput is taken from the switch every 5 minutes
    I don't think it's a measure of throughput, the tools that I've seen go after In/Out Octets (bytes) and then do their own throughput calculation based on byte counter change since the last polling cycle divided by the duration since the last polling cycle.

    In that regard, using shorter polling cycles almost always produces higher values for 95% billing. Say I have a magical test case application that bursts to 100% of port speed for 1 minute out of every 5. On a 1 minute polling cycle those 100% bursts will be happening 20% of the time (overall) so my 95% utilization graph will show 100% of port speed. On a 5 minute polling cycle, however, my utilization will consistently be 20% of port speed. (100% usage for 1 of 5 minutes, averaged over the whole 5 minute period)

    I think the best solution to this problem is to match the polling interval that your upstream providers are billing you at. Either that, or take your data from more frequent polling and average it out to the polling interval of the upstream. I believe that produces the fairest billing data for all parties.
    Eric Spaeth
    Enterprise Network Engineer :: Hosting Hobbyist :: Master of Procrastination
    "The really cool thing about facts is they remain true regardless of who states them."

  11. #11
    Join Date
    Aug 2002
    Location
    UK
    Posts
    1,040
    Yep. What Eric has described is how I understand it to work. As you reduce the polling period (e.g. from 5 minutes to 1 minute) you will almost certainly increase the 95%tile value. Unless of course your usage is very constant, in which case there won't be any difference.
    Robin Balen
    Gyron Internet Ltd - http://gyron.net/
    UK colocation, managed hosting and connectivity services with 100% uptime SLAs

  12. #12
    Join Date
    Aug 2000
    Location
    Sheffield, South Yorks
    Posts
    3,627
    I concur with Robin and Eric. 95th is generally calculated as:

    delta_in / sample_period_in_seconds

    or

    delta_out / sample_period_in_seconds

    So a smaller sample period, will result in higher peaks if the traffic was sustained for <smaller sample period or less>
    Karl Austin :: KDAWS.com
    The Agency Hosting Specialist :: 0800 5429 764
    Partner with us and free-up more time for income generating tasks

  13. #13
    Join Date
    Jan 2004
    Location
    North Yorkshire, UK
    Posts
    4,164
    Interesting, never really thought about it in depth but that does make sense. Thanks.

    Dan
    █ Dan Kitchen | Technical Director | Razorblue
    █ ddi: (+44) (0)1748 900 680 | e: dkitchen@razorblue.com
    █ UK Intensive Managed Hosting, Clusters and Colocation.
    █ HP Servers, Cisco/Juniper Powered BGP Network (AS15692).

  14. #14
    Join Date
    Feb 2004
    Posts
    963
    Quote Originally Posted by RazorBlue - Dan
    It does annoy me when providers use the 5 minute model because small traffic spikes can look like they lasted a lot longer than they did and cost you more money.

    Dan
    This is why it's 95th percentile. You cut off those peaks. If you're peaking often enough that this is a problem, then you're probably being billed accurately.

  15. #15
    Join Date
    Feb 2006
    Posts
    189
    Quote Originally Posted by RazorBlue - Dan
    Interesting, never really thought about it in depth but that does make sense. Thanks.

    Dan
    Well maybe I will think twice about going with RazorBlue now.

  16. #16
    Join Date
    Jun 2006
    Location
    Chicago
    Posts
    462
    Well you don't necessarily cut off all the peaks, just those sustained for short periods of time that don't occur regularly. 95th is great, but even being around it for years, I still get mixed up as Dan, so I wouldn't go below the gut just yet I'd say that matching the polling rate with your upstream is the best bet for the most accurate readings. As far as not using RazorBlue, attroll you'd be missing alot of redundancy and 24/7 support/management. But thats just my .02

    Jordan

  17. #17
    Join Date
    Jan 2004
    Location
    North Yorkshire, UK
    Posts
    4,164
    Quote Originally Posted by attroll
    Well maybe I will think twice about going with RazorBlue now.
    I'm not responsible for our network, it's not my department . But I did find out although we poll every 30 seconds that's just for the purpose of making the graphs more useful, the actual 95%ile is measured over 5 minutes as standard which seems to be what everyone else is using.

    Dan
    █ Dan Kitchen | Technical Director | Razorblue
    █ ddi: (+44) (0)1748 900 680 | e: dkitchen@razorblue.com
    █ UK Intensive Managed Hosting, Clusters and Colocation.
    █ HP Servers, Cisco/Juniper Powered BGP Network (AS15692).

  18. #18
    Join Date
    Feb 2004
    Location
    Bay Area, CA
    Posts
    521
    Quote Originally Posted by nitrohosting
    On another note, anyone know off hand if it's easy to modify cacti to use the same interval (i.e. 5 mins) for all graphs (daily, weekly, monthly etc)?

    Kris
    Just gone through trying to do same. I ran into some issues all outlined in this cacti forum thread: http://forums.cacti.net/viewtopic.php?t=16254

  19. #19
    Join Date
    Jan 2007
    Location
    Chicago
    Posts
    11
    Right, a 5 minute polling is more cost effective on you, and 1 minute is more money for the provider, so defiitely don't go after 1 minute polling.

  20. #20
    Join Date
    May 2004
    Location
    Lansing, MI, USA
    Posts
    1,548
    Wouldn't it be 6 of one, 1/2 dozen of another? Remember, if you poll every minute as opposed to every 5 minutes, you have 5 times the data stored, so your top 5% is 5 times larger.

    If you do 1mbps average every 5 minutes, or 1 minute of 5mbps and 4 of next to nothing, you will actually be billed LESS on 1 minute polling as your 5mbps minutes would likely be struck in the now 5 times larger top 5%.

    Sure, 5 minute polling may give you some extra leway to hide extra usage, but 1 minute polling gives you extra reinforcement of non-usage. I think it cuts pretty much either way, though it'd be interesting to see both 1 and 5 minute polling calculations from the same data. I really would bet you'd find some where 1 minute was better, some where it didn't matter, and some where 5 minute was better.
    Jacob - WebOnce Technologies - 30 Day 100% Satisfaction Guarantee - Over 5 Years Going Strong!
    Website Hosting, PHP4&5, RoR, MySQL 5.0, Reseller Hosting, Development, and Designs
    Powered By JAM - Professional Website Development - PHP, MySQL, JavaScript, AJAX - Projects Small & Large

  21. #21
    Join Date
    Nov 2005
    Location
    Minneapolis, MN
    Posts
    1,648
    Quote Originally Posted by WO-Jacob
    Wouldn't it be 6 of one, 1/2 dozen of another?
    It might seem that way, but nope. Since network traffic is usually very "bursty" by nature, substantial differences will show up with shorter averaging intervals.
    Quote Originally Posted by WO-Jacob
    If you do 1mbps average every 5 minutes, or 1 minute of 5mbps and 4 of next to nothing, you will actually be billed LESS on 1 minute polling as your 5mbps minutes would likely be struck in the now 5 times larger top 5%.
    If I have a repeating pattern of 5mbps for 1 minute followed by 4 minutes of 0mbit, my 95% billed rate would be 5mbit. 1 out of 5 minutes = 20% of my samples will be 5mbit. When you throw out the top 5, the next lower 15% are still 5mbit. With 5 minute averaging those samples will be 100% 1mbit.
    Quote Originally Posted by WO-Jacob
    I really would bet you'd find some where 1 minute was better, some where it didn't matter, and some where 5 minute was better.
    I'll take you up on that bet! I'd be interested to see a 1 minute polling interval produce a lower 95% billing rate than a 5 minute polling interval, especially for a server that isn't powered off or otherwise dead.
    Eric Spaeth
    Enterprise Network Engineer :: Hosting Hobbyist :: Master of Procrastination
    "The really cool thing about facts is they remain true regardless of who states them."

  22. #22
    Join Date
    Jan 2004
    Location
    North Yorkshire, UK
    Posts
    4,164
    Quote Originally Posted by WO-Jacob
    I really would bet you'd find some where 1 minute was better, some where it didn't matter, and some where 5 minute was better.
    I've talked to our network engineer and we're giving it a try on three network ports with pretty different traffic patterns to see the difference in 95th percentile, and put the 1 minute theory to the test.

    I'll post once it's been running long enough to give some reasonable results.

    Dan
    █ Dan Kitchen | Technical Director | Razorblue
    █ ddi: (+44) (0)1748 900 680 | e: dkitchen@razorblue.com
    █ UK Intensive Managed Hosting, Clusters and Colocation.
    █ HP Servers, Cisco/Juniper Powered BGP Network (AS15692).

  23. #23
    Join Date
    Jun 2006
    Location
    Chicago
    Posts
    462
    Hehe, I'm interested to see this. Thanks for volunteering Dan

    Jordan

  24. #24
    Join Date
    Nov 2004
    Location
    Atlanta, GA
    Posts
    464
    Here are some pictures, we have both a 5 minute average and a 1 minute average here. The graph is just over a week right now, but if you want more data, I can get more data.

    Below is the text 1 minute 95th billing. The 5 minute is on the graph. It is 6.69 for the 1 minute and 7.45 on the five minute.

    2/3 colo0127 houdc01ed01.zogmo.com 3.57 0.14 12.53 0.52 6.69 0.24

    Well the upload didn't take you can view them here.

    http://www.zogmo.com/1minuteaverage.JPG
    http://www.zogmo.com/5minuteaverage.JPG

    Cheers,

    Linn
    Linn Boyd

  25. #25
    Join Date
    Feb 2006
    Posts
    189
    After looking at those graphs I would say the customer would benefit from 5 minute intervals.

Page 1 of 2 12 LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •