Results 1 to 9 of 9
  1. #1

    Mysql server hosting to support massive loads...

    Hey all. I am currently developing a project that could potentially need to support a mysql database which would be hit with millions of queries per minute.

    Realistically, the load will likely never get that big, and if it did, I would first need to have a server capabale of handling thousands.

    Right now I have a shared host which obviously wont cut it. I dont know that much about server management. I contacted a buddy of mine which once ran his own webhost out of a rack in florida. I am waiting to hear back from him.

    Every few seconds a user will be hitting the server with two queries and then updating a row. I can likely do some query caching to help aleviate, but the nature of the project means that most queries will be unique.

    Will a dedicated server handle a few hundred users doing this at once? What about a few thousand? Obviously beyond that, if millions of folks are using it, id likely have to hire someone to manage the servers and run it ourselves, correct?

    Or do companies exist which handle this kind of scaling from minimal use all the way up to millions of queries at once?

  2. #2
    Join Date
    Jun 2003
    London, UK
    For this level and type of database load you would want at the very least to use SLC SSD, if not using RAM disks (or RAMSAN) depending on the amount of data you need to store.
    Darren Lingham - UK Webhosting Ltd. - (0800) 024 2931 - Quality UK Windows and Linux hosting since 2003
    UK WordPress Hosting - Fast, easy, cloud based WordPress Hosting

  3. #3
    Join Date
    Oct 2006
    If you're just starting out with a query every couple of seconds, that's only 1 instance with a few operations, none concurrent. Most shared hosts would even be able to handle that. You could start out with a VPS somewhere, then just upgrade as your userbase/traffic grows. That doesn't take into account any spikes in traffic in a short period of time though...

  4. #4
    We have been having amazing success with pci-e based SSD systems that do very very very heavy IOPS. I would look into that.

  5. #5
    Join Date
    Mar 2002
    Philadelphia, PA
    Cache where possible.

    Others have mentioned some good suggestions, for something like this I would probably setup a cluster.
    Linux junkie |

  6. #6
    Join Date
    Aug 2008
    Seattle, WA
    For something of this scale, you need a managed services partner who can evaluate your queries on an ongoing basis and make adjustments. At the end of the day, that's going to be far more important than what hardware you have in place.

    Just for example, if the millions of queries are ever duplicates of eachother, inserting a caching layer in the middle could stave off half of your database hits. If they're mostly read queries, you could scale out a master-slaves replication cluster where you have many read-only nodes and one or a few write-only nodes.

    You need a host who understands databases in great depth. The hardware will come naturally, and is the easy part.
    Seattle, WA - Gigabit Carrier & Network Services Provider
    Uptime | Atlas Networks

  7. #7
    Caching will be tricky if not impossible with this particular set of queries. I do not forsee a possibility that two queries will ever be the same. The results of two may be the same, but not the queries themselves.

    Most data is read only once it is created. So replication with read-only would make a lot of sense.

  8. #8
    Have a look at quackit for resolving the queries.

  9. #9
    Join Date
    Oct 2003
    The Netherlands
    Are you sure about the "millions of queries per minute" requirement?

    If you're serious about this kind of scale and want to use MySQL you should probably have a chat with Percona.
    They also have their own tweaked version of MySQL and know their stuff.

Similar Threads

  1. Replies: 1
    Last Post: 01-17-2009, 02:57 PM
  2. MYSQL loads
    By vsev12 in forum Web Hosting
    Replies: 7
    Last Post: 01-26-2006, 03:17 PM
  3. ways to lower mysql and apache loads on high traffic server?
    By recko11 in forum Hosting Security and Technology
    Replies: 16
    Last Post: 05-14-2005, 11:15 AM
  4. MySQL & server loads
    By jubbs in forum Dedicated Server
    Replies: 17
    Last Post: 05-07-2004, 07:45 PM
  5. Replies: 17
    Last Post: 09-25-2003, 12:57 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts