Results 1 to 25 of 33
-
05-08-2009, 12:45 PM #1Newbie
- Join Date
- Oct 2006
- Posts
- 8
Q8300, 8GB ram and it's not enough
I'm currently running a very large (vBulletin) forum on a single server. Up to now, I've been upgrading as the forum grows, tweaking what can be tweaked and at the moment, for about two hours every day (during peak time) the CPU maxes out and the webserver starts dropping connections. This isn't a massive problem at the moment, you just have to refresh more at these times. However as the traffic grows, this will only get worse.
I'm looking at my future upgrade options (the site shows no signs of slowing growth).
As far as I can tell, I have two options:
1. Upgrade to a more powerful single server
- a. Core i7
- b. Dual (two CPUs on one board) Core2 Quad Xeon
- c. Dual (two CPUs on one board) AMD Opteron
2. Two server solution, one server for MySQL, other server for memcache, xcache and webservice.
The former option is, obviously, much less expensive. Though i7 is incredibly expensive, it will offer a significant improvement. However, I'll most likely be facing the same problem in 6 months to a year...
Also, I rent. I am wondering whether to continue renting, or just front the money and colocate.
Thoughts?Last edited by Paraguay; 05-08-2009 at 12:51 PM.
-
05-08-2009, 12:51 PM #2Web Hosting Guru
- Join Date
- Sep 2007
- Location
- Toronto, Canada
- Posts
- 260
If your needs keep growing and maxing out the hardware you have while you are doing everything you can to tweak then moving to a two server setup or one "monster server" may be your best bet.
A single CPU quad core for each machine will most likely suffice but if your budget can allow for it moving to Dual CPU Quad core now may save you from future growing pains. Equip the DB with fast SATA or SAS drives with a RAID setup and plenty of memory and your load will be better off spread between your two machines.
With a low number of servers it is often not worth colocating but you can go ahead and work the numbers with a host and see what the price differential is between dedicated and colocated.
-
05-08-2009, 12:53 PM #3Junior Guru Wannabe
- Join Date
- Mar 2009
- Location
- Louisville, KY
- Posts
- 43
Separating SQL from Web is the best practice as SQL takes a lot of resources
>> Melisa Z. - mzilic@maximumasp.com
>> MaximumASP.com >Smart Hosting, Smart Choice
>> Unparalleled service, quality and value
>> How can we help YOU?
-
05-08-2009, 12:55 PM #4Doh!!
- Join Date
- Jan 2001
- Location
- NJ
- Posts
- 2,343
I would suggest def to go a split sever setup. web and mysql make sure the mysql is a SAS Drive setup. Also make sure you have proper backups of your database as for a forum the data is changing in virtually real time so real time backups are pretty much required.
From this solution as your forum grows you can scale to multiple web and mysql servers with load balancers and such.
So something like a Q9400 Web Frontend and a Dual 5420 or 5520 backend database server would work beautifully and give you the growth and expandability you need without jumping servers every few months and maybe even providers.
Also make sure your server is secure by having a proper firewall and web application firewall otherwise your site is open to hacking.Jay
-
05-08-2009, 12:59 PM #5Junior Guru
- Join Date
- May 2007
- Location
- Canada
- Posts
- 183
I agree with this. If you were to separate your resources, you will fair much better, and, not need a super high end machine for both the web and sql server. Can you post your stats for your forum, so that we can see what kind of usage you have right now?
Another note, what kind of tweaking have you done so far? Perhaps an upgrade is not needed as of yet.Quality Hosting - http://www.robohostingsolutions.com
My Blog - http://solidservers.ca
Freelance Server Administration (decent rates!) - Chris(at)SolidServers.ca
-
05-08-2009, 01:05 PM #6WHT Addict
- Join Date
- Jan 2005
- Location
- Croatia
- Posts
- 142
I think a 2 server solution would be best. Colocating is an option, but if you need to upgrade these twice a year then I'm not sure replacing all your hardware twice a year is going to save you much. Unless of course you start going the load balancing route and just keep adding additional servers to accommodate the requirements.
You might also want to have a look at LiteSpeed. I saw some pretty amazing performance increases with sites that switched to LiteSpeed from Apache.I am a leaf on the wind. Watch how I soar.
-
05-08-2009, 01:34 PM #7Junior Guru Wannabe
- Join Date
- Apr 2009
- Posts
- 36
If you are running a single cpu / quad core then the obvious choice to me is to step up to the the dual cpu/quad core setup or the i7. I currently manage a new i7 setup from softlayer and it is a pretty nice setup.
It really depends on how long term your looking for. I personally think that you should try an i7 system and see how it does for you, its a pretty significant upgrade. However, long term you will probably want to go with two server setup.
-
05-08-2009, 01:45 PM #8Newbie
- Join Date
- Oct 2006
- Posts
- 8
Thanks for the advice guys. Honestly, there is still room for tweaking on the server I have to improve performance (switching to a higher performance webserver software, changingg how attachment thumbnails are stored, layout modifications and further caching - currently only using xcache - of php)
The vast majority of my load is MySQL, and the MySQL itself is working as well as it can under the CPU limitations.
I'll look into splitting the load across two servers when I next upgrade, with a dedicated SQL server. For now i7 seems really expensive.
-
05-08-2009, 01:48 PM #9Web Hosting Master
- Join Date
- Nov 2007
- Location
- Dallas, TX
- Posts
- 9,064
I concur with most of the other posters -- a split server setup (small cluster) would likely be ideal, as MySQL can be separated that way.
You may also want to check into configuration-based performance optimization.
-mike
-
05-08-2009, 01:52 PM #10Corporate Member
- Join Date
- Feb 2008
- Location
- Houston, Texas, USA
- Posts
- 3,262
You have two options. You can approach a provider and ask for a discount on a server bundle (2-3 server with one of them being a monster server, one DB server + switch or internal VLAN) or rent an octal rack (cabinet/8) and purchase your own bandwidth/equipment.
What is your bandwidth usage?
RegardsUNIXy - Fully Managed Servers and Clusters - Established in 2006
Server Management - Unlimited Servers. Unlimited Requests. One Plan!
cPanel Varnish Plugin -- Seamless SSL Caching (Let's Encrypt, AutoSSL, etc)
Slow Site or Server? Unable to handle traffic? Same day performance fix: joe@unixy
-
05-08-2009, 02:02 PM #11Junior Guru Wannabe
- Join Date
- Apr 2009
- Posts
- 36
This is usually the case in forum setups. There isn't a ton you can do to make mysql use less cpu. The i7 is pretty expensive but they are also pretty new. Softlayer was running specials on them so I would assume other places might as well. There are plenty of things within forum software that can be tweaked or removed to be less tasking on the database. As forums grown the "nice little" features really add up.
-
05-08-2009, 03:37 PM #12Web Hosting Master
- Join Date
- Nov 2007
- Location
- Dallas, TX
- Posts
- 9,064
-
05-08-2009, 04:21 PM #13Web Hosting Guru
- Join Date
- Oct 2004
- Location
- Oakville, ON
- Posts
- 263
I really think at this time if your at the stage you say you are. It would be time to seperate the servers. This will allow you to have more growth and less load on the server.
Regards,
Josh Dargie
Kualo.com
Developer & Designer-Friendly Web Hosting
-
05-08-2009, 04:29 PM #14Junior Guru Wannabe
- Join Date
- Apr 2009
- Posts
- 36
I guess it also depends on how much the cost versus the benefit would be and how much money you lose during downtime. If you are able to migrate to a new server that will last you 6 months to a year then I would take that route and then when you decide to split the servers up you will be getting better/faster servers at usually lower rates. If you split the servers now and go with enough long term capacity that you are good for multiple years and you don't actually need it yet then you are just tossing the money away. I'm not sure this came out exactly how I wanted but hopefully the point is made.
-
05-08-2009, 04:32 PM #15Junior Guru
- Join Date
- Feb 2005
- Location
- Scotland, UK
- Posts
- 185
SQL is all about RAM (for cache) and fast disk access.
For the future you want to:
- Fine tune your mysql config
- Fine tune your databases
- A lot of RAM on your SQL machine for cache
- Fast disks (someone mentioned SAS above I think) with a decent RAID controller (High cache, battery for the write cache)
Also, not 100% sure because I haven't checked it in great detail but I have heard reports of the i7 platform being a step backwards so double check this beforehand.
Hope this helps.Sean McRobbie - Specialising in virtualisation since 2005.
‡ www.openitc.co.uk - We create, we host, we connect - Fully Managed VPS & Dedicated Hosting
-
05-08-2009, 04:33 PM #16Web Hosting Master
- Join Date
- Oct 2008
- Posts
- 2,253
I would check out 10tb.com you may be able to get better specs then the price your paying. Like others said disable extra features and or scripts
Leader of the new anti sig spamming club.
-
05-08-2009, 04:34 PM #17Junior Guru Wannabe
- Join Date
- Apr 2009
- Posts
- 36
-
05-08-2009, 04:35 PM #18Web Hosting Master
- Join Date
- Mar 2009
- Posts
- 569
If you want to Try tweaking first,
then try replacing Apache with Lighttpd or nginx.
Then Ultimate goal is to have a separate Database server and HTTP server
-
05-08-2009, 04:49 PM #19Web Hosting Master
- Join Date
- Oct 2008
- Posts
- 2,253
if your willing to pay extra litespeed is good. You can pay monthly and see if it helps the loads for now. if not you can go back to apache or lighttpd
Leader of the new anti sig spamming club.
-
05-08-2009, 06:24 PM #20THE Web Hosting Master
- Join Date
- Jan 2003
- Location
- Chicago, IL
- Posts
- 6,957
Litespeed could help with this, though at this point, I haven't seen anything to confirm whether this is a HTTP/PHP issue or a MySQL issue.
How many resources are used by each of those? How much memory are you using for each of those? You're sure this isn't disk IO related, what are your disk IO numbers like?Karl Zimmerman - Founder & CEO of Steadfast
VMware Virtual Data Center Platform
karl @ steadfast.net - Sales/Support: 312-602-2689
Cloud Hosting, Managed Dedicated Servers, Chicago Colocation, and New Jersey Colocation
-
05-08-2009, 06:36 PM #21******* Unleaded
- Join Date
- Feb 2004
- Posts
- 3,849
Before you decide, take a look at which of three areas is the bottleneck. These are: memory, disk, and cpu.
When you have figured out which area is the culprit, then post back and ask for advice on that particular area.
The likely culprit is disk io, and in particular IOPS.
If it *is* IOPS, then colocation is probably a better approach for you because you will need to build a disk subsystem that can take the beating. Disk subsystems tend not to be terribly sophisticated in rental offerings. The exception is where the provider does custom built to purpose servers. That usually requires a year or more contract.edgedirector.com
managed dns global failover and load balance (gslb)
exactstate.com
uptime report for webhostingtalk.com
-
05-08-2009, 10:57 PM #22Web Hosting Master
- Join Date
- Nov 2003
- Location
- Toronto, Ontario
- Posts
- 651
As has already been mentioned, I would say the a sure solution for your situation is clustering... regardless whether you rent capacity off a shared cluster like mosso, or build your own.
Obviously renting capacity on a cluster is likely going to be less expensive and allow you to burst much higher during peak periods without paying to keep that capacity live 24x7x365.Kevin
-
05-08-2009, 11:03 PM #23Junior Guru Wannabe
- Join Date
- Nov 2004
- Location
- USA
- Posts
- 78
I maintain a large linux forum with over 6 million posts, 829k users and 1 million threads. Here is some advice I can give you after dealing with a massively growing forum the last 4 years.
1. Get a second server and setup a master/slave replication system with mysql. Within vbulletin enable the master/slave configuration in the config file. This will help you by leaps and bounds, but take point #2 in to serious consideration. I've found this to be more important then offloading http traffic.
2. Tweak, tweak and tweak your mysql configuration files. If you want I can share mine, send me a PM if you wish. Grab this utility and run it often and learn what the output means, it will help.
http://mysqltuner.com/
3. Eliminate any unnecessary plugins/products. If you are using the "Thank You" plugin, I've found this to be problematic for large forums and suspect it can cause data corruption.
4. Look into using sphinxsearch, check vbulletin.org for information on it. It's pretty crude but if #1 isn't a option and you want to make a change to a couple of vbulletin files this is worth looking into. However, it does have its problems and limitations.
That should be a good start, but off loading some of that mysql traffic will really help. Feel free to send me a PM if you have questions.. Good luck!
- Ryan
-
05-08-2009, 11:14 PM #24Newbie
- Join Date
- May 2009
- Posts
- 16
clustering is more reliable from one server but this is depending on your budget
any way you may get a 2X high servers specifications , buy it , colo it ... that will limits the spending on the time and will release you from future upgrade ,one forum on two servers will work for the next 10 years
if you need more limits on price! get a used servers high specifications ,, it's all over the web,,
and I have Some ..
-
05-09-2009, 01:07 AM #25Web Hosting Master
- Join Date
- Sep 2008
- Location
- Dallas, TX
- Posts
- 4,568
I love the Xeon's(They are the same thing as Core2Quads' pretty much). Personally, I would probably get a mid/high end Dual Intel Xeon, like a L5410 and buy a bunch of SATA drives and RAID 10 them so you have the redundancy and speed, or if you can afford it buy SSD/SaS drives. But a good configuration will be a few hundred per month, like $400.