Sounds like you need to split your web and db servers, so that one does not do all the work. If you are stuck with what you have, you need to start doing some tweaking to your apache and mysql configuration files to optimize for what you have. Faster drives would help as well. I wrote a little article a while back about tuning mysql: http://solidservers.ca/2009/07/tuning-tweaking-mysql/
I've worked quite a bit with vBulletin over the course of the past few years and this is a question that comes up quite often as communities grow, expand and the traffic begins to roll in.
1). Upgrade The Memory/RAM to 8GB
A system with 4GB of RAM is great, if it's a desktop, but when it comes to a busy on-line community that is processing multiple requests every minute (every second even), the additional boost in RAM will come in handy. It will especially come in handy for #2.
2). Install an OP-Code Cacher
MemCache is supported by vBulletin as is xCache. They need to be installed as a root user or using SU. My experience is primarily with xCache so that is the one that I personally recommend that you use.
xCache will cache the most frequently accessed PHP files to RAM which drops the number of requests to the MySQL database and thus reduces the overall load placed on the server from MySQL.
Setting the value to 32-64MB is suggested, however, if you receive a high amount of OOM (Out of Memory) errors in the xCache control panel, then you may want to increase that to 128MB depending on your setup. That said, xCache has a built-in flushing system (garbage collection) which will automatically flush out files that are expired and allow them to be re-cached.
3). Optimize Your MySQL Configuration (my.cnf)
If you are using a standard MySQL installation, chances are MySQL has not been optimized and you're running on a OOB install. Optimizing my.cnf to make use of query-caching and a number of other built-in features will drastically reduce the overall load placed on the server by reducing the number of requests needed.
There are a few suggestions floating around the web. If you need a pretty good my.cnf configuration, shoot me a PM with your current my.cnf file and I'll be more than happy to make some modifications for you (for free, of course!).
4). Optimize PHP (php.ini)
As with any PHP install, if you don't need something, disable it or don't configure PHP with it. There are a number of functions that can be disabled in PHP and combining that with the use of xCache will greatly boost PHP performance.
5). Optimize Apache (apache.conf)
This is really a tricky area as it needs to really (and honestly) be constantly tweaked according to the traffic you're receiving. You'll need to test the waters a bit and see what works best for you and then continue tweaking until you only need to modify the values every now and then.
The other route is to split into 2 servers, one for PHP/Apache/E-Mail and the other for MySQL. You can still apply the above to a dual-server setup. This is a more costly route (of course, depending on what you pay for the above), but if you continue to grow and receive that sort of traffic, it is where you're heading.
A single server will only carry you so far unless you plan on dropping some serious cash on a Quad-CPU (not quad-core) setup with 12-32GB of RAM and even then, HD I/O is going to be your one disadvantage as it won't be able to keep up with the CPU.
But as others suggested, I also strongly recommended taking a second look at your MySQL configuration (my.cnf). Tell your host to change the MySQL config memory model to make full good use of at least 2-3 gig of your ram.
Remember to ask your managed hosting personnel to tell you what *caused* the slowdown. Most of the time, it would be IO from our database.
ok thanks, and what should my php memory limit be ?
That is the least of your concern. Its just a limit to make sure that a single script does use an absurd amount of memory, the default is something like 32 or 64M, and I think 32M is plenty.
Nginx is a great choice, and yes, absolutely fast-cgi, because cgi has horrible performance.
It is really important to tune your configuration values correctly. I use Lighttpd on sites that receive ~100requests/second of dynamic (PHP) only content and I've found you can dramatically improve or degrade performance through "bad" configuration.
memcache did the job for my VB community. Having a separate drive for your log files, and a separate drive for your /var/lib/mysql (in my case a SSD drive) would boost your performance as well. On the other hand, as everybody else pointed out, 4GB of memory seems to be low for that kind of traffic.