I've just moved from a hosting company to a VPS service, and I am seeking general suggestions and opinions on a few things, rather than making it specific to my the companies I use.
I moved from a shared hosting service where load average is often 3 or of that order. The server response is marginally acceptable... pages of popular CMS show up with a few moments of delay. So I moved to a VPS with a decent plan, certainly more memory and disk than I needed. (From the experience when I run my own server before switching to the hosting company, when I was using a single PIII 500MHz for a web server.)
The server response was superb for the first few days, even when the load was quite high with background tasks. Then when search engine crawlers rushed in, the load average went from 0.1 to 1.0 constantly, with unacceptable server response time. Googling for 2 seconds, I found suspected cause of high CPU usage and disabled a function and also adjusted robots.txt to lighten the load. It lightened, and the load is usually 0.1-0.3 range now. The thing is that, the server response didn't come back to the same level as before. I've optimized mysql database and did usual things. Loading of static files is very fast, particularly because I compress and cache them in lighttpd, but anything PHP is very slow and I wonder where did the server juice drain to. Frankly, my old hosting service for a few bucks a month is running much faster at load average of 3 than this VPS at 0.2, comparing the same, new, dynamic contents without cache.
At the hosting service, I didn't use Fantastico or anything and installed packages from tar files by hand, so the scripts are identical. Hosting services may be doing a lot of optimization on the server, maybe with lots of buffering and caching, ... but this shouldn't really help the loading time of a new dynamic content.
What am I missing? Did google permanently suck my server juice or they just hid it? What tuneup or optimization I should try?