I've just moved from a hosting company to a VPS service, and I am seeking general suggestions and opinions on a few things, rather than making it specific to my the companies I use.
I moved from a shared hosting service where load average is often 3 or of that order. The server response is marginally acceptable... pages of popular CMS show up with a few moments of delay. So I moved to a VPS with a decent plan, certainly more memory and disk than I needed. (From the experience when I run my own server before switching to the hosting company, when I was using a single PIII 500MHz for a web server.)
The server response was superb for the first few days, even when the load was quite high with background tasks. Then when search engine crawlers rushed in, the load average went from 0.1 to 1.0 constantly, with unacceptable server response time. Googling for 2 seconds, I found suspected cause of high CPU usage and disabled a function and also adjusted robots.txt to lighten the load. It lightened, and the load is usually 0.1-0.3 range now. The thing is that, the server response didn't come back to the same level as before. I've optimized mysql database and did usual things. Loading of static files is very fast, particularly because I compress and cache them in lighttpd, but anything PHP is very slow and I wonder where did the server juice drain to. Frankly, my old hosting service for a few bucks a month is running much faster at load average of 3 than this VPS at 0.2, comparing the same, new, dynamic contents without cache.
At the hosting service, I didn't use Fantastico or anything and installed packages from tar files by hand, so the scripts are identical. Hosting services may be doing a lot of optimization on the server, maybe with lots of buffering and caching, ... but this shouldn't really help the loading time of a new dynamic content.
What am I missing? Did google permanently suck my server juice or they just hid it? What tuneup or optimization I should try?
Are you sure you want to turn Googlebot away with the robots.txt? You will lose traffic if this is the case.
In a dedicated server, although the server specifications may not be great, however, there are more shared resources for your site to use. Whereas for VPS, although the host node specifications can be superior, but you are only getting a small and confined resources.
Check out your QoS alert if you have a Virtuozzo or HyperVM control panel, see whether your VPS exceeds any of the resources allocated.
Alternatively, if you want, you can do these commands and paste the output here for us to comment on:
1. cat /proc/user_beancounters (virtuozzo)
2. free -m
SecureAX Singapore - Virtualization, Private Cloud Computing & Managed Datacenter in Singapore
- Managed Virtual Private Servers, Dedicated Servers & Colocation Services in Singapore
- Gigabit backbone at Equinix Singapore, M1 & Telin Datacenters with Private Layer 2 for Disaster Recovery setup