Besides optimizing software I would recommend adding more RAM, because you have a nice fast CPU, but low RAM, this can be and probably is a bottleneck in your system. You will need additional RAM for caching PHP scripts, mysql queries, running more apache processes. You hardly hit swap now, but to ensure overall stability add more RAM.
Your server seems to be runnig out of memory. Adding another 512 MB of RAM will help, though I will put as any RAM as the motherboard allows. 2GB would be better.
Fluid Hosting, LLC - HSphere Shared and Reseller hosting - Now with HIGH AVAILABILITY Fluid VPS - Linux and Windows Virtuozzo VPS - Enterprise VPS with up to 2 GB guaranteed memory! Get your N+1 High Availability Enterprise Cloud Equinix Secaucus NY2 (NYC Metro)
That's the next thing i will do, Add 512 MB of Ram or 1 GB. Right now I am saving for it, i just bought 250 GB hard drive that will be added to my server soon. Things get to be expensive when you buy them with one-time fee. I will install TruckMMCache soon, any recommendation for apache?