Have you watched the health meters on the server. Chances are your RAM is being taken and not released. A VPS isn't what it is cracked up to be and the first person who ever called it a mini dedicated server, well, I will leave the next sentence out.
VPS's work in very specific situations and in others they totally fail to do their job. You failed to mention how much of the CPU you are actually getting, it is 2 cores but what slice are you receiving.
Most of the chances are that once your software starts to fill up the database, your server load goes high due to several pending requests waiting to be executed, causing it to elevate the server response time. please check the server load when you experience the problem.
If I am correct, then you ought to switch to a server wih atleast 4 cores and 4 gb ram.
██ l Host Monsterz Pvt Ltd Superb Support Network DES 64 bit Secured Network
██ l Linux Shared/Reseller Hosting Virtual Server Dedicated Server cPanelฎ
██ l Nginx Softaculous Fantastico RVSite Builder WHMCS 24x7 Available Server Admin
██ l Premium Managed VPS Reliable & Sustainable Instant Setup Raid Drives
Exactly! For the usage you're stating, your server should cut it in heartbeat. If anything, your specs are over qualified for such a low usage app, even if your queries are not optimized. Consider a linux vps, don't install a hungry panel like plesk & you should be ok.