From a quick look they start at around $99/mo (excluding specials etc...) You would have to find a VPS that allocated two full 2Ghz cores JUST to your VPS (and 2GB ram, etc...) and then you're still in a shared environment.
Keep in mind that Magento is pretty resource heavy, I agree you should look to optimizations first and see where that takes you.
I host 10 sites on the same ded. server and 100tb (hosting company) told me that the cause for the whole server to be slow is a problem in my video site's code.. I'm planing to fix this soon, but it can take 1-3 months.
I can't afford to have my ecommerce site (the magento one..) slow for that much time, that's the reason Im looking for a vps..
My current server specs:
Intel Quad Core Xeon 3220 - 4 x 2.40GHz
* 2 x 500GB HDD
* 8GB RAM
* 1 GigE (1000Mbit) dedicated port
* 100TB monthly transfer
* Unmetered Bandwidth (324TB) + $399/mo
Magento runs fine on a VPS, disk speed is important though. We have many clients who migrated from a dedicated/shared hosting to our cloud. Because of the disk speeds (SAN) they all have a much quicker site now.
Good luck with your search! Go with a provider which is known to have good disks speeds (iops/seeks, not throughput).
YISP - High Bandwidth dedicated servers and colocation in YISP-AS(Amsterdam)!