Here's why 1 and 4 don't work: A "cheap VPS" usually means unmanaged, and an unmanaged VPS is only as secure as you make it.
That being said, here's my opinion about your intended use and the minimum requirements I would look for: 512MB RAM and more than 2 GB disk space, 1000 MHz processor share is fine.
You can run Wordpress on smaller boxes, but you'll be happier with the performance of WP in a 512MB environment. I have a WP blog on a 512MB VPS from Burst.net, and it works fine. There are some configuration things to take care of that take days to accomplish (MySQL tweaking, etc.) So a lot of the "extra RAM" is taken up by MySql caches, which makes the site run pretty fast (its a small "auto-blogging" site that gets about 7500 page views / month.) Every time I post the URL here I get multiple intrusion attempts, so I'll let you know the URL via PM or email. (Really guys, lay off the hacking attempts).
You can get a VPS like this, without a control panel, Apache web server software, MySQL and PHP for the budget prices you see quoted (Burst.net is $5.95 a month). You then have to figure out how to install that software, and secure the server. And maintain it. To get the same thing in a fully managed version, expect to pay about 4 times that amount if you catch one on sale in the offers forum, say $25 to $30 a month. You can get a semi-managed "VPS for a single site" type of plan for about $10 a month, but you usually don't get shell access to do your VPN stuff with.