Are you looking for a fully managed VPS (ie you just use it, but system admin tasks like php upgrades, troubleshooting, OS patching etc. are performed by your provider), or an unmanaged VPS where you do all of these things yourself with root SSH?
There are different recommendations depending on which type of service you're looking for... Most of the ones listed above are of the unmanaged type, or otherwise claim to be "managed" yet also give you root SSH (why would you need it!?), suggesting they merely mean they'll reply to a few support tickets for you rather than proactively manage the server.
I'm using Clustered.net for the production side of things and VPS.net for prototyping (and in the future US hosting)
Clustered.net does very well for me, high performance nodes, solid network, great uptime. I do sometimes feel their staff is a bit overworked though (lots of things require tickets) but they are definitely fast (never have to wait long for any request to be picked up)
VPS.net is very shiny and lets you create VPS nodes quickly, but boy do they still have work to do
Regarding managed vs unmanaged. I have noticed that unmanaged packages are significantly cheaper than managed but I'm not competent enough with SSH yet to be able to manage a server myself.
If I decided to go down the unmanaged route, once the VPS is set up by the provider, what would I need to do?
depends on the flavour of vps. Most should have a preconfigured firewall allowing only ssh and maybe http/https.
When we use ssh we deny root logins via ssh. Root is an obvious username for people to guess. Create a dummy user like johns. You can then log in as this user then SU super user to log in ad root
you may even want to change the port ssh listens on. (careful you don't lock yourself out) some providers may offer console access so you can login even if you have messed up the firewall/ssh config.
Another trick I have used in the passed is install webmin. Instead of allowing webmin ports through the firewall you can tunnel the connections to webmin and other apps through ssh (putty allows this) this keeps the number of ports open on your firewall to a minium, certainly for admin use anyway
It's also more than "just" security (patching/firewalls etc.) - you also have to handle any software installation/upgrades/troubleshooting on your own.
Unmanaged is strictly hardware + network, though some providers may provide a limited amount of assistance to you in other areas before they want to charge additional fees.
If you don't know what you're doing in terms of server administration, my advice (as a provider offering managed, Layershift, and unmanaged, WizzVPS - hence consider myself somewhat impartial) would be to strongly consider a managed service; you'll likely save yourself a lot of headaches!
Generally VPS users want to actually use the service, rather than spend all of their time administering it...
If you have not handled a VPS before then make sure you opt for a fully managed VPS. You can improve your knowledge by understanding the various aspects of the VPS and if there's any problem you can get in touch with the VPS provider.