I look for a good datacenter with a good uptime SLA and good support. Also, Look for places that sell cPanel. Customers love cpanel, thats why I use it. I use www.sagonet.com for most of my servers because its everything I'm looking for. My one other server is www.theplanet.com, but their support is getting slower.
I also use sagonet and think it's WONDERFULL. They also offer the InterWorx control panel on their RedHat machines. It is similar to cPanel but lacks some of the bells and whistles: http://www.sagonet.comhttp://interworx.info
When looking to buy dedicated servers from a company, you should look for the things you need; however your needs must be reasonable at the same time.
Can you manage your own server or do you have someone to manage it for you? If so, then you won't need such speedy support as others, and you'll most likely be looking more into network and the SLA.
If you cannot manage your own server, or don't have someone to do it for you, then you'll certainly need a fully managed service. This comes at a cost, however if you go for the cheaper plans you can get fully managed but with a slightly not so perfect network.
Which is more important for you, and of course your clients? If support is more important for you, then you know your prices will be higher so your clients will have to pay more; unless you go for a cheaper solution with a not so perfect network. However then this means more downtime; what will you do when your clients complain about the downtime? Will you and them be ok about the downtime?
There are lots more things to think about, however the above are just a few. We all have our own favourite datacentres for one reason or another; however no datacentre is perfect for everyone. Each person requires something very different and unique, and that's why not everyone is using the exact same datacentre.