We have access to all of our servers, so this isn't a problem for us. However, what most people do is make sure that the colo facility is able to have someone either at the datacenter, or on call in case there is an emergency at all times. Some places provide free remote hands service, which means they will replace hardware (that you provide) free of charge.
Before you decide on a colo facility, make sure you ask them about this (and maybe even talk to a few current customers). As long as you plan well in advance, you should be able to handle any hardware maintenance / failure with little downtime for your clients.