It depends on what the host you sign up with is offering. In some cases you will have to provide the support, but if you want your host to do this, you will have to look for a slightly specialized host. Many hosts call this "end-user support," (or "outsourced support") and is becoming more common now days.
Normally 99% of hosts who sell reseller plans don't offer support to your clients. Unfortunantly this is something you have to do at your own. We provide the reseller control panels and hosting and you support your clients. This is how business works.
As stated earlier you should be looking for providers who have End-User support included in their reseller plans. There are a few that come to mind, but I would suggest searching this forum and getting a list together. Also you may want to check up on the TOS and see the restrictions with end user support as it is a service in generally limited to tickets and email. There will most likely be number restrictions on amount of times they will support your customers per month.
There is also outsourced support companies aswell.
I have come across many such reputed hosting companies who mention that they will offer support to their reseller's end user clients too.
When you are a reseller, 90% of the server wide issues are handled by your hosting company. Hence 10% of the minor issues are not difficult to be handled by yourself. And if you are that busy, as mentioned earlier, if your host is not offering end user support, try searching for a good support provider.
What it may come down to is just getting comfortable with people and ask questions of any prospective reseller or vps host.
Even if you do not see an open offer of third party support, you might ask anyway. No reputable host will fault you for asking.
Meanwhile learn as much as you can before directing your clients. If you do get stuck, always tell the client you will find out what the answer or solution is as soon as possible. There are forums and directories all over the net that can and will help.
A good part of this business is be able to solve problems. Sometimes everyone needs another opinion or a little guidance. It's nothing to feel bad about.