If I have a reseller account on Enom and a customer order a domain through me, but I do not have enough credits in my Enom account, what happens?

Well, that would depend on your billing software... if it's set to automatically process domain orders (bad idea!) it will fail.

If you don't have enough credit in your eNom account, then the domain is will not be registered.

(The registration attempt will fail with an error that says something like, "Insufficient credits".