I want to able to give out mysql usernames. So that my customers can enter their username and password for a mysql/php program. And without a hosting control panel.

Not sure if you realized this but you just made a statement about what you want to do...

I'm going to assume that you meant to add in the end:
"How would I do this?"
or something...

In that case:
http://dev.mysql.com/doc/mysql/en/User_Account_Management.html

MySQL user administration is fairly simple if you're comfortable with command-line/shell aspect of server control.

I'm using MySQL for WIndows (if that matters). And where do I go to find the "shell?"

drive:\mysql directory\bin\mysql.exe

ie. c:\mysql\bin\mysql.exe

You can run this via the command prompt:

cd c:\mysql\bin\
mysql

For windows you can use the MySQL CC

http://dev.mysql.com/downloads/mysqlcc.html

Hope this helps.