How do I do it...? (I apparently installed it incorrectly, and I'm now having problems with it and Apache.)

Was it RPM file ? or you compiled it ?

It was an RPM file.

What kind of problems are you having? Can you list some error messages?

Hello ,
Do that
rpm -qa | grep j2sdk | xargs rpm -e
it's will search for java and uninstall it automatic
if you still wanna to show result of java make :
rpm -qa | grep j2sdk
or
rpm -qa | grep java
then do rpm -e of them
Regards

What if it was a BIN file?

Just delete it, and remove the information from your PATH. If it was installed via RPM, then rpm -e should get rid of it (and all files that it installed).