I am wondering anyone have any idea how to make a frontpage web templates like those which provided at pixelmill.net? I search all over the web for the tutorial but I can't seems to find one.

If you need template to be editable with frontpage - check Template Monster (http://www.templatemonster.com/)

All their html templates are compatible with Frontpage as well.