I have done it right over with yum twice.. I have noticed that you can experience problems with ssh and a couple other packages. I have had one instance that ssh would be active but would deny access to root because some packages didnt update correctly namely pam. Also if you are not running the 64bit kernel and try to run the 64bit binarys you may have problems.
just curious, what difference between CentOS 32bit and 64bit ? and do we really need 64bit ?
One of the biggest leaps in computing history was when the 32 bit processor was introduced. Same with the 16 bit processor. 64 bit will be the norm one day as well, so the simple answer to your question might be, yes, we need it.