I am building a new server to replace my old web server.
The old one has some SSL certs used on some domains.
How can I transfer those to the new server?

Different OS/Control Panel has different method but basically you need to move the public, private keys over with the certificate.

If you're using WHM to manage these domains, and the domain name will not change, the cert should transfer over when you do the full account transfer.