If you want ftp.domain.com to have its own cpanel login and so forth, then just add an account for it as you would any other domain, and specifically assign it the IP you want it to have.
If, on the other hand, you just want ftp.domain.com to resolve to a given IP, then just edit the DNS zone record for domain.com in WHM, and add a new A record pointing ftp.domain.com to the IP you want.
(Edit: Be sure to enter the domain as either ftp or ftp.domain.com. with the trailing dot when adding an A record in the zone)
I fail to see how settin up ftp server is related to dns!
If you want to make ftp.domain.com point to some IP then create A dns entry in dns zone... WHM can do it.
If you want to setup ftp then? What is the problem really?
About cpanel annonymous ftp -> its really not worth time because of lack of permissions, ratio etc. Anyway every cpanel account can create annonymous ftps wits some specific login like [email protected]