Yes, it is. You can also restart it while you're connected as your session will keep running the old binary in memory. This will be cleared when you logout of that session. Any new sessions (after the restart) willuse the new binary.
However I would strongly suggest that you turn TELNET on for the duration of the upgrade until you've successfully logged in to the restarted ssh daemon. It's all too easy for there to be a problem and you'll be locked out if you don't do this. Disable once you're confident everything is OK.