I just got access to my VPS account and the first thing I did was securing ssh. So I set it in such a way that only I (my ip address) can have access to ssh on a different port. I configured everything correctly and restarted ssh. I logged out and tried to reconnect. Now I can not get into ssh anymore, but I didn't do anything wrong for sure. What can I do now?
You don't really need "something like webmin". After reconfiguring, and restarting, sshd, do not log out. But do log in again to test that the reconfiguration works, while you're still already logged in.