File System Backup on SSH server on different port

I have remote VPS server and since I have limited space there, I want to schedule a File System Backup to my backup server with SSH connection. But my backup server is using different port for SSH. I tried putting backup.server:port in SSH host name, just like I did in Virtualmin Scheduled Backups, but its not supported. I’m currently using this command to create a backup:

tar -zcvp --one-file-system / | ssh -p 123 root@backup.server “( cat > /backup.tar.gz )”