Restricting sshd server to specific addresses fails

[Joe]

This covers it:

Limit SSH access to specific clients by IP address

So, you can do something like this (at the end of the sshd_config, as match sections have to be last):

Match Address 192.168.1.0/24
    PasswordAuthentication yes
    PubkeyAuthentication yes

And, in the default section disable both of those options.