I should check my SSH logs.

My intuition is that since the SSH server reports what auth methods are available, once a bot sees that password auth is disabled, they will disconnect and not try again.

But I also know that bots can be dumb.