It works well enough to just require some action to be taken on both ends. Push a button on the opener (or an already-paired remote), then pair the remote while the opener is in the pairing state. It’s possible for a passerby to intercept, but they’d have to have very good timing.

Pressing a button on the opener is physical contact. That's the entire idea that the OP was trying to relay, that you need some physical way to prove that you're eligible to pair. Not that the key itself had to be hard-wired for the process to proceed.