Could set up 6 digit long extensions and only ever issue a few hundred of them in total.

Guess wrong 3x and goodbye.

Can also set some/most/all to go to voicemail so they can get in touch with you, but not really.

Or blackhole the invalid extensions to /dev/null voicemail but then you run the risk of legit misdials and you never get some important message.

The real vs “fake” number issue could be worked around by having your cell phone provider forward all calls to your VoIP number. It’s baked into gsm, don’t need a phone after initial setup: https://www.geckobeach.com/cellular/secrets/gsmcodes.php