Interesting, I'm surprised you went with Whisper, I found Parakeet (v2) to be a lot more accurate and faster, but maybe it's just my accent.

I implemented fully local hands free coding with Parakeet and Kokoro: https://github.com/getpaseo/paseo