Proper diarization still remains a white whale for me, unfortunately.

Last I looked into it, the main options required API access to external services, which put me off. I think it was pyannotate.audio[1].

[1]: https://github.com/pyannote/pyannote-audio

I used diarization in https://github.com/jhj0517/Whisper-WebUI last night and once it downloads the model from HuggingFace it runs offline (it claims).