Aww, I literally just implemented this using whisper.cpp and ffmpeg lib, code is even similar...