new | show | ask | jobs Github

ukuina 16 hours ago

What's the easiest way to add speaker separation to this?

▲

sipjca 15 hours ago | parent [-]

Hey! It’s actually in progress right now, probably will come this week :)

▲

simonw 15 hours ago | parent [-]

Awesome! I found the in-progress diarization PR here: https://github.com/handy-computer/transcribe.cpp/pull/85

Looks like it's using IBM's Granite-Speech-4.1-2B-Plus https://huggingface.co/ibm-granite/granite-speech-4.1-2b-plu... and/or MOSS-Transcribe-Diarize https://huggingface.co/OpenMOSS-Team/MOSS-Transcribe-Diarize

▲

sipjca 14 hours ago | parent [-]

Yep, but I am in the process of also porting NVIDIAs Sortformer for multi speaker diarization as well :)

I’m not sure how many specific models will be supported as the library is more focused on transcription specifically. But the models which support diarization natively must be supported I think. And parakeet multitalker was the primary driving force for this change

▲

oezi 14 hours ago | parent [-]

How close do you aim for when it comes to drop-in vs whisper.cpp? Are timestamps per word and character something aimed for? How about multi-lingual transcription or hallucination suppression?

The github page doesn't seem to go into depth on these orthogonal topics. May have missed it.

	▲	sipjca 8 hours ago \| parent [-]
		Eventually I would like to be more fully drop in compatible, right now some feature support is a bit sparse. And whisper has so much work done to it over the years so it’s hard to support every possible thing. Right now it’s a more bog standard implementation than anything special. Right now stabilizing the core header is probably among the primary goal, but if people want to contribute model specific things im happy to review test and pull in. Whisper is a good case for this as there is a header extension already so it’s easier