can you turn this into a decision model

Is it even worth using apple's? Maybe they have improved it but when they first started using it to transcribe voice memos it was laughably bad (in my experience). I defaulted to whisper.cpp

With MacWhisper you can select the Apple models, it took almost 4x as long to transcribe the same file versus this software and MacWhisper/Parakeet3.

Yes. Planning to train a multilingual reader. Have not done a prototype yet.Will do one once I have the right dataset

I only see that EmbeddingGemma 2 is Apache 2.

"EmbeddingGemma 2 is made by Google DeepMind and released under Apache 2.0. This is an independent project, not affiliated with or endorsed by Google."

The reference to Apache 2.0 on the page refers to Google's base model, not this app. Based on the page alone, the app is free, but the source code is not open-source.

https://huggingface.co/google/embeddinggemma-2

It's not being played through a desktop audio stack, they're operating on the waveform decoded from the audio file. There's no constraints relative to the "natural speed" of the source, just like for video transcoding, it's limited by compute.

during transcription, nothing is actually "playing" the file in the way a human would to listen to it. the audio is stored as a sequence of bytes representing the amplitude of the physical sound waves over time (the recording device measures, or "samples", the amount that its microphone is vibrating and records that amount, some tens of thousands of times per second) with some metadata and compression applied so that other software knows how to interpret the bytes correctly. this can be given to an audio program to convert the bytes back to physical sound waves using your speakers, but it's just bytes at the end of the day, so any software that can read bytes out of the file can do so at whatever speed it wants, regardless of how long the actual recording is.

so you have a file containing bytes representing the sound wave, and you have existing recordings (read: sound waves represented as sequences of bytes) of people speaking in different languages and contests, and you can feed all those existing recordings into a model along with corresponding transcriptions, and you can get a model out which can identify patterns in recordings of people speaking to then go backwards and see a pattern of sound waves and correctly associate it with a specific token, no need to play anything back. pauses, stutters, etc. are themselves patterns of sound waves that can be identified and worked around to identify just the meaningful parts of speech.

in principle, you could probably learn to do this too, by sitting down for hundreds of hours meticulously starting at waveforms, and this is, very roughly, what signal analysts do: look at waveforms and try to identify patterns. ai just leverages this principle to turn mundane speech into text.

No more than an AI media generator creating the media in realtime.

Probably significant parallelism. Split the audio into a lot of slightly overlapping portions to ensure the transitions between words are caught. I’m sure there’s some relatively simple ways to ensure you don’t double up on words.

Multimodality seems to just help with everything

Correctly transcribing requires context too, no?

Only English for now. I dont see any roadblocks for at least the major European Languages(Not sure about German though with their compound words)

They say its word error rate is about 5%, so not great. Interesting as a proof of concept. The model this is based on is REALLY tiny.

WER is not good enough for professional use, fine for hobby projects

That's close to what this is: the reader is a CTC head on the audio encoder, but its output words are EmbeddingGemma's own text embeddings for ~229k words, instead of a separately learned vocabulary. I also tried a plain letter-level CTC head on the encoder. It did a bit better on clean audiobooks (7.0 vs 8.2 WER at the time) but much worse on harder audio like a lecture (16.5 vs 10.8), mostly from misspellings.

First thing I thought of lol

I've been using Whisper with great results; GPU acceleration gets the job done, nothing leaves my box, and the medium model is overkill.

https://openai.com/index/whisper/

https://github.com/openai/whisper

TrueScribe supports tranlating Hindi transcripts in both directions. Not free though.