Thanks! What an excellent question, I’m not sure I have a good answer. I kind of became an open source maintainer by accident as Handy became popular

Certainly I am very lucky that quite a few people donate to Handy, and also some people and organizations who sponsor the work I do

To be honest I just love contributing to open source and wish to continue to do so. So anyone who supports this is good to me. Organizations which believe in OSS and push it forward are typically most aligned with me

Of course you can always email me (contact@handy.computer) and we can discuss in more detail

OS-native dictation on iOS requires uploading your address book to Apple on every request, even if you don’t use iCloud. I unfortunately have to leave it disabled for this reason.

Totally understandable, but I’ve found that software that transcribes everything after I finish recording actually works better for me. I’ve tried both kinds, and systems that continuously type what I’m saying distract me from completing my thought. I end up reading what’s being typed and noticing transcription mistakes instead of focusing on what I’m trying to say.

I often prefer to dictate everything in my head about a particular thing for 5–10 minutes and then go through it afterward. I find that much more useful because it doesn’t break my thought process the way continuous transcription does.

> The continuous part is crucial. Many software will paste whatever I said after I have stopped recording, but that is not useful.

It really depends on how one uses transcription.

For example, I really value being able to open different windows, and look at graphs, or scroll some data while I'm dictating, because it can help me with providing some support information for what I'm saying.

Some apps can even take into account things you copy or look at as part of the transcription's context to improve the results [0].

[0]: https://superwhisper.com/docs/common-issues/context#types-of...

You can fairly easily modify [Handy](https://handy.computer) to do this if you want

I’m planning on having it as a first class feature of the app too just too many other issues to work on first

You know English doesn't work like that. The word you're saying only becomes clear with the surrounding context. Eg, 'there' vs 'their'.

Apple Dictation does this, or something similar, in my experience. Some apps (e.g. terminals in my experience) buffer the entire transcript but in most apps it's identical to typing as you speak. Have you tried it?

This is what I attempted with https://github.com/electronstudio/low_latency_dictation

However the accuracy of the real time models is poor, so I did a second pass with a higher accuracy model before committing the text.

You used to be able to do this with dragon naturally speaking (don’t remember if that was it’s exact name) 10 ish years ago

Agreed. It's something I've found annoying about a few systems.

It looks like the rust bindings have streaming examples so hopefully there is a nice solution here.

TensorRT and CUDA is effectively the same speed as CPU for the speech to text models I was testing via ONNX at a huge binary bloat penalty. WGPU is hard to ship and also equivalent speed or slower. This may not be the case for LLM or other models but the runtimes did not seem well supported for what I needed to do. ONNX is incredibly well optimized for CPU, best in class even, but the other execution providers at least for STT seemed lacking.

I did this investigation before creating transcribe.cpp it would have been much more convenient and save me literal months of work. Happy to share the repo and binaries produced as well, but it was mostly throw away work to profile how to ship accelerated ONNX in Handy.

Yeah I’m working on it, Linux is a big pain point especially Wayland

Once things are more or less ironed out on MacOS and Windows a lot of attention will be turned towards Linux

I know a lot of Linux PRs are open it just takes me so long to get around and test them. And often multiple different implementations trying to fix similar issues which is a lot of overhead sometimes

Hey, yep author and maintainer here! Certainly sponsors help and the wonderful community who donates to Handy as well! Mozilla AI was very helpful in getting this work off the ground. It was a pipe dream for me to build for Handy and they helped to sponsor me so I could make time to take this project seriously and get a v0.1.0 release out the door

I agree this should be everywhere and I hope to distribute libtranscribe some day properly so it is more a system library! It will take time to stabilize but I think we can get there

Contributions are always welcome! There’s a WIP diarization PR rn, and after it’s merged would love to have support if it fits well into the interface. And if not would love to figure out a good interface for it

Yes, I’ve put a PR up on pypi for extra storage for CUDA but it has not been accepted yet afaik

If there’s any issues or improvements on the bindings I would love help to make the DX the best it can be

Hey! It’s actually in progress right now, probably will come this week :)

>This makes these projects so much more trustworthy and easier to approach:

>> Were any of the words here written using AI? Nope. They came from my mouth or my fingers.

I have to push back on this a bit, as I believe (quite strongly) that we're shaped by the tools we use; text-to-speech LLMs are still LLMs, and generally their mistakes are shaped by the expectations inherent in their training. This, in turn, shapes the words that appear on the screen. For those who regularly use them, you then learn which word sequences are likely to be accurately transcribed, and this definitively becomes part of your thinking process. Over time, the LLM becomes tangled into your thinking; the use of AI, even in this way, very much can and often does shape the resulting words.

It very much depends on the hardware! An M4 max is being compared against a Ryzen 4750U with an integrated GPU!

The M4 max has probably 10x the compute and memory bandwidth hahaha

Not in the library itself, it’s pure inference. Some models have this trained out of them anyhow. Otherwise this is a post processing task which is not really inference

Parakeet unified for me no longer does this and it’s also a streaming transcription model!

But the answer largely depends on you, the languages you speak, and personal preference. Whisper is still excellent and supported in transcribe.cpp

Cohere Transcribe is also excellent, but many of the new models are as well

> parakeet on a mac which transcribes all my uh's and aahs

You should be able to fix this by playing with the mic speech floor. It happens when to much ambient stuff slurps in.

It's actually gaslighting you, you don't say that many ums and ahs ;)

More or less yes, for whisper.cpp, just trying to make local transcription more accessible to anyone building an app, etc

Thanks :)

saw the demo its impressive but the audio was robotic

Hey! It’s actually in progress right now, probably will come this week :)

That would be Diarize.cpp, not Transcribe.cpp.

Out of the box no probably not, but if people are interested there’s probably ways forward

Have you published your app? I would love to take a look and test it

Ha amazing, love to hear it

Pretty sure I saw Handy using it; if you have the latest version, you’re probably already demoing it.

Any that also support translation? How much harder/ easier of a problem is local translation compared to transcription?