Remix.run Logo
▲ Rebuilding my local voice assistant pipeline(blog.platypush.tech)
54 points by todsacerdoti 4 hours ago | 11 comments
▲mewse-hn an hour ago | parent | next [-]

Interesting write up, I've been vibe coding an android voice assistant using a similar pipeline without a wakeword - VAD > ASR > LLM > TTS

I've found parakeet to be much, much better than whisper and I made a 4 bit quant of parakeet 110M for my crappier android test devices, it would probably fit a raspberry pi:

https://huggingface.co/mewse/parakeet-tdt_ctc-110m-onnx-int4

▲GaggiX 21 minutes ago | parent [-]

There is also parakeet-redux, a 1.58bits quant of parakeet-tdt-0.6b-v3 that runs as well as the original model: https://huggingface.co/moondream/parakeet-redux

The model is actually a distillation of a different checkpoint called parakeet-ultra, same size and architecture as parakeet-tdt-0.6b-v3 but better, that's why the 1.58bit quant works so well.

▲mewse-hn 13 minutes ago | parent [-]

Oh crazy I hadn't seen this yet, I might be able to unify my 0.6B vs 110M selector, thanks!

▲mmmooo an hour ago | parent | prev | next [-]

older amazon echo dots and shows really make great hardware for this, combined with homeassistant, and can be had for very little cost. e.g. shameless plug: https://github.com/ygelfand/echolocal

▲nl 15 minutes ago | parent | next [-]

I just commented in another thread that I built this: https://github.com/nlothian/earwing

Beam forming microphone array, custom UI, custom case. Maybe costs more than a second hand echo dot though (hardware is probably around $30).

▲neumann 17 minutes ago | parent | prev [-]

I got an echo show rooted and ROMd that I am loving - put it in the kids room as a music player in kiosk mode, but have been now wanting one for myself with wakewords. This looks like the ticket.

▲fragilerock 3 hours ago | parent | prev | next [-]

Interesting read! I have my own local voice assistant and I implemented exactly the same question mark trick just the other day.

The overall pipeline is quite similar to my previous setup, but I found the STT -> LLM -> TTS pipeline just infuriatingly slow. TTS and STT with whisper/piper worked fine and reasonably fast for me locally, but the real bottleneck was always the fact that they're not running concurrently with the LLM call. I ended up forking an open source openai realtime API ESP32 plugin and patching it to my liking. It's a bit more expensive in terms of API cost but I only use it sparingly and it's so much nicer to use now.

▲dofm an hour ago | parent | prev | next [-]

Just what I want to read — favourited to come back to!

▲bityard an hour ago | parent | prev | next [-]

Is this complimentary to, or an alternative to Home Assistant?

▲malfist an hour ago | parent [-]

Home assistant doesn't really provide much out of the box, its mostly a modular/pluggable ecosystem and one of the things you can install are various voice assistants

▲stavros 3 hours ago | parent | prev [-]

This is really useful, thanks! The other day I bought a small screen and didn't know what to do with it so I made an assistant that talks to my bot:

https://m.youtube.com/shorts/3FseVyAG5uM

I used microwakeword too, and training my own wakeword was pretty simple (basically just recording a few tens of samples). Highly recommended.

I also wanted to be able to issue some voice commands for lights and other things in my home with low latency, so I added a short-circuit plugin that runs right after recognition. If the recognized command matches one of the regexes, it's immediately executed, which has a latency of under a second. Otherwise, it gets sent to my assistant, which usually will tweak the regex (or add a new one) to improve the whole system as it fails.