Yes however the server for all that did not cost a hundred bucks. A lot more than that :P Of course it's not the only thing I use it for either, and I view it as an investment in learning too. And it wasn't terribly expensive either. Two GPUs of 500 euro in total and 100 worth of components, the rest was all surplus stuff I had from upgrading my gaming PC. Then some dockers for things like whisper (and home assistant's "wyoming" wrapper around it) and ollama etc.
The only issue right now is that home control is relatively slow because home assistant plonks way too much context into the pipeline. It would work much better with an elective tool model like MCP but they don't have this yet.
PS: You could also choose the new Pine64 voice thing, It's looking pretty decent. It wasn't out yet when I bought mine.