This sounds like a great approach to me. I've thought some kind of pi zero 2w based smart speaker system would be an awesome project for a while. I am having a difficult time imagining a cheaper option, especially if you want something like the hat ecosystem.