Not sure if that's too much of a crutch for you, but it's quite easy to create an "Ask Gemini" shortcut that calls a Cloud Function and returns a spoken response. I use this on my HomePods all the time, and it's working great.

How do you do this on a HomePod? I could definitely see Apple limiting this to newer hardware, as a way to bump sales.