Intuitively, I’ve always had an impression that using an analogue circuit would be feasible for neural networks (they just matrix multiplication!). These should provide instantaneous output.

Isn’t this kind of approach feasible for something so purpose-built?

https://futurism.com/scientists-create-ai-glass

You might wanna look at https://taalas.com/