No, if you burn your model into the silicon you don't need memory as the output of a layer flows through circuitry directly into the next one. No I/O to memory of any kind. You still need a bit of memory for in flight answers, but that's it.

You can pipeline in software too, and even with a dedicated circuit, you need to put weights somewhere, because you're not going to have a special partially-applied FP8 FMA unit for each of your trillion model weights.

I can accept the idea of specializing a circuit for a specific model shape, but I'm not seeing a need to specialize a circuit for the weights inside the shape.