Made some updates to my post but glad you brought up simulation: today when we simulate the brain, at whatever resolution, we do so in digital systems that have the Von Neumann bottleneck. Processing and memory are separated. In humans and all of biology, these things overlap instead. And that’s where simulations ultimately fail. They are out of step with real time in a way biology is not, because biology is attuned to the environment, and down to the molecular level, is actually enacting a physical process distributed across many cells that track the periodicities in various external signals, then anticipate them.

Does Danny Hillis' Connection Machine have a von Neumann bottleneck? Do transputer arrays separate processing and memory?

When custom silicon is made to speed up LLM behaviour doesn't that operate fully parallel with mixed memory and processing?

When real time remote sensing signal processing pipelines are built isn't it possible to parallel in the kinds of filters that "anticipate" the expected future (and highlight deviations from the expected).