Because it is directly analogous. Neural nets (whether biological or artificial) are not the best way to execute lots of deterministic computations quickly and reliably. That's why we invented computers.

I'm not convinced at all that this is the best way to reduce latency; there are many other ways of doing that.

Having a calculator in our brains would be handy of course, but a gigahertz multi core computer is still going to be better at anything that needs to do a lot of computation and or a lot of data.

Exactly. They've implemented a VM inside a transformer, turned an O(1) memory access call into O(n), optimized it down to O(log n) and wrote a post about how smart they are.

It's a nice bit of engineering, if you don't subscribe to YAGNI. If you do, you must ask the obvious question of what capability this delivers that wasn't available before. The only answer I've got is that someone must have been a bit chilly and couldn't figure out the thermostat