> Yes, but at least now we're comparing artificial to real neural networks, so the way it works at least has a chance of being similar.

Indeed, and I wasn't even saying it's wrong, it may be pretty close.

> What would be cool would be if we could to some "mechanistic interpretability" work on the brain's language generation circuits, and perhaps discover something similar to induction heads.

Yeah, I wouldn't be surprised. And maybe the more we find out about the brain, it could lead to some new insights about how to improve AI. So we'd sort of converge from both sides.