This is a fantastic visualization, but it and the rest of the literature all boil down to "input text goes in, we do some linear algebra on that and the model weights together, and... magic comes out." Of course, the precise incantations of the linear algebra _are_ important, the whole thing is worthless without the attention method, but that's just a method, a fairly simple one at that relative to what it does.

How does it get from the ideas to the intelligence? What if we saw intelligence as the ideas themselves?