Yes, I didn’t mean that literally. Since “attention is all you need” it’s more like “predict the full output in full for this blob of input text”. That doesn’t change the “wtf is going on with the weight understanding bit” in what I wrote.

> it’s more like “predict the full output in full for this blob of input text"

Hm. I would actually go so far as to say "predict" is just the wrong word to use in 2026, as it smuggles in an out-of-date picture of what LLMs are that's from the pre RLVR era, and which is frequently used to suggest they don't model anything real in the world but rather the statistics of language, and thus have all sorts of limitations that they've now surpassed ("an LLM will never solve a major open mathematics problem, they're just predicting tokens"). These days I tend to use the more neutral phrase "generate the next token"; I'd only say "predict" in the technical context of (say) the pre-training part of the training run where the object is indeed to minimize the cross-entropy loss function by predicting next tokens in existing text. Generating a token with a modern LLM is better described as something like, "given a context and the weights from the totality of its training, including not just next token prediction with pre-training but also solving vast numbers of math and coding problems with RLVR, the LLM generates tokens one by one." (Consider that an LLM trained purely to optimize for next token prediction with pre-training is terrible at programming and math tasks, as we all remember from 2024. In that sense the amazing results we've seen in the past two years are due to the departure from pure "next token prediction" that RLVR made possible.)

Put another way: would it be useful to describe the AlphaZero chess engine as "just a next move predictor"? (It, too, was a large neural network trained with RL.) I would rather say: "it's playing chess by generating the next move given the state of the board." Which is what humans do too, though we do it with different hardware.

Anyhow. I'm not sure the weight understanding is all that strange. (It's impressive, but so are many thing LLMs do these days.) LLMs are very very good at coding and math problems, of which this is one, and in particular, their RLVR training presumably includes many tasks involving neural networks, doing tasks with PyTorch, etc. It's amazing and impressive, I agree, but I'd say less amazing than proving major open theorems in mathematics, which we already know they can do.