> The whole 'its only predicting the next ___' has got to die.

Isn’t it the truth? Just like computers are mostly a lot of switching on and off. While we know the theory behind why we are switching things on and off, we still haven’t extracted the meaning behind the weights of an LLM (if there is one).