Even the greatest LLM will only just give you a snapshot of a perceived world state. You’ll only ever get one state, input, to output. Each snapshot in sequence is what will perceptively appear to us as AGI initially.

If we stick with the frames analogy, we know the frames of a movie will never give us a true living and moving person (it will never be real). When we watch a movie, we believe we are seeing a living breathing thing that is deliberate in its existence, but we know that is not true.

So what the hell would real AGI be? Given that you provide the input, it can only ever be a super human augmentation. That along with your own biological world state forming, you have an additional computed world state that you can merge with your biological world state.

We will be AGI, is the implication. Perfect weights will never be perfect because they are historical. We have to embrace being part of the AI to maximize its potential to be AGI.