Remix.run Logo
▲ kenjackson 2 hours ago

It seems like your definition of AGI is too focused on implementation details. IMO it has to be focused on behavior. Once you say something like “the weights do not change” then I think we’ve lost the plot. Instead say this is what an intelligent being can do, but a GenAI cannot.

▲HarHarVeryFunny an hour ago | parent | next [-]

I agree that definitions of intelligence should be based on capability rather than implementation, but given that we're discussing LLMs where the intelligence is in the weights, then "the weights don't change" can obviously be taken as pointing out that the model can't learn - the best it can do is memorize and load memorized stuff into context.

Memorizing facts is not the same as learning skills. Implementation-wise memorization can just be RAG, but learning of skills does need to be in the "weights" - whatever is driving the actual behavior of the model.

▲imtringued an hour ago | parent | prev [-]

I don't get it. Weights don't change means the model can't learn.

This is not merely an implementation detail, continual learning is the core of AGI.

▲brainwad 38 minutes ago | parent [-]

It means the weights by themselves can't learn. But they could be used in a harness that can learn in other ways. Insisting that it be done at the neural net level seems anthropocentric - because we form memories via neural connections, that everything else must, too.

Also: I wouldn't say a human with amnesia who relied on notes to their future selves lacks intelligence, so I'm not sure being able to form memories is in fact necessary.

▲ranyume 12 minutes ago | parent [-]

Even with a harness, there are things LLMs can't do. e.g. Write 50 entertaining short stories, each one completely different from the other but all sharing one motif.

In the example, LLMs choose the shortest path that would be verifyable by an evaluator. So the model is essentially blind to "entertaining" and "completely different". The LLM will output stories with different variations of the same structure.

If the LLM can somehow read other works and derive from them, it will merge the motif with the story being read.