Remix.run Logo
▲ HarHarVeryFunny an hour ago

I agree that definitions of intelligence should be based on capability rather than implementation, but given that we're discussing LLMs where the intelligence is in the weights, then "the weights don't change" can obviously be taken as pointing out that the model can't learn - the best it can do is memorize and load memorized stuff into context.

Memorizing facts is not the same as learning skills. Implementation-wise memorization can just be RAG, but learning of skills does need to be in the "weights" - whatever is driving the actual behavior of the model.