He's saying that pre-training an LLM alone can't do it, but if you run an LLM in a loop with tools (like any coding agent) then it can. Also, the technique his group came up with should be used more:

> This is the weakness of deep learning that is alleviated with a new algorithm that my group presented in Nature a couple of years ago. Our “continual backpropagation” made one small change: every so often a less-used neuron would be re-initialized to small random weights. This allows the variation to continue and plasticity to be retained.

Here's the paper: https://www.nature.com/articles/s41586-024-07711-7

It has a fair number of citations, but I haven't looked into how much it's used.

Sorry this makes no sense — humans also use tools to evaluate their discoveries.

Kant said something like this: knowledge can’t be obtained by pure thinking, it needs interaction with the world.

This is obvious to me so why is the author making a claim that LLMs can make knowledge without access to environment but purely through thinking in aether