Computers are not special. They are just a heat engine like everything else. We feed them concentrated energy that they dissipate to do work. They do work on data: we give it data (some of it is called code) and it gives us back data. It's all about the information content, how does that data communicate something and relate to the world?

"Training" is just upfront work. Why on Earth people expect to get from the machine that processes data some novel information that did not exist before?

This whole fantasy hinges on not understand the sheer amount of data these LLMs are being trained on, and some magical thinking about it producing some novel information ex nihilo somehow. I will never understand how intelligent people fall into this patterns of thought.

We can only get from computers what we put into them.