I guess self contained RSI can only possible if the information contained in all of recorded human knowledge to date is "reality-complete", ie sufficiently captures enough about reality that a "perfectly optimum learning algorithm" is theoretically able to reconstruct everything there is to know about our physical reality.

If the algorithms are insufficiently optimum or the recorded knowledge is of insufficient fidelity, then we'd find ourselves at a local optimum and would need to interface with reality.

A huge part of learning is to probe reality and observe effects, so I think even for current RSI to increase chances of success we would structure it so it can interact with an external environment of some sort, and receive inputs. It would be needlessly limiting otherwise.

Basically, but I think there’s some nuance here and some deeper questions.

What is intelligence? Problem solving. Learning. Prediction. The ability to model reality. There’s various ways to define it but it’s something like a superposition of those ideas.

How do you know you are intelligent?

You have to try to do those things.

The sum total of human knowledge and culture is the output of the output of a five billion year evolutionary process that selected for agent survival, which resulted in selection for intelligence among a wide range of other adaptations.

Can you figure out intelligence from that? Is intelligence even one thing, a theorem or algorithm that can be solved? If you did… how would you know?

That’s the hard part I think. Embodied humans “knew” they were getting smarter (in the evolutionary feedback sense) when they got better at hunting and defending and surviving and playing social games to form complex societies.

What metric would an RSI system use? If it’s the wrong metric you’ll spiral off into a kind of madness or overfit and collapse. How do you know it’s the right metric without testing it? How do you test it?