Remix.run Logo
▲ dangom 3 hours ago

I'm not sure I really buy into the way these concepts are currently defined. What progress to date was not RSI and did not build on top of earlier progress?

The main characteristic of our physiology is that of adaption, and we have fairly grounded research on progressive overload as a means to guide adaption to whichever direction we want. Should we then call humans RSI machines if they go work out or read a book, or study to get better at the activities they carry out?

Define an objective, choose an optimization algorithm, bring a loss down. If it keeps going down, do I call that RSI because the loss descended from the step that came before it?

Now, I see the nuance that the optimization algorithm changes over time, leading to interesting higher order effects the likes of which Hoefstadter describes in the GEB, but fundamentally, the idea is the same.

I also don't see what an intelligence explosion is supposed to mean. If we increase the models efficiency by 10x, and reduce costs by 10x, so we have 100x more software and math proofs being shipped by unit of time in say 2028, does that really solve all of our problems?

Say that gets distributed over the world. Do we get every human to be computer literate? Say that gets concentrated. Do we solve problems no one has the means to even care about?

I would consider an intelligence explosion when the models tell us whether our efforts are well or misdirected as looking back from the hindsight of 10 years ahead, and capable of explaining why in simple English.