This is the "recursive self-improvement" they've been warning about.

I've always laughed at that phrase, thinking "haven't these things been improving themselves since the very beginning? This is all just iterative gradient descent, each step is a 'self-improvement'", but this really is different.

I wonder if we'll ever discover a new branch of linear algebra or something that allows us to find the optimal weights much more efficiently? Maybe AI will help us discover it?

It doesn't seem like there's room for recursion here though. A smart model can share its smarts with a dumb one, but the ceiling remains unchanged. Perhaps there's juice to squeeze in verifiable dimensions re: software, math, but there are similar ceilings to be found in those places. We can't keep asking harder and harder questions infinitely while remaining able to verify the answers.

It is astounding, but I don't think it's well set up for anything besides faster approaches to existing limits.

> This is the "recursive self-improvement" they've been warning about.

Well, except for the self part since it improved the performance of a smaller, dumber network.

And the recursive part, since it couldn't improve the performance of a larger network.

Good luck with figuring out what it does and enforcing any safety switches and rules I guess...