>why wouldn't that same model be able to find ways to improve it's own algorithms beyond that of the capabilities of human mathematicians / ML researchers

Because algorithms have lower bounds, and the computational characteristics of LLMs are well-characterized by papers like https://arxiv.org/abs/2310.07923 . No amount of intelligence can make something faster than a mathematically-proven lower bound, any more than it could make 1+1=3 (that's why every single successful production transformer architecture has some form of O(N^2) attention layers, because it's mathematically impossible to achieve the same expressive power without any). There is room for speedup where current implementations are slower than the proven lower bound, but not when they're already close to it.

> There is room for speedup where current implementations are slower than the proven lower bound, but not when they're already close to it.

Sure, but I'm obviously not limiting research to improvements on current approaches only.

We know the brain is far more energy efficient and sample efficient than current AI. There is clearly better algorithms out there.

The question is who will find those next big algorithmic improvements like the transformer architecture? Will it be AI or humans?

My bet would be AI.