> ...imply ... a sort of transcendental introspection into their own brains, which if true would be absolutely groundbreaking.

> but ... it's bog standard reverse engineering, like any human would do, just vastly faster

For some values of vastly faster these are functionally equivalent and so equivalently groundbreaking.

Arguing that LLMs spontaneously evolve new capabilities outside of their training set (being able to naturally interpret weights and manipulate them to change their function) is a completely different proposition from being able to pattern-match Pac Man really fast from a pseudo-graphical representation (which is what they actually do here).

It's like ChatGPT coming up with a brand new brownie recipe that's surprisingly good vs. coming up with a working design for a cold fusion reactor.

Yeah the brownie is cool, but it's not that cool.

If faster intelligence was “groundbreaking” we would still be worshipping pocket calculators.