Remix.run Logo
▲ reducesuffering 6 hours ago

I'm baffled this is surprising. This has been the goal and every data point on the trajectory has been leading here and beyond. Opus 5.5 was created by many research engineers using Opus 4.6 extensively. Opus 6 is being created by using even more intelligence and agents on Opus 5.5. They are approaching autonomy and Opus X+1 will soon be rapidly created autonomously from Opus X. Given we have practically zero interpretability into the models, and we have not solved alignment, they will recursively improve out of our control. The engineers in A.I. labs are actually generally scared of what's unfolding. They don't know at what point this RSI takeoff occurs and what will happen after.

▲vessenes 6 hours ago | parent | next [-]

Yes - “opus 5 helped train 5.5” is extremely cool but not shocking. Hand editing weights without tooling to change outcomes is shocking.

▲bluebands 6 hours ago | parent [-]

I thought it had tooling, didn't it?

▲pizza234 6 hours ago | parent | prev [-]

The "surprise factor" is explained in the article - training LLMs until now has been a largely different process:

> Today, recursive self-improvement is often operationalized as a model writing code that launches training jobs. An LLM that can inspect, debug, and improve its own weights, or those of a newer model, offers a richer, more native path to it.

▲reasonableklout 4 hours ago | parent | next [-]

It is also a much less interpretable and auditable path towards recursive self-improvement. At least with training code, human engineers can inspect and reason about the output.

▲reducesuffering 5 hours ago | parent | prev [-]

"training LLMs until now has been a largely different process"

Yes, but nobody should be surprised when large amounts of intelligence results in changes to this process in support of the obvious end goal! Lesswrong.com discussions about AI were predicting this behavior a decade ago, before the transition from machine learning to the Attention paper and LLM's. The intelligence process will change but the world's premier capital and human intelligence have been laser focused on this and the trajectory has been crystal clear for those paying attention to second order effects and not the immediate here-and-now... It's quite astonishing how much discussion on HN is about present LLM capabilities being the limit of AGI and immediately proven wrong within 6 months.