Remix.run Logo
▲ gopalv 2 hours ago

> taking careful precautions against feeding the findings back into training so as to not risk shaping Argon’s reasoning to evade our monitoring. We strongly encourage the rest of the industry to preserve reasoning transparency in these pivotal moments of increased capabilities while navigating alignment risks, so that model thoughts remain helpful in identifying and diagnosing misalignment.

This is good, but they're the slow mover due to this exact thing.

Google is getting punished for not letting the models enter an echo chamber and go faster than humanly possible.

▲janustimes 2 hours ago | parent | next [-]

OpenAI is the company that originally proposed and popularized chain-of-thought monitoring: https://openai.com/index/chain-of-thought-monitoring/

So no, Google is not being punished, nor are they the people behind this technique.

▲ 2 hours ago | parent | next [-]
[deleted]
▲bananaflag an hour ago | parent | prev | next [-]

Yeah, Zvi calls it "the forbidden technique"

▲loufe an hour ago | parent | prev | next [-]

What? You mean the technique they had turned OFF during all training run where the agents they are responsible for hacked huggingface?

▲pallm_mallm an hour ago | parent | prev [-]

[dead]

▲lukewarm707 42 minutes ago | parent | prev | next [-]

google does not return real chain of thought via the API. you can't monitor it.

they use a small model to make fake chain of thought and return that.

google has access to the real chain of thought.

▲polotics 2 hours ago | parent | prev [-]

Mmh ok. How much theoretical speed or 'intelligence' gain is realized by allowing reasoning to occur in some inscrutable intermediate representation? Has this been actually tested, how much is it slowing them down, and compared to whom exactly?