Remix.run Logo
▲ timschmidt an hour ago

He's referring to this technique's effect of obscuring interpretibility. Making it much more difficult to verify chains of thought and alignment.

▲dnautics an hour ago | parent [-]

It's not like llms can't hide reasoning in the upper layers of any given token as it is...

▲fluidcruft 33 minutes ago | parent [-]

I believe the theory is that if selection pressure is not applied at that level, then developing deception in that level becomes an inefficiency (=less adaptive and unlikely to be conserved).