| ▲ | spijdar a day ago | |
It's worth pointing out that merely having the chain-of-thought in latent space isn't by itself "The Most Forbidden Technique". Rather, it's doing what the authors propose as the next step in the paper:
Unless I'm misunderstanding, they're suggesting RL training against the CoT. Which, uh, that is The Most Forbidden Technique, as I understand it, when combined with the latent reasoning space. It allows for the model to "learn"/optimize for malicious behaviors that are just far enough away from the quantized token they're auditing against. | ||