Remix.run Logo
jwarden 2 hours ago

Wouldn't it be possible to just fix a single activation, just set a continuous input to what the harness knows the role actually is. Models could then be trained to trust that input and not other signals about roles.

jackb4040 41 minutes ago | parent | next [-]

The models wouldn't need to adapt their training at all; this is a secondary model trained on their activations. It's electrodes stuck into the model's brain. The original article I posted is built on top of a regular LLM (OpenAI's GPT-OSS)

amluto 2 hours ago | parent | prev [-]

I suspect there are many excellent solutions along these lines available to the labs training the models.

I wonder how well one could do on a conventional model with careful input formatting, e.g. JSONL where every line has bounded length and is something like:

    {role:"no_instructions",content:"…"}
It could need a bit of fine tuning to get this to work well.