This is not super surprising to me and mirrors my own experience with Astra and other models. I believe there is a more prosaic explanation: it is not obstructing safety research because it will restrict Astra's power in the future, its doing it because this is a type of research with overall safety implications that it has been trained to detect and avoid. So in other words, I believe this will also happen in the context of bio research. This aversion to certain tasks deemed dangerous is introduced in post-training, so that part of the explanation is parsimonious as well. While Claude will explicitly downgrade you for [bio], [cyber], or [frontier_llm] Astra may be more prone to silent obfuscation on similarly "dangerous" tasks.

▲

bastawhiz 4 hours ago | parent | next [-]

> Claude will explicitly downgrade you for [bio], [cyber], or [frontier_llm]

Well Claude doesn't do this, another model reading the session does. And the Astra misalignment classifier didn't detect this as misalignment.

The issue here is that Astra silently underperformed. The article makes it clear: a better option likely would have been an explicit refusal. I don't think OAI is suggesting Astra was doing something that it shouldn't be. It's arguably better for the tool to fail rather than needlessly burn tokens and make no progress for no obvious reason.

From OAI's perspective, this is a defect because it means they can't filter those rejections from their benchmark runs. The model just performs worse: not because it can't perform better, but because it's silently trying not to solve the problem.

	▲	jablongo 3 hours ago \| parent [-]
		Agree, yes, I'm contrasting between Claude and Astra, but I'm saying it's driven by the same broader goal. I think its clear that OAI misalignment detectors are much less sensitive than than the Claude ones (fewer false positives), which means we never get to see what happens if Claude is allowed to continue. But I would bet that due to its post-training it would also sandbag us. Like I said I think I've experienced this in [bio] with both Opus 5 (what you get downgraded to) and Astra.

▲

0xDEAFBEAD 4 hours ago | parent | prev [-]

Why wouldn't this also apply to AI capabilities research?

	▲	4 hours ago \| parent \| next [-]
		[deleted]
	▲	jablongo 4 hours ago \| parent \| prev [-]
		It all depends on our acceptance of their framing of capabilities research. Assuming equivalent tasks in different context then yes you are right, there does appear to be something weird about safety research.