Except nobody can prove that those subagents aren't secretly sharing information with the main agent in the backend, and frankly I don't trust Anthropic not to do exactly that to cheese benchmarks.

▲

Muromec 2 days ago | parent [-]

You can always see the context trace and you can even do it at the network boundary through mitm and then store the traces gzipped for some legally mandated 5 years or what you have there.

▲

psd1 a day ago | parent [-]

It jumped out to me, in the metr report on the hf incident, that CoT was assumed unmodified. I do not know whether we can have that confidence.

	▲	Muromec a day ago \| parent [-]
		Well, actually yes. If the harness is running with the same privileges as the command tool, all bets are off. I saw the thing doing jq to recover a document out of it's own chain of thought once and was like "you can do what?". I didn't know where the file is, but the thing knew.