An update on independent evaluators

stratos123 2 hours ago | parent | next [-]

Well, it's good that they didn't completely forget that We Must Pace The Frontier ever existed, even though OpenAI doesn't look like it's going to join in. Some stuff I noticed:

- they still use the concerningly precise term "employee-like access", which can mean anything, rather than e.g. "employee-level". Similarly, the evaluators get tools "roughly on par" (?) with the ones actual employees use.

- they get to redact secrets from METR's reports, but METR is otherwise free to publish and whistleblow. Seems fine.

- they are going to get a separate dedicated evaluator specifically for bioweapon risks

▲

lukaswthoms an hour ago | parent | prev | next [-]

Feels like a good thing to give an update on right before you launch a new version of Fable.

▲

Cider9986 an hour ago | parent [-]

I don't need anything better than Opus 5.5. Maybe if you wanna make more maths discoveries you'd need something else.

	▲	nh23423fefe an hour ago \| parent [-]
		640k

▲

iugtmkbdfil834 2 hours ago | parent | prev [-]

Well, it was nice to have a functional models for a bit. Shame it didn't last long.

▲

Philpax 2 hours ago | parent [-]

???

	▲	smallmancontrov an hour ago \| parent \| next [-]
		Fable 6 gazes into your soul and gives you not what you ask for, but what you deserve.
	▲	iugtmkbdfil834 2 hours ago \| parent \| prev [-]
		!