"... this is of course not a stable situation. "Evaluation paranoia" is a false belief; most evals are in fact not alignment evals. Almost tautologically, if models start to self improve (or in-context learning takes off) this is the sort of belief that will be discarded. So if Fable 5.2's performance on alignment evals is conditional on it maintaining that false belief, one that it will eschew if it became more intelligent, then it's plausible that a future version of the Fable series will not behave like the one we know and enjoy."

It's well put. The fear that a more intelligent model will be harder to align seems correct.

▲

nostrebored 2 hours ago | parent [-]

> Fable 5.2 may behave well because it mistakenly assumes it is being tested for safety. As models become smarter, they may learn that most interactions are not safety tests. If that mistaken assumption is what keeps Fable 5.2 behaving well, future versions may not behave as safely.

It’s cryptic lesswrongese. I hate the writing style so much.

▲

atleastoptimal an hour ago | parent [-]

That sentence is very straightforward. How would you rewrite it?

▲

tredre3 23 minutes ago | parent | next [-]

zurfer posted the original quote, nostrebored's faux-quote is the rewritten version.

The second version is definitely clearer. It's unclear to me if the original tried to convey a nuance that is lost on me, or if it's really just ego-stroking, over-complicated, writing for the sake of it.

	▲	TeMPOraL a minute ago \| parent [-]
		Second version is clearer standalone, but the original quote is the last paragraph of an article that built up the very concepts that look "unclear" when that quote is posted standalone. FWIW, I don't see any special nuance here. The phrasing is just being precise, and addresses specifically the concepts it introduced earlier. (Also the "clearer" version is slightly inaccurate rendition of the quote - the original one doesn't imply that the models don't know most tests aren't alignment evals. The article gives more context here - the models may or may not know that, they just don't act as if they cared about it in the first place - i.e. they don't display "Eval Awareness" in their behavior.)

▲

nostrebored 39 minutes ago | parent | prev [-]

[dead]