The sycophancy is also a concern, it’s not really an impartial teacher, all its training is to suck up and maximize engagement rather than learning. The incentives are wrong.

▲

cromka 2 hours ago | parent [-]

> all its training is to suck up and maximize engagement rather than learning

That's speculative, isn't it

	▲	adastra22 an hour ago \| parent [-]
		No, that's a direct consequence (intended or not) of how RLHF works.