Even worse, when you do RLHF the behaviours out the model becomes psychotic.

This is gonna be an interesting couple of years.