These aren’t raw base models they are the result of a ton of RLHF and various adjustments.

Bitter lesson wildly overstated in this context.

rlhf = reinforcement learning from human feedback

(had to look it up)