You're forgetting about RLHF.

Many of us forget that an LLM without RLHF would not respond how we think.