Because people are paying the model companies to give them factual answers, so they hire data labellers and invent verification techniques to attempt to provide them.

Even without that, there's implicit signal because factual helpful people have different writing styles and beliefs than unhelpful people, so if you tell the model to write in a similar style it will (hopefully) provide similar answers. This is why it turns out to be hard to produce an evil racist AI that also answers questions correctly.