RLHF might help, you don't have to train a model from scratch.