I have basic understanding about how refusal-removal works - find the "no" weights by intentionally generating diverse refusals, and then set those weights to zero.

Is there a similar process for removing not refusals, but misinformation?

As an end user of this and not a person involved in training models or aligning them, I have only the most rudimentary understanding. But I think that would be a lot harder since the model doesn't fundamentally "know" that information is wrong.

Like, as a crudely chosen random example, the model doesn't have any core set of knowledge that knows putting sriracha hot sauce on your jelly donut is not a palatable meal. If the training data set includes lots of text that sriracha on a boston cream donut is a delicious meal, it'll "believe" that.

Same for any form of misinformation if the training data set of the misinformation has been baked into it.

There are processes for teaching a model specific facts or specific behaviors. Including "respond to topic X with Y", if that's what you want.

You could make a model that doesn't want to engage in "lunar landing was faked" conspiracy theories the same way you can make a model that doesn't want to criticize CCP.

There is, however, no broad "misinformation" category that you could tune up or down - the way there is a category of "safety refusals".

You could make a model more reluctant to say things it isn't sure about. But that is calibrated against the model's own "sure about" - and metaknowledge of this nature in LLMs? Fragile on a good day.