For Alpha Zero, the "better data" was trivial. The environment of board games is extremely simplistic. It just can't be compared to language models.

The problem with language is that there is no know correct answer. Everything is vague, ambiguous and open ended. How would we even implement feedback for that?

So yes, we do need new models.