Likely won't improve much. They trained on every text already.

most of the gains from the past year and a half have not been from web data, but from synthetic data and agent rollouts with RL.