By then AI would be good enough to clean them all up...

To make my comment more on-topic: why do you think this is going to be the case? What newer LLMs will be trained on?

well you are assuming that there's not going to be any new progress and that we are going to be stuck with whatever LLM version we have currently