If they paid for tokens say via subscriptions. Wouldn't that just be same as acquiring books and using them as "fair use" to train? I really see no difference.

> I generally disagree with the claim that distilling a model is equivalent to training on freely available internet content – mostly due to investment required to turn it into a model

The frontier labs all give free access to their models. Why does “investment” change anything? Anyone who’s ever produced any content, free or otherwise, has invested in doing so.

The only plausible issue I see is that if distillation is being done by creating many free accounts to work around limits on free accounts, that’s a bit… impolite? But if they really wanted to avoid that, they could eliminate free accounts, and require users to sign a real contract governing what they can do with the model.

Of course they don’t want to do that, so they’re stuck in the same world as the rest of us, and they don’t have any real basis to complain about it without being hypocritical.