I generally disagree with the claim that distilling a model is equivalent to training on freely available internet content – mostly due to investment required to turn it into a model – but piracy is another story. Pretty inexcusable.

▲

Ekaros 6 hours ago | parent | next [-]

If they paid for tokens say via subscriptions. Wouldn't that just be same as acquiring books and using them as "fair use" to train? I really see no difference.

	▲	nicce 6 hours ago \| parent \| next [-]
		They did not need to destroy the models they got with pirated content. Would have been more fine if they would have needed to buy the books afterwards and train based on then, again.
	▲	buzzin__ 6 hours ago \| parent \| prev [-]
		"And when their eloquence escapes me Their logic ties me up and rapes me" Police, 1980

▲

antonvs an hour ago | parent | prev [-]

> I generally disagree with the claim that distilling a model is equivalent to training on freely available internet content – mostly due to investment required to turn it into a model

The frontier labs all give free access to their models. Why does “investment” change anything? Anyone who’s ever produced any content, free or otherwise, has invested in doing so.

The only plausible issue I see is that if distillation is being done by creating many free accounts to work around limits on free accounts, that’s a bit… impolite? But if they really wanted to avoid that, they could eliminate free accounts, and require users to sign a real contract governing what they can do with the model.

Of course they don’t want to do that, so they’re stuck in the same world as the rest of us, and they don’t have any real basis to complain about it without being hypocritical.