It’s in the same neighborhood but isn’t really apples to apples. Distilling LLMs is to take a synthesized result that comes from huge amounts of innovation and computation, while the other is scraping what already exists as is. It is fair to say you stole our multi-billion dollar intellectual output in that scenario.

▲

lelanthran 19 hours ago | parent | next [-]

> It’s in the same neighborhood but isn’t really apples to apples. Distilling LLMs is to take a synthesized result that comes from huge amounts of innovation and computation, while the other is scraping what already exists as is.

Hang on, why is scraping the public pool of knowledge not taking "a synthesized result that comes from huge amounts of innovation and computation"?

You think that that all those github repos that LLMs trained on, were not the result of innovation and computation?

How many years of human innovation and cycles of computation during compilation were involved in bringing something like GCC or LLVM to their current status?

Those LLMs trained on every single research paper available online - were those papers not the synthesised result of billions of dollars of research, effort and (importantly, for you anyway) computation?

LLMs trained on the collected works of every author in existence. Were all those works just "as is"?

> It is fair to say you stole our multi-billion dollar intellectual output in that scenario.

No, we didn't. We simply took the model as-is.

▲

azinman2 19 hours ago | parent [-]

The Chinese models are the result of just as many papers, GitHub repos, etc… AND the synthesized results of those.

▲

lelanthran 19 hours ago | parent | next [-]

> The Chinese models are the result of just as many papers, GitHub repos, etc… AND the synthesized results of those.

Right, but they aren't the ones whining that other people are getting "the synthesised results" for free.

	▲	adventured 16 hours ago \| parent [-]
		I'm pretty sure the Chinese cloning isn't being arrived at for free regardless. Fable is quite expensive for example. If Anthropic has a real problem with API use, they can always raise the price.

▲

anigbrowl 15 hours ago | parent | prev | next [-]

And you can download the Chinese model weights and run them yourself - admittedly not too practical for Kimi K3 unless you're a big corporation, but eminently doable for others. The hardware to run Deepseek r4 uncompressed is about about $30k no=ew, well within the power of a small company or financially secure individual. Compressing and/or getting creative with hardware could bring that down quite a bit.

The difference is that the Chinese are sharing the models with everyone.

▲

vrganj 18 hours ago | parent | prev | next [-]

Yeah real convenient that the line would be exactly after pilfering all of human knowledge work, but before American model providers.

▲

api 15 hours ago | parent | prev [-]

... and they're at least releasing the results.

▲

flaunf221 18 hours ago | parent | prev | next [-]

> comes from huge amounts of innovation

Thousands of years of human innovation taken without any permission.

Everyone should steal everything not nailed from other AI companies. Then steal everything nailed and take the nails too. At least this way a tiniest bit might return back to society.

▲

anigbrowl 16 hours ago | parent | prev | next [-]

a synthesized result that comes from huge amounts of innovation and computation

The published algorithms like the transformer architecture are not patentable. You spent a lot of money on compute and China used the uncopyright-able output to steer its own training models? Too bad. I feel especially unsympathetic to OpenAI, who went from being a presenting itself as a benevolent nonprofit to a very-much-for-private private entity over night.

▲

ciupicri 19 hours ago | parent | prev | next [-]

You could say that both of them stole, but different stuff.

▲

azinman2 19 hours ago | parent | next [-]

Don’t forget that the Chinese models are also built on top of huge amounts of “stolen data” as well, beyond the distilled. So it’s basically all of the above. However, there’s no mechanism for the NYT or an author or anyone in the US to sue the Chinese companies that took their work.

	▲	ChrisLTD 19 hours ago \| parent [-]
		and if you're an author that lives outside the United States?

▲

jdkee 19 hours ago | parent | prev [-]

You can't steal intellectual property, only infringe on the copyright holder.

▲

azinman2 19 hours ago | parent [-]

The Chinese models poked and prodded better models for training data and to avoid having to pay humans to RLHF themselves. You can call it infringement, theft, whatever, but it’s quite obviously “not ethical” to me.

	▲	throw-the-towel 19 hours ago \| parent [-]
		Looks like these humans' jobs got replaced by AI.

▲

Danox 18 hours ago | parent | prev | next [-]

Sorry, no leg to stand on and relatively speaking no sympathy for the model makers or the data-center builders…

▲

azinman2 18 hours ago | parent [-]

I assume you don’t use any LLMs then

	▲	kbelder 14 hours ago \| parent [-]
		I use them and I'm fine with them training on our information and knowledge. It's what it's for. No one takes it or steals knowledge; they just copy it. But because of that, I'm also ok with the Chinese doing it. The worst they might be guilty of is breaking a terms of service. The only incoherent position is that it's good for one and not the other. You can consistently think it's bad in both cases, or good in both cases.

▲

polymer8563 19 hours ago | parent | prev [-]

bet