Remix.run Logo
adrian_b 3 hours ago

I doubt that it has ever been possible to obtain enough output tokens from OpenAI or Anthropic to be useful for training other LLMs.

In any case, had that been possible in the beginning, it stopped being possible long ago, because any suspicious accounts would be banned and the cost would be prohibitive even if they were not banned.

On the other hand, anyone can train new LLMs using the open weights Chinese LLMs, or the much fewer open weights LLMs with other origins, like the NVIDIA LLMs.

So in reality it is much more plausible for a US company to use Chinese LLMs for training, than vice versa.

ivanovm an hour ago | parent | next [-]

it is certainly possible and being done all over the place. there's a black market that chinese labs use to buy frontier american llm trajectories by the millions through US intermediaries. they're not even particularly shy about it, i have been offered $0.7 per opus 4.8 call

there's also a market for chinese labs sending checkpoints to US companies to be trained on US compute and sent back

i'm surprised that so many people take chinese tech reports about how they train their models at face value tbh

ux266478 an hour ago | parent | prev | next [-]

> and the cost would be prohibitive

The government of the Peoples Republic of China provides massive subsidies and incentives for R&D. The cost is absolutely not prohibitive, it's not even a factor. You are massively underestimating how much capital is involved in both countries respective industries. 500 billion on indirect compute? 好!

NooneAtAll3 2 hours ago | parent | prev [-]

> So in reality it is much more plausible for a US company to use Chinese LLMs for training, than vice versa.

that's... exactly the point?

make it easy to steal tech from opponent, not from you