Remix.run Logo
make3 3 days ago

It's about the claim of whether these companies could develop a similarly powerful model without larger companies building their own first, which is an important point, and it's likely not the case.

It's also about the larger companies explaining why they can't be as efficient, of course they can't, they're not just ripping the outputs of another model that someone else invested billions to train.

IncreasePosts 3 days ago | parent | next [-]

Why would that matter? OpenAI or whatever frontier lab couldn't have built their frontier models without the entirety of humanity unknowingly developing their training set for 5000 years.

It would be one thing if Moonshot was breaking into OpenAI servers and stealing trade secrets, but the only thing they are doing is looking at the output of the program, which is exactly the service that OpenAI offers. So, at best, this is a ToS violation. Sucks for the frontier labs I suppose, but live by the sword - die by the sword.

mrhottakes 3 days ago | parent | prev | next [-]

> they're not just ripping the outputs of another model that someone else invested billions to train.

True, they're simply ripping the inputs that humanity invested thousands of years and trillions of dollars to produce.

make3 3 days ago | parent [-]

no argument from me here

doctoboggan 3 days ago | parent | prev | next [-]

Yeah agreed, from one standpoint I couldn't care less that they did a "distillation attack", but I am interested in knowing if China is able to develop open weight frontier models without the prior existence of a huge model to distill from.

PaulHoule 3 days ago | parent | prev | next [-]

Simply knowing it is possible to do something makes it easier to do.

cindyllm 3 days ago | parent [-]

[dead]

xcf_seetan 3 days ago | parent | prev [-]

> they're not just ripping the outputs of another model that someone else invested billions to train.

If they payed for inference, doesn't they own the output? So if I pay for a model to generate code, isn't that code mine to do with it whatever I want? Just curious.

make3 3 days ago | parent [-]

Not arguing for the morality of it, but if we're going by the law because that's what you're using in your comment ("don't I own" which only matters wrt the law), then you explicitly accepted a Terms of Use which excludes distillation as a use case.

Now of course they themselves trained on the whole Internet for free, etc.