Remix.run Logo
strictnein 4 days ago

[flagged]

sailingparrot 4 days ago | parent | next [-]

Yes they distill, but if you think you can trivially get a frontier-level model by "just" distilling from Claude's public API. you fundamentally do not understand the amount of work that goes into a modern post-training stack.

Without even talking about the fact that any distillation that was done was on Opus, as the timeline of Mythos/Fable vs Kimi 3 release dates just do not match in any plausible way.

If you want to read an educated take from someone that has actually spent the last few years working on post training I recommend Nathan Lambert's: https://x.com/natolambert/status/2079616308203942332

trollbridge 4 days ago | parent | next [-]

“Distillation” is just a term of art these days. Usage of competitors’ models these days is mostly around RLHF (minus the H, I guess).

strictnein 4 days ago | parent | prev [-]

Very interesting, thanks!

anfogoat 4 days ago | parent | prev | next [-]

> Do we need 40 people saying the same thing about how they don't feel bad and it serves them right and all that surface level stuff on every single one of these?

Apparently we do, given that we've got govt officials wasting time, money, and effort on whining about this now.

jrflo 4 days ago | parent | prev | next [-]

Agreed, it's becoming a bit of an echo chamber anytime Chinese models are discussed here. This is really par for the course for China though, in physical manufacturing they've been doing this kind of thing for decades, I'm not surprised the mentality has persisted into the AI race. Looks like rather than jumping ahead, they will remain persistently 6 months behind.

recursive 4 days ago | parent | prev | next [-]

The responses will continue until distillation-posting improves.

If we keep hearing accusations about how someone distilled something from someone, it seems like one of the few reasonable responses.

If my friend keeps complaining about how the inside of his car is wet, I will probably keep telling him to close the windows when it's raining, even if he thinks that is a tiresome take, lacking in insight.

calendar938 4 days ago | parent | prev [-]

Let's be real. In reality, it's the Chinese kid getting the 1600.

strictnein 4 days ago | parent [-]

lol, true true. Should have used a different example.