Remix.run Logo
sailingparrot 3 days ago

Yes they distill, but if you think you can trivially get a frontier-level model by "just" distilling from Claude's public API. you fundamentally do not understand the amount of work that goes into a modern post-training stack.

Without even talking about the fact that any distillation that was done was on Opus, as the timeline of Mythos/Fable vs Kimi 3 release dates just do not match in any plausible way.

If you want to read an educated take from someone that has actually spent the last few years working on post training I recommend Nathan Lambert's: https://x.com/natolambert/status/2079616308203942332

trollbridge 3 days ago | parent | next [-]

“Distillation” is just a term of art these days. Usage of competitors’ models these days is mostly around RLHF (minus the H, I guess).

strictnein 3 days ago | parent | prev [-]

Very interesting, thanks!