Remix.run Logo
kamranjon 7 hours ago

The fact that API based distillation is even a conversation right now makes me feel like the U.S. has their heads so far in the sand that it’s not really excusable.

These Chinese labs are producing novel models, publishing their techniques and sharing their open weights and the first topic of conversation is how they stole from U.S. AI labs.

Setting aside the fact that it doesn’t make any feasible sense to do API distillation, these models are outperforming frontier models on a number of benchmarks, and often times run more efficiently by several orders of magnitude.

We have to stop crying distillation, it’s getting embarrassing and at this point feels even a bit delusional.

tristanj 6 hours ago | parent | next [-]

There's little doubt that Kimi K3 was distilled off Claude.

Anthropic stated in February that Moonshot AI (the creator of Kimi) distilled ~3.4 million exchanges from Claude models, as explained in their press release https://www.anthropic.com/news/detecting-and-preventing-dist...

kamranjon 2 hours ago | parent | next [-]

It’s so funny to me that Anthropic can make claims like this one with zero evidence provided.

DeepSeek and others like Minimax are publishing deep research on Multi-Head Latent Attention and Mixture of Experts, Multi-Token Prediction, novel Sparse Attention approaches, I mean they trained long context models on a fraction of the resources and gave everyone the recipe.

Chinese labs might not have the funding of labs like Anthropic, but at least they provide the receipts.

tristanj 2 hours ago | parent [-]

There's reproducible evidence of Kimi K3 spontaneously identifying itself as Claude https://x.com/denisewu/status/2077984660211269870

This behavior is exactly what you'd expect from a model distilled from Claude.

Someone even took the time to analyze Kimi's ambiguous identity, in great detail: https://github.com/rgreenblatt/which_claude_is_k3/blob/main/...

And there's an entire Reddit thread discussing this https://www.reddit.com/r/LocalLLaMA/comments/1m2w5ge/did_kim...

That doesn’t prove Anthropic’s specific 3.4m-session allegation, but calling it “zero evidence” is no longer credible.

Kimi K2.5 was worse in a hilarious way, it identified itself as Claude and referenced Anthropic's Constitutional AI as some of its guiding principles https://huggingface.co/moonshotai/Kimi-K2.5/discussions/38

staticman2 2 hours ago | parent | next [-]

> This behavior is exactly what you'd expect from a model distilled from Claude.

This is not at all what I would expect because it's trivial to change the training data to replace Claude with Kimi. In fact I'd argue it's almost certainly not saying that due to distillation.

tristanj 2 hours ago | parent [-]

I encourage you to review the links before committing to a position. The writeup on K3's anomalous trans-model identity is very comprehensive.

K3 reproduces Claude's internal model identifier when prompted, something which the real Claude models themselves do not emit. This is highly suggestive that K3 was trained on Claude metadata (API logs, tagged synthetic data), rather than Claude's chat outputs.

And it's well documented that Chinese labs are buying large amounts of raw Claude metadata https://www.chinatalk.media/p/how-to-buy-cheap-claude-tokens...

staticman2 an hour ago | parent [-]

For context you just asked me to read a document that starts with:

"Caveat: fully AI-generated research."

And that you quoted or paraphrased directly.

kamranjon 2 hours ago | parent | prev [-]

By evidence I mean logs, I mean IP addresses, I mean timestamps. They claim millions of requests, let’s see literally any of them?

I don’t consider a tweet by Denise Wu, who works at Anthropic, to be reproducible evidence.

I don’t consider “Caveat: fully AI-generated research.” To be someone taking time to analyze anything in great detail.

Because two AI models produce vaguely similar front-end styles when generating similar prompts I also do not consider to be of much value?

I think this is what I mean when I say the U.S. has its head in the sand. The Chinese labs are releasing ~60 page research reports with citations and analyses and evidence and Anthropic is throwing up defensive blog posts with zilch. I’ve seen more detail in a tech blog from Uber than anything I’ve seen from Anthropic.

tristanj 34 minutes ago | parent [-]

You've backtracked significantly here.

"Zero evidence" as you claimed earlier isn't accurate. You've moved the goalposts from "evidence" to "raw internal logs I can independently audit," which is a different and very high standard. Sure Anthropic didn't publish logs, IP addresses, or account IDs of the accounts involved. But that's true of any cybersecurity breach/abuse disclosure ever made. Companies are furtive to reveal how they detect fraud, because doing so exposes the signals used to detect bad actors, and makes future abuse easier. Not revealing the "evidence" you're asking for is industry standard practice. You're complaining that Anthropic is following industry standard practice, and conveniently defining the "evidence" you need as something Anthropic is never going to publish.

> I don’t consider a tweet by Denise Wu, who works at Anthropic, to be reproducible evidence.

Is the issue here that she works at Anthropic? Because Denise Wu doesn't work there.

> I don’t consider “Caveat: fully AI-generated research” to be someone taking time to analyze anything in great detail.

The experiments were run by Ryan Greenblatt, who is a real AI safety researcher (at Redwood Research).

The identity experiments and Greenblatt analysis are trivially reproducible. The methodology, code, and metrics are all there in the Github repository. You can ask your preferred AI to independently replicate these results, and it will give you a result within an hour.

You’ve also reduced the evidence to “two models producing vaguely similar front-end styles,” which is not what either analysis shows.

From the analysis, Kimi K3 identifies itself as Claude 15% of the time. How do you explain that? Qwen and GPT identify themselves as Claude 0% of the time.

If a long document is too much analysis for you, someone else made a simple chart which measures the KL divergence between Kimi K3 and other major models. They found K3 is unusually similar to Fable 5 & Opus models. That is, Kimi K3 has an very similar style and phrasing to that of Anthropic models.

https://typebulb.com/u/lab/you-re-relatively-right/full

overfeed 6 hours ago | parent | prev [-]

While it sounds like a lot, do you suppose 3.4 million sessions come even close to being sufficient to train a frontier model?

Assuming each session was 10,000 words each, that's 34 billion words; lets call it 50 billion tokens (0.05 trillion) unfairly pilfered from Claude. That left Moonshot needing to scrounge for the other 14.950 trillion training tokens required for a baseline frontier model.

tristanj 5 hours ago | parent [-]

3.4 million is the number of sessions Anthropic detected. The actual number of Claude sessions trained on is likely >100 million. There are tens of thousands of accounts funneling Claude sessions into Chinese labs https://www.chinatalk.media/p/how-to-buy-cheap-claude-tokens...

They are used for post-training, i.e. calibrating the model to understand and use tools/command line more effectively.

overfeed 4 hours ago | parent [-]

> 3.4 million is the number of sessions Anthropic detected. The actual number of Claude sessions trained on is likely >100 million.

That's an increase of only a single order of magnitude, increasing my estimate of exfiltrated tokens from 0.05 to 0.15 trillion - a far cry from the 15 trillion required.

> They are used for post-training

Possibly - it may be too much data for post-training, unless further curation was done. However, this is not distillation; you know it, I know it, Dario knows it, but "Distillation Attack" is a short, memorable, sciencey-sounding, political sound-bite with enough malevolence to be deployed on the floors of congress, or by the usual fear-mongering newstainment talking heads.

tristanj 3 hours ago | parent [-]

You're conflating pre-training data volume with post-training data volume.

Nobody is suggesting Moonshot used 15 trillion tokens of Claude data to pre-train a base model from scratch. That would be impossible and nonsensical.

This is entirely about distillation, which happens during post-training (alignment and SFT). Here, datasets are measured in millions or billions of tokens, not trillions. 50 billion Claude tokens is far, far than enough to copy Claude's reasoning logic, writing style, and tool-use ability to the pre-trained base model.

> However, this is not distillation

I don't understand how you're so caught up on the term "distillation". Distillation is using a larger model's outputs to train a (weaker) student model. Which is exactly what's happening. It's a standardized term that has been in use for a decade.

overfeed 40 minutes ago | parent [-]

There is a lot of supposition going on your part and mine. IMO, Chinese labs are not dependent on OpenAI/Anthropic outputs; they definitely use the outputs, but along other training/post-training data.

Now that Anthropic hides the real thinking tokens in a way that precludes future CoT distillation, we'll find out which side is correct based on whether Chinese AI labs close the gap or not.

My bet is they'll close the gap; nothing about frontier AI is magic, once something is shown to be possible, experienced practitioners almost always figure out how to accomplish the same feat, though not always on the same way. This is why frontier US labs keep leapfrogging each other every few months.

overfeed 6 hours ago | parent | prev [-]

> We have to stop crying distillation, it’s getting embarrassing and at this point feels even a bit delusional.

It's a PR campaign - when they say its an "attack" they don't mean on Anthropic - but on America itself. What kind of American can let such a brazen attack go unanswered? At the very least, they ought to demand the dangerous, pinko, stolen models be banned in all 50 states, and pay whatever price demanded by the patriotic, freedom-loving, all-American AI labs that can never be accused of stealing.