Given these models could not have been trained in the first place if they had to license every line of random fan fiction on the internet, I think distillation also being fair game is a tradeoff everyone should be willing to take (unless they want to decelerate, but that's a different conversation).

How is distillation an "attack" but gigascraping the Internet to the point of crashing servers and everyone needs Cloudflare and Anubis now not an "attack"?

I'm not aiming for a what about kickflip here: I'm saying we need to either agree on some rules or stop crying foul. Maybe the coherent legal theory is that neural networks and intellectual property don't interact. That would be weird but it would be consistent, a market could price it, I could do coding stuff and know if I was illegaling.

But this weird gerrymander that no judge will really rule on in an emphatic way is like, bad for the planet, bad for markets, bad business.

There are a lot of reasons to look forward to DeepSeek Huggingface drop kicking the unambiguous frontier weights in like, November, but I think my favorite one will be "who's distilling now bitch?"

    >  The problem with distillation attacks

Apologies for repeating myself here, but what you call "distillation" is function approximation.

I feel for the teams at Anthropic and Open AI, but unlike startups from prior eras; Anthropic and OpenAI have decided to be in the business of selling compute. Not creating a product that uses compute, but a product that's math running on compute. This is different from what Google is (or, rather was. As always, RIP Google 1998-2019).

Google's algorithm might be math, but Google search isn't. Google search is a process that's continuously operating in the background. Google crawls pages. Google stores and indexes what it finds. Google then exposes this to retrieval via its algorithm. User uses algorithm.

Now, let's compare that to AI models. When Anthropic serves Mythos / Opus etc, they're taking input or x from their user, doing compute, and then serving the result of the Mythos / Opus function, i.e.,

    f(text) -> (text_transform)

According to Stone-Weierstrass, given enough values of y for f(x), anyone can approximate this function.

The fidelity and sophistication of this approximation definitely requires a lot of cleverness and effort, and it is arguably an imposition on Anthropic and OpenAI. But on a long-enough timeline, they don't even have to poll Anthropic or OpenAI. As the internet is flooded by PRs, content, emails written by Mythos / Claude, and just people otherwise sharing the results of Claude prompts, then there's an ever increasing set of data to approximate the f(x) that's f_Claude.

Eventually, in the future, anyone will be able to create a good enough approximation of the f_Mythos. Which is Anthropic's product.

Anthropic and OpenAI can now wage war on mathematics and the open-ended compute. Or, they can adapt and build a better product.

Choosing Option B was the Silicon Valley option / choice. I think the OG large-scale Valley lobbying effort, the Semiconductor Industry Association, was unique in that it prioritized and chose to do real research.

https://en.wikipedia.org/wiki/Semiconductor_Industry_Associa...

https://en.wikipedia.org/wiki/Semiconductor_Research_Corpora...

This helped the industry to survive and outcompete the pressure they were facing (at the time).

This is nothing like music piracy.

American labs have ripped everything out of the internet. And now they cry someone else is “stealing” from them. Cry me a river.

Did you enable the flicker-free TUI mode?

Why is it hard to scroll?

That varies by jurisdiction. In the United States, copying the phone book (or otherwise copying facts from someone else's collection) has been legal since 1991:

https://en.wikipedia.org/wiki/Feist_Publications,_Inc._v._Ru....

And funnily enough, most laws about compiled databases might not apply, for one.

And then there's new updates related to AI that fully take out LLMs from protection.

There's little doubt that Kimi K3 was distilled off Claude.

Anthropic stated in February that Moonshot AI (the creator of Kimi) distilled ~3.4 million exchanges from Claude models, as explained in their press release https://www.anthropic.com/news/detecting-and-preventing-dist...

> We have to stop crying distillation, it’s getting embarrassing and at this point feels even a bit delusional.

It's a PR campaign - when they say its an "attack" they don't mean on Anthropic - but on America itself. What kind of American can let such a brazen attack go unanswered? At the very least, they ought to demand the dangerous, pinko, stolen models be banned in all 50 states, and pay whatever price demanded by the patriotic, freedom-loving, all-American AI labs that can never be accused of stealing.

That’s true up until the point where you can ask the hardware you made to make its own application layer.

This line of thinking makes no sense because it assumes that labs that distill from frontier models are doing nothing else. It's the classic "the Chinese can only copy" mentality, and it's going to end poorly for American companies.

I'm pretty sure that all labs are distilling each others' LLMs, maybe apart from Anthropic and OpenAI. It would be stupid not to do it, because it's cheap and effective. But that's not the only thing they're doing. If you think K3 and GLM-5.2 got this good only from distilling frontier models, you're not paying attention to Chinese labs' publications.

Distillation from a teacher model solves the self-start problem, that is, building a model to the point where it reason coherently. Without distillation, solving self-start is incredibly difficult since it requires millions of high quality training samples. Creating that kind of dataset takes an enormous amount of effort.

Once a model becomes competent enough to perform complex reasoning, a teacher model is no longer necessary. The model can now reason about its own behavior and build a better version of itself through recursive self-improvement (RSI).

Kimi K3 is capable of RSI.

In public with budgets that don't risk destroying the American economy presumably. Yes it may be slower.

There doesn't need to be progress at this point. Some models even from 1 or more years ago are useful for some purposes

The existing models are still going to exist. As hardware improves, there will be a day where it might cost a tenth of a penny to churn through 100M tokens a second of Opus 4.8. Established compute providers will invest in improving the models incrementally when margins drive them to look there.

Answer the question "how much does 5 cents of LLM computation in July 2026 cost in July 2005" and you'll have the answer to your question.

Don't forget to account for all the costs. It's not just that CPUs are X times slower. Memory is X times smaller, too, and networks are X time slower. And all this hardware is many times more expensive.

If I'm getting my mental estimation right, training a 2026-frontier-class LLM in 2005 would be somewhere on the order of all the computation power in the world at the time. It's not that many more factors of magnitude before you end up at "all the computation power in the world up to that point".

Is this some form of rage bait? 2005 we hadn't the GPUs, we have today. There are other factors, but I think this is the big one. The mathematics of building an LLM are really old, we just hadn't the hardware to do the needed calculations.

Because the transformer architecture that enabled modern LLMs wasn't invented until 2017[1]?

1: That's the "T" in GPT fyi, even though Google is the author of the research paper that changed everything

Moore's Law or something. Were you alive in 2005? The Nintendo DS getting the Opera browser was a big deal. THAT 2005 with today's LLMs? Hilarious.

We didn't have the compute required (GPUs powerful enough to parallelize forward and backward pass). This compute is what allows us to train from human knowledge or distillation.

because you had neither the chips or the information in 2005. You have probably on the order of 5000x to 10000x more GPU compute today than you had in 2005 and three to four magnitudes more openly available data.

The first "L" in LLM does the work. In 2005 you had no Github, Stackoverflow, Youtube, common crawl and no archive of digital ebooks.

>People infringe on Anthropics IP

No.

Authors do not infringe on IP when they read another's book, nor should the lumber company be able to dictate how I use planks and if I can resell them if i'm done with them.

You're framing it as if the added value of the author or lumber company, awards them consideration when somebody uses the products to create more value.

IP law was always a big mess, and these questions cross far into ideology instead of law; but I do not understand people who think we need an ideology where more IP-law is good for society.

> People infringe on Anthropics IP

Unless someone literally stole the weights somehow (which is not out of the question, I doubt either oAI/Anthropic have the capabilities to prevent a state-level actor getting those weights), distillation from generations is not infringement on anyone's IP nor is it stealing nor is it an attack. It can't be. As long as you pay for tokens you get to do whatever you want with them. Someone saying you can't doesn't mean it's an attack or their IP or whatever. They either sell the tokens or not. They can decide to not sell them to anyone, but again that's not stealing.

And their ToS are a joke. Imagine how people would react if MS had ToS saying that you can't use MS software to develop solutions that compete with MS. They'd be laughed out of the room. Somehow it's ok for token sellers to decide what you do with the tokens? Why? If you pay for something you get to do whatever you want with that output. Train, distill, whatever.

The output of Anthropic's models is not Anthropic's IP, as that would destroy their market, if Anthropic owned all the software it generated, and all the content. So distillation, which is just using those outputs is always going to exist.

> People infringe on Anthropics IP

Anthropic’s model outputs contain no IP. This is actually a simple legal proposition (rare in this field!) that derives from the fact that only specific classes of IP exist: copyrights, patents, trade secrets, and trademarks. Examining each, it is clear that API outputs do not qualify. Anthropic disclaims copyright in outputs; the outputs are not patented; the outputs are not secret (a prerequisite to having trade secrets); and trademarks are irrelevant in concept.

I'm pretty sure that LLM output is not intellectual property. Nobody owns it, and it can't even be copyrighted. So using output from Anthropic's LLMs in ways Anthropic does not condone is not IP infringement.

Whether or not its legal to distil models, it is obviously morally permissible to do so.

Anthropic, OpenAI, etc do not deserve legal protection.

Considering they were the original infringers, I don't know how anyone can expect tears to be shed here. The best we can hope for is for all these cancerous - and they really are the definition of a cancer - money burning entities to all fall apart to distillation attacks like these.

anthropic model output is not their IP

that would be existential doom for them because then they have a case to claim ownership of their users' codebases

no corporation would sign off on that

The value is simply that it is easier. The same way it is easier to ask someone who has experience for advice than reading hundreds of textbooks.

Regardless of whether it’s intellectual property or it isn’t intellectual property, it doesn’t actually matter. If AI doesn’t stop seeing diminishing returns in scaling up, and it hasn’t yet in the 10 years since the attention/transformers paper, the advent of AI will be the most important development in the history of humanity. Controlling that machine, or at least having one of your own, is an existential problem for nation states. It’s like a matter of national defense.

Do you really think intellectual property laws will prevent this in practice? It’s like as if we said, “hey, USSR, you can’t make a nuke, too! We patented that already.”

Asking China to not distill our models down is equally as ridiculous.

Anthropic’s IP is basically null and void for how they created it. And they might not want to try and challenge this in court, considering how they had to settle for using text books they had no right to use