To everyone praising Open weight models, could you answer a simple question?

If Anthropic doesn't make money because of distillation attacks, how would they convince investors to invest in them, such that it makes financial sense for Anthropic to train even bigger models?

Assuming it is preferable for everyone that we get better models in the future. Distillation attacks remove the financial incentive.

▲

__bjoernd 19 hours ago | parent | next [-]

This assumes all the open models are just a result of distilling Anthropic models. Which remains to be proven.

And if they are, the point remains that Anthropic has a brittle product advantage that users and investors should be cautious about.

▲

asadotzler 14 hours ago | parent | next [-]

OpenAI says they're not just distillation. "Some observations on Kimi: 1. It's a very good model! I don't think its performance can be explained away by distillation or anything like that."

If you put any faith in claims made by OpenAI's "Head of Strategic Futures," then it's hard to believe these models are just the result of distillation.

▲

simianwords 19 hours ago | parent | prev [-]

> investors should be cautious about

if they are cautious, what would make them invest in newer bigger models without the expected return? generosity?

▲

drakythe 19 hours ago | parent [-]

Sunk cost fallacy.

So much money has been invested in Anthropic and OpenAI at this point that to declare it a loss and walk away could potentially destroy a lot of VC firms, and a non-trivial chunk of the US Economy.

▲

simianwords 19 hours ago | parent [-]

Sunk cost fallacy is not relevant to future investment.

▲

SideburnsOfDoom 3 hours ago | parent | next [-]

> Sunk cost fallacy is not relevant to future investment

Outside of future choices, what in the world is sunk cost fallacy in fact relevant to? Do you understand words?

▲

daadx 11 hours ago | parent | prev [-]

what an absolute plonker

do you know how ROIC is calculated?

Good luck hiding your huge sunk cost of bilions and billions in there bro. I wish people who had zero understanding of actual finance would never comment about it.

Moreover if they declare their existing assets are bunk, the valuation is marked down, especially after now pricing in failure risk - this is catastrophic for VC's.

Again, bro, just be quiet.

	▲	simianwords 4 hours ago \| parent [-]
		sanest comment here

▲

Alpha3031 19 hours ago | parent | prev | next [-]

I don't think the timeline is plausible for Moonshot to have done any distillation from Fable, which means any distillation training data would have been from Opus.

▲

swiftcoder 19 hours ago | parent | prev | next [-]

The frontier labs have been tightly controlling model access lately to prevent further distillation, and the Chinese labs are still catching up rapidly - it's likely that whatever moat they used distillation to overcome early on is already dry

▲

surgical_fire 18 hours ago | parent | prev | next [-]

[Pending evidence that Chinese models are only based on distillation]

▲

throwa356262 19 hours ago | parent | prev | next [-]

While I am sure some do distillation (even r/LocalLLaMA is full of randos bragging about it), I wish the companies named by Anthropics would respond to these accusations.

I think at least deepseek can easily take Anthropics to court and win a defamation case. The number of requests allegedly done from deepseek IP range is so small it is barely enough to run a few benchmarks.

▲

watwut 14 hours ago | parent | prev | next [-]

> f Anthropic doesn't make money because of distillation attacks, how would they convince investors to invest in them,

I dont care if that company dies. Or if OpenAI dies. I dont mind investors investing into other things either.

▲

bellowsgulch 19 hours ago | parent | prev [-]

You can simplify your question even more:

To everyone praising free and open source software, if software companies don't make money, how are they and YOU going to get paid, PERIOD?

	▲	simianwords 18 hours ago \| parent [-]
		Commoditising complement