Remix.run Logo
WhitneyLand 4 hours ago

False dichotomy right?

Are Chinese labs impressively innovating? Clearly.

However this doesn’t rule out possible gains due to distillation.

I don’t know the degree of the latter but both things could certainly be true.

SirHackalot 3 hours ago | parent | next [-]

Didn't Anthropic train on our collective data just to sell it back to us for $100/month? On top of that, Apple is suing them over alleged IP and trade secret theft by ex-Apple employees. Hard to feel too sympathetic, and I’m not an Anthropic hater in particular…

DashAnimal 3 hours ago | parent | next [-]

That Apple lawsuit is against OpenAi, just for clarity

SirHackalot 2 hours ago | parent [-]

Wow, I should never comment first thing in the morning... Thanks for the correction, you’re right. I will see if I can still edit my comment.

hugopuybareau 3 hours ago | parent | prev [-]

The only data-related lawsuit Anthropic got was the books nah ? And they paid only a minor part as paid agreement compared to what they would have paid losing the trial

SirHackalot 2 hours ago | parent [-]

Yep, that second part of my comment was an article I read about OpenAI and my mind mixed it up with Anthropic. My mistake.

culi an hour ago | parent | prev | next [-]

Fable was available for a few weeks before Kimi K3 came out. If it was a distillation attack, then that's a truly groundbreaking technological feat to distill a model like Fable in 2 weeks

fnord123 3 hours ago | parent | prev | next [-]

Also possibly true: Anthropic is running Kimi locally in their hardware and "distilling" it.

an hour ago | parent | prev | next [-]
[deleted]
cma 3 hours ago | parent | prev | next [-]

If they can distill fable into a full model post training run in ~15 days without the real thinking traces, yet we know Claude chats degraded with the thinking traces removed (chat resume bug from earlier in the year they reported stripping thinking to shed load as being the cause of degradation), how big can this degree be?

api 4 hours ago | parent | prev [-]

“Distillation” is just indirectly pirating the largely pirated training data used to train the original model.

“You stole my warez!”

serial_dev 3 hours ago | parent [-]

If we do it, it's training a model. When they do it, it's distillation attack. - Anthropic

esafak 3 hours ago | parent [-]

"You are distilling what I have rightfully pirated."