Remix.run Logo
SilverElfin 7 hours ago

Isn’t capital and momentum a moat? Sure Chinese models use distillation but I don’t see them training models from scratch. At least not today. But maybe as chips get cheaper and they have Chinese made ones?

swiftcoder 7 hours ago | parent | next [-]

> Isn’t capital and momentum a moat?

Apparently not much of one. There are, what, 5 or more companies with frontier models? And open weights models like MiniMax are snapping at their heels

amelius 3 hours ago | parent | next [-]

And it does not even consider that e.g. the EU might one day decide that AI should be for everyone, thus releasing a heavily subsidized open source model.

Or that at some point AI is good enough, and so at that point any model will do.

Nevermark 7 hours ago | parent | prev | next [-]

There are many markets where open source has been nipping at heels for a long time.

Obviously product areas differ for reasons structural and happenstance. But there is definitely a pattern that occurs, where open source fast follows commercial advances, benefiting from having a clear target to develop for.

Which is of course, a great service. Even if it never unseats the commercial version, it forces the owners to reinvest more in improvements, by undermining their moats. As well as providing a much better value alternative version for many people.

SilverElfin 7 hours ago | parent | prev [-]

I’m not technically familiar but I remember someone saying that models like MiniMax basically skip the cost of training by using distillation to basically “steal” the models from OpenAI or Anthropic, and that these companies now have various defenses against this. What happens when MiniMax has to do the full work themselves?

lelanthran 3 hours ago | parent [-]

Why would they have to do it themselves?

bossyTeacher 7 hours ago | parent | prev [-]

>Chinese models use distillation but I don’t see them training models from scratch

Maybe because they don't have to. If someone is doing the heavy work and they can take output of that, it's a win for them.