| ▲ | vanuatu 4 hours ago | ||||||||||||||||
To me its clear that it is decel the only reason other labs can catch up is because the frontier labs can be distilled, and they siphon a % of the labs' revenue to reinvest into the next iteration full accel would mean nationalizing the big 2 labs and locking in manhattan project style until RSI (Edit: some great counterpoints in the replies. my view has definitely been changed!) | |||||||||||||||||
| ▲ | m_ke 4 hours ago | parent | next [-] | ||||||||||||||||
only if you only get your news from main stream business press and Big Lab propaganda channels There's no chance K3 is a distill of Fable, it came out way too soon after the limited fable release to be feasbile. If you look at all of the top ML conferences, chinese labs contribute way more to advances in ML than "Open"AI and Anthropic: https://www.reddit.com/r/TheMachineGod/comments/1pi4q7f/pape... This K3 release just helped every other lab on the planet stay in the race by making it possible for them to build on top of it, placing them at the frontier starting line instead of having to spend billions of their own dollars and risking it all to attempt to catch up. The open source contributions I linked to above will move the whole field forward and reduce the costs of training and inference for everyone. Open science compounds on it self, every new advancement pushes the field forwards and opens up new grounds for future improvements. It is impossible for a single closed lab to consistently stay ahead of the rest of the field, especially in a huge growing research area like machine learning. The only only advantage the big labs have is money, but the naive scaling game is not sustainable long term when you have to pay 10-100x more then the fast followers and we start getting more and more open models or use case specific models that can handle 90% of high volume use cases. Research is a high variance, low expected value activity, meaning that the few large concentrated labs have to be conservative with their bets and double down on proven things when scaling up. The rest of the field is like a diversified portfolio, with thousands of players making smaller riskier bets that only require a few of them to succeed (like K3 did here, and DeepSeek a year ago) EDIT: also if you look at most of the work from OpenAI, it's mostly taking existing promising open research work and scaling it up. (except for things like CLIP and etc from Alec Radford) | |||||||||||||||||
| |||||||||||||||||
| ▲ | calebkaiser 4 hours ago | parent | prev | next [-] | ||||||||||||||||
OpenAI's head of strategic futures publicly stated that you can't explain the quality of the newest Kimi via distillation. Further, you can just read the papers released alongside most open models. Plenty of hugely influential research results published that drive the frontier forward. It's not like these models are just existing architectures downloaded from Huggingface and trained on frontier lab APIs. | |||||||||||||||||
| ▲ | ZeroGravitas 4 hours ago | parent | prev | next [-] | ||||||||||||||||
Distilling can be done by fast follower closed models too, so this argument against open models doesn't hold up. | |||||||||||||||||
| ▲ | pluto_modadic 4 hours ago | parent | prev [-] | ||||||||||||||||
being able to replicate it in the open means that there's nothing special about frontier models. Frontier models would have to do something extraordinary or unique, or unreplicatable, because clearly there is no moat, and US companies are sitting on huge nvidia valuations and get surprised when competitors beat them. | |||||||||||||||||