Remix.run Logo
foolswisdom 2 hours ago

> Take a task, any medium-sized task, decently scoped that you'd trust to give to Sonnet to finish without a hitch. Now give it to ANY open-source frontier model and watch them struggle and go in circles while failing tool calls and randomly assuming things.

Claude used to be much worse than it is now, just as bad the open weights models are. And the open weights were worse. The labs will also try to keep the lead, but at some point people start seeing real value from open models. Maybe you say they're not ready yet for medium tasks, but everyone sees the writing on the wall.

brunooliv 2 hours ago | parent [-]

I hope you're right and I want you to be right, but, even seeing the current hype around local models, etc... and open-source models, I think the industry is currently under a big confusion where they see the benchmarks of things like Kimi, GLM, Qwen, they play with it via opencode, and they think like: "Wow this is pretty good, I want to deploy this". But they don't understand how the KV cache grows over time and can take almost as much memory as needed for a 30B param model, they dont understand that a quantized model WILL NOT be the same as a full precision one, and they surely don't see the engineering work needed to serve inference to even tens of customers at a decent quality and latency level.

The biggest moat of these giant labs and models is increasingly shifting towards deployment capabilities and (debatably) having better (proprietary) harnesses.

The models themselves can be impressive on benchmarks, but unless they can be served reliably to customers either at scale, hosted somewhere, or even on edge with predictable latency and memory usage, then frontier will always be leading.