Meta did not abandon opensource. I would love to see a smaller distill, or a moe of this size but the benchmarks seems competetive as long as it isnt benchmaxed witch i would not be suprosed if it is.

▲

ignoramous 3 hours ago | parent [-]

> Meta did not abandon opensource

Open weights*

I don't think outside of the Big 3 (Ant, OAI, GDM), given the strong competition from China, any other Lab has a chance at capturing the coding market if they aren't open weights (save for xAI whose latest Grok looks every bit good & will probably rely on Cursor for distribution instead of going open weights). There's literally no other selling point, as the capabilities have mostly converged by now among the chasing pack.

▲

dannyw 37 minutes ago | parent | next [-]

Don’t sleep on NVIDIA and Nemotron.

It’s not completely open source, but they actually release their pretraining and post-training datasets with some redactions for (cough) pirated content.

They also have very good code and playbooks for actually doing a fine-tune, CPT, etc.

Even if you’re not tuning a Nemotron model, its mixes are very excellent for your replay data slice; or general experiments. Way better curation and quality than Dolma, etc; or other large huggingface data mixes I tested.

	▲	nickludlam 6 minutes ago \| parent [-]
		Yes, I second Nemotron. I'm using Ultra remotely and Super locally, and I find them very useful for RAG-like problems. I wouldn't really use them for coding.

▲

ComputerPerson 2 hours ago | parent | prev [-]

There was a good discussion yesterday on the DeepSeek Flash release thread about this.

There's a large market, very large, who want the best regardless of what it costs. Probably a large enough market to keep that domain of research afloat (as opposed to shifting research manpower to cost cutting).

The reasoning is just that the marginal cost of AI is very secondary to fixed costs of the businesses themselves; it's not an excuse to sacrifice performance.