> People are realizing that what they need isn't more general intelligence, it's more specialization. A small but well tuned coding model...

It’s not quite as simple as that. Several studies have shown the opposite: models trained on more diverse knowledge tend to cross-pollinate across domains. So a more generalized model can actually perform better than a specialized one.

That’s why you’re not seeing tons of tiny models (one for Python, one for Pascal, one for Rust, etc).

▲

kennywinker 3 hours ago | parent | next [-]

This is definitely the position of the big ai companies.

But it doesn't match my experience. Qwen3.8 27b is clearly smarter at coding than MANY bigger models. gpt-oss-120b for example, is almost 4x the size, and performs way worse at coding tasks.

It's clear to me that you can build small models that work well at specific tasks.

Python vs Rust is probably too fine grained a way to build a model. Coding in general seems like a better target.

There will always be a place for large generalist models, no doubt. But I think that place is much smaller than the big ai companies are counting on.

▲

ericd an hour ago | parent | next [-]

Gpt-oss—120b is like 1000 years old in AI years, whereas Qwen 3.8 27b is pretty young. What you’re seeing is that parameters aren’t apples to apples, and at a given parameter level, the new models are much, much better than the ones from a year or two ago. Like, to a comical degree.

	▲	MichaelZuo 35 minutes ago \| parent [-]
		Wasnt this known by everyone who cared to pay attention? It practically became a joke about how a huge amount of the training data for GPT-4 was bottom of the barrel reddit vomit and obvious bot spam. Leading to many bizarre edge cases.

▲

vunderba 3 hours ago | parent | prev | next [-]

I think we’re in agreement.

I make heavy use of smaller local models on a daily basis (Qwen3-VL for auto-captioning images, Gemma3:27b for some translation work, etc.). Gemma3:27b is a good example of a very capable general purpose multimodal model and has handled almost everything I've thrown at it from sentiment analysis to documentation writing.

I suppose I was drawing a distinction between specialized and general intelligence versus small and large. I don’t think those are necessarily mutually exclusive.

▲

ac29 an hour ago | parent | prev [-]

gpt-oss-120b only has 5B active parameters, so its not surprising Qwen3.8 27B outperforms it (Qwen3.8 is also ~13 months newer, which is forever in LLMs)

	▲	kennywinker an hour ago \| parent \| next [-]
		Fair enough. I’ve barley touched oss-120b, so i didn’t know it was so few active params. For a direct comparison, qwen3.6-35b-a3b is still better at coding than oss-120b. And Qwen3.8-27b is still better at coding than opus 4.1. Yes, if you list off models 27b is better than it’s all older models. But that’s my point - newer models are better than older models at the same AND much smaller size. That’s because model size matters less than they say. Training data and model architecture matter more.
	▲	anon373839 an hour ago \| parent \| prev [-]
		No, it’s not the active parameters. Qwen 3.8 Flash has 6B active and it smokes both models.

▲

vmg12 2 hours ago | parent | prev | next [-]

> more diverse knowledge tend to cross-pollinate across domains

Yeah, the cross domain transfer learning from RL is overstated by a lot.

▲

lelanthran 3 hours ago | parent | prev [-]

Problem is conflict of interest: the studies are mostly from the providers of the biggest models, or someone who received free tokens to do the research.

	▲	phoghed an hour ago \| parent [-]
		It would be nice to hear exactly how the conflict of interest has impacted the specific studies and how they are wrong rather than conspiracy theory level speculation and hand waving at the entire category