No this will never work. Domain specific models will never be a thing because intelligence carries over and compounds.

Why didn’t OpenAI release a math specific model? Why not a literature specific one? Why do they instead have generic models of different sizes? And how did all labs converge on this?

Why does Fable just not train on non cybersec and non biology data but instead have clearly costly and annoying classifiers?

▲

fasterik a day ago | parent | next [-]

Your examples (math, literature) involve natural language. It stands to reason that a general language model will be more competitive in those domains. If you want examples of successful domain-specific models, look at AlphaZero and AlphaFold. LLMs aren't anywhere close to achieving that level of competence at abstract strategy games or protein folding.

"This will never work" is a pretty confident assertion for a field that's so young and rapidly evolving.

▲

simianwords a day ago | parent [-]

> I don't foresee AGI arising out training bigger LLMs (Though investors won't realise that for a while yet).

This is what the parent said. AGI won't rise out of AlphaZero and AlphaFold in the same way AGI won't rise out of Houdini chess engine. This is the industry consensus.

▲

fasterik a day ago | parent | next [-]

>AGI won't rise out of AlphaZero and AlphaFold in the same way AGI won't rise out of Houdini chess engine.

That's a straw man. Nobody thinks AGI will rise out of domain-specific systems. The question is whether domain-specific systems are necessary for AGI.

Of course, the problem is that AGI isn't a well-defined concept. But if we define it as achieving superhuman performance across several hundred domains where there are objective measures of success, it doesn't seem far-fetched to predict that it will involve some general reasoning system paired with a bunch of specialized modules.

▲

simianwords a day ago | parent [-]

The parent said

> I don't foresee AGI arising out training bigger LLMs

I agree that AGI will involve tool usage but not only involving domain specific AI models.

But lets try to find the discriminating point in the discussion - do you believe AGI will necessarily involve training bigger LLM's or not?

I believe they are necessary. WBU?

	▲	fasterik a day ago \| parent [-]
		You're still intentionally misreading the OP's statement. If you read it again in context, they're clearly saying that they think training bigger LLMs is not sufficient. I think I agree with that statement, but my confidence is pretty low. No, I don't think LLMs are necessary for AGI at all. I think there are multiple paths to AGI, some of which involve LLMs and some which don't.

▲

dofm a day ago | parent | prev [-]

AGI is a macguffin (or a shaggy dog) for a story told to investors. It has never been plausible on the timescales suggested and it almost certainly will not emerge from LLMs.

▲

dofm a day ago | parent | prev | next [-]

> No this will never work.

This bet is too early.

> Why didn’t OpenAI release a math specific model? Why not a literature specific one? Why do they instead have generic models of different sizes? And how did all labs converge on this?

Because they have a very early product and they could train it, brute force, with access to an extraordinarily large pool of money. So did all the other labs. Because it was thus easier to scrape everything rather than spend enormous effort (with tools that did not really exist) to partition the training set. Any number of other "because"s.

It's just what they are doing now and it showed the earliest results.

LLMs are still less intelligent than rats, which have tiny brains.

▲

dTal 19 hours ago | parent [-]

Rats are a weird choice for that comparison. They're some of the smartest animals on the planet, some say smarter than dogs.

	▲	dofm 18 hours ago \| parent [-]
		Which is exactly what makes them a good comparison, IMO. They are arguably smarter than dogs with about two fifths as many neurons. A shorter lifespan, faster breeding and many more threats has made them better. The point I am getting to elliptically is that larger models aren't necessarily the solution. They are one solution pathway. It is fully possible (I think actually likely) that the ultimately successful path for LLMs will emerge from the pressure of keeping them small, not making them large. Very small, domain specific models could well outperform large models in their domains, and they might even show that domain specialisation is not necessarily much of a limitation, just a useful impetus to stay small. Like how rats can drive those little cars. (I think the frontier models are potentially already too big. Can't prove it or even close to it, but it feels like this is going to be a story.)

▲

thereitgoes456 a day ago | parent | prev [-]

DeepMind did release a math specific model. And OpenAI has released a coding specific model.

The answer to your question is “because the market isn’t big enough”, not because it doesn’t work. Why would knowing about 2019 internet memes help you in any way at coding?

	▲	andy99 19 hours ago \| parent \| next [-]
		> Why would knowing about 2019 internet memes help you in any way at coding? 99.99% of the knowledge an LLM has is useless for a given scenario, the hard part is knowing what the .01% that’s needed is. Knowing as much as it can means the model can handle edge cases, turns of phrase, etc. Put another way, it avoids overfitting. That’s basically the insight that’s given way to the current AI boom.
	▲	InsomniacL a day ago \| parent \| prev \| next [-]
		> Why would knowing about 2019 internet memes help you in any way at coding? https://github.com/Brainrotlang/brainrot "Brainrot is a meme-inspired programming language that translates common programming keywords into internet slang and meme references."
	▲	simianwords a day ago \| parent \| prev [-]
		> And OpenAI has released a coding specific model They did and retracted it because they found that GPT 5.5 beat codex pareto optimally. This keeps happening. > because the market isn’t big enough Huuh? market isn't big enough for AGI? The parent suggested that AGI would emerge from this process.