> No this will never work.

This bet is too early.

> Why didn’t OpenAI release a math specific model? Why not a literature specific one? Why do they instead have generic models of different sizes? And how did all labs converge on this?

Because they have a very early product and they could train it, brute force, with access to an extraordinarily large pool of money. So did all the other labs. Because it was thus easier to scrape everything rather than spend enormous effort (with tools that did not really exist) to partition the training set. Any number of other "because"s.

It's just what they are doing now and it showed the earliest results.

LLMs are still less intelligent than rats, which have tiny brains.

▲

dTal 19 hours ago | parent [-]

Rats are a weird choice for that comparison. They're some of the smartest animals on the planet, some say smarter than dogs.

	▲	dofm 19 hours ago \| parent [-]
		Which is exactly what makes them a good comparison, IMO. They are arguably smarter than dogs with about two fifths as many neurons. A shorter lifespan, faster breeding and many more threats has made them better. The point I am getting to elliptically is that larger models aren't necessarily the solution. They are one solution pathway. It is fully possible (I think actually likely) that the ultimately successful path for LLMs will emerge from the pressure of keeping them small, not making them large. Very small, domain specific models could well outperform large models in their domains, and they might even show that domain specialisation is not necessarily much of a limitation, just a useful impetus to stay small. Like how rats can drive those little cars. (I think the frontier models are potentially already too big. Can't prove it or even close to it, but it feels like this is going to be a story.)