Wrote about it here https://simianwords.bearblog.dev/why-domain-specific-llms-wo...

In short: there’s a reason OpenAI doesn’t have health care model, chemistry model, mathematics model etc. If they want a smaller model they nerf all domains together like in GPT nano. Why? It’s because intelligence compounds and if you take away capability of mathematics from a model, you remove capabilities in all dimensions.

So basically you r saying diversity matters

Interesting though ..