I’ll say this until I am blue one the face. Nerds (software dev, maths etc) see how good LLMs are at things they care about and assume they will be broadly applicable in future.

There’s no reason to think this.

> There’s no reason to think this.

There are lots of good reasons to think this.

The first is that LLMs used to be bad at each of these nerd things, then toppled them like dominos. There's a pattern over time.

Another reason to think this is g. Across every known measurement, human (and animal) intelligence is convergent. Being better at one thing correlates with being better at another thing. Although the reason is not perfectly clear, the pattern is well established, and seems to apply to LLMs too - GPT-6 is smarter than GPT-3 at everything, not just math. There's no reason to expect different for GPT-9.

There is every reason to think this. Its about available quality data. The data that was easiest to fetch was already there, then we got some more data by asking experts to create datasets for post training. Once this is over, we will get to the outer world that didn't get to hoard it for bots to take it. This will certainly change. Put on a smart glasses and record what you do to fix a pipe. In 3-5 years, rinse and repeat.

Just like the nerds thought about computers, or the internet, or video games, or smartphones, or crypto, or... whatever.

The most passionate, intelligent people are usually a decent indicator of where culture is going to go.

> Nerds (software dev, maths etc) see how good LLMs are at things they care about and assume they will be broadly applicable in future.

I don't think this explanation is sufficient:

For example, many nerds care about 3D printing. On the other hand, my experiments (and the experiments of many nerds who I know) to let LLMs create files for 3D printing lead to horrendously lacking results.

Or many nerds care about linguistics or puns. Whenever a new LLM comes out to which I have easy access, I do the test, and let it explain some specific German jokes to me that are based on convoluted puns in the German grammar. Until now, no LLM that I had access to could give satisfying explanations of the puns on which these jokes are based.

Or even for coding (many nerds do care about elegant, sophisticated code): the LLMs that I could test were already overchallenged with the following task: I had written some code that is a very "artisanal", "clever" improvement of an algorithm over the version that one would find in a textbook. The task for the LLM was simply to write some code comments/documentation about the mathematical ideas upon which my improvement over the textbook version of the algorithm is based. It wasn't capable to do this. On the other hand, for a junior programmer, I would in such a situation expect that he goes through every single line and thinks through the algorithmic ideas so that he can learn from them.

---

Thus: even for "nerdy" topics, it is very easy to find tasks where LLMs still suck. So, my hypothesis about the nerds that you mention in your post is that these nerds are rather people who want to believe (with religious fervor) that the current LLMs are exceptionally good instead of just looking into a slightly different direction than where the tech billionaires want the users of LLMs to look at.

Correct spot on

> Nerds (software dev, maths etc) see how good LLMs are at things they care about and assume they will be broadly applicable in future.

It's worse than that. I am one of those nerds (a programmer), and I see that LLMs are shit at the things I care about. I have zero reason to believe that they will be good at other things either.