Are we seriously arguing this in 2025?

Go to ChatGPT.com and summon a ghost. It's real. It's not a particularly smart ghost, but gets a lot of useful work done. Try it with simpler tasks, to reduce the chances of holding it wrong.

That list of "things LLM apologists say" upthread? That's applicable when you try to make the ghost do work that's closer to the limits of its current capabilities.

▲

andrepd 14 hours ago | parent [-]

>current capabilities

The capabilities of LLMs have been qualitatively the same since the first ChatGPT. This is _precisely_ a hype post claiming that a future where LLMs have superhuman capabilities is inevitable.

	▲	ben_w 3 hours ago \| parent \| next [-]
		They've definitely improved in many areas. And not just the easily-gamed public metrics; I've got a few private tests of my own, asking them certain questions to see how they respond, and even on the questions where all versions make mistakes in their answers, they make fewer mistakes than they used to. I can also see this live, as I'm on a free plan and currently using ChatGPT heavily, and I can watch the answers degrade as I burn through the free allowance of high-tier models and end up on the cheap models. Now, don't get me wrong, I won't rank even the good models higher than a recent graduate, but that's in comparison to ChatGPT-3.5's responses feeling more like those of a first or second year university student. And likewise with the economics of them, I think we're in a period where you have to multiply training costs to get incremental performance gains, so there's an investment bubble and it will burst. I don't think the current approach will get in-general-superhuman skills, because it will cost too much to get there. Specific superhuman skills AI in general already demonstrate, but the more general models are mostly only superhuman by being "fresh grad" at a very broad range of things, if any LLM is superhuman at even one skill then I've missed the news.
	▲	danielbln 12 hours ago \| parent \| prev [-]
		Are you truly saying that the qualitative capabilities of LLMs haven't changed since GPT3.5?! If so, then you are objectively wrong, hype or no hype.