Can "LLMs are bad at counting" be generalized to "LLM are better in complex stuff but make more mistakes in simple"?

fluoridation 4 hours ago | parent | next [-]

I would phrase it as "LLMs are good at big picture stuff and bad at fine detail", or to put it another way, they're accurate, but imprecise and with low reproducibility.

	▲	bregma an hour ago \| parent \| next [-]
		It is my experience that it's the opposite. LLMs are very very precise but wildly inaccurate. They might give you 17 significant digits but be off by 10 orders of magnitude, to use a metaphor.
	▲	benj111 an hour ago \| parent \| prev [-]
		But where does that leave us when programmers treat themselves as architects with the AI doing the drudge work? As seems to be the fashion. It then means you have 2 parties focussing on the big picture and no one focussing on the details.

▲

ozlikethewizard 5 hours ago | parent | prev | next [-]

Its more LLMs are better at vague problems with multiple non perfect solutions, and struggle at problems that require precision.

▲

klodolph 5 hours ago | parent | prev [-]

No, I don’t think so. LLMs are good at a lot of simple tasks, but bad at certain simple tasks. Moravec’s paradox in a new iteration.

It applies to humans too. Calculus is “simple” but it takes something like sixteen years to train a human to do it, if all goes well. Meanwhile, most humans think that inverse kinematics is, like, the easiest thing in the world (it’s a super complicated task).

	▲	fluoridation 4 hours ago \| parent [-]
		Calculus is definitely the harder task, considering it took a species developing the cognitive capacity for symbolic reasoning for it to show up, whereas any animal can figure out how to position its limbs. Yeah, we figured out how to make CAS programs before inverse kinematics software, but that's because computers were made to solve numerical problems, not to replace the cerebella of chordates.