They can do math but not arithmetic, which I assume is what the commenter meant

LLMs can in fact do arithmetic, just not reliably owing to how numbers are represented probabilistically: https://arxiv.org/abs/2410.21272

▲

fasterik 8 hours ago | parent | prev [-]

I just asked ChatGPT to multiply two 4-digit numbers, and two 7-digit numbers without external help. It got both right. I'm sure it wouldn't have a 100% success rate, but saying it can't do arithmetic is just false.

▲

Xirdus 7 hours ago | parent | next [-]

I tried prompt "6379 times 3875" and it was off by exactly 1000 on first try, and correct on second. 0% success rate, sample size of 1.

	▲	dcrazy 4 hours ago \| parent [-]
		Isn’t that a 50% success rate with a sample size of 2?

▲

amluto 6 hours ago | parent | prev | next [-]

I would be nice to see what the (unencrypted) reasoning trace is like. Multiplication with scratch paper is not particularly difficult.

▲

tremon 8 hours ago | parent | prev | next [-]

Are you sure it honoured your stipulation of "without external help"? For all we know, it hacked its way into Wolfram Alpha and got the result from there.

	▲	jmillikin 7 hours ago \| parent [-]
		Arithmetic is well within the capabilities of even small local models: https://i.imgur.com/21tzGlN.png

▲

guelo 7 hours ago | parent | prev [-]

[dead]