Arithmetic Without Numbers – How LLMs Do Math

There is a beautiful MathOverflow thread on how mathematicians imagine concepts, https://mathoverflow.net/questions/38639/thinking-and-explai....

Very often it involves spatial thinking. Vide one example there:

> Once I mentioned this phenomenon to Andy Gleason; he immediately responded that when he taught algebra courses, if he was discussing cyclic subgroups of a group, he had a mental image of group elements breaking into a formation organized into circular groups. He said that 'we' never would say anything like that to the students. His words made a vivid picture in my head, because it fit with how I thought about groups. I was reminded of my long struggle as a student, trying to attach meaning to 'group', rather than just a collection of symbols, words, definitions, theorems and proofs that I read in a textbook.

▲

Npovview 3 hours ago | parent | prev | next [-]

Turing Award Winner: Thinking Clearly, Paxos vs Raft, Working With Dijkstra | Leslie Lamport

https://www.youtube.com/watch?v=U719vQz-WFs

Leslie Lamport : "I am not smart. I have the gift of abstraction."

Real mathematics isn't about details. Its about concepts and abstractions and how we compose them (LLMs are good at those aspects).

▲

iammjm 6 hours ago | parent | prev | next [-]

Why doesn’t it just call tools such as Mathematica for such operations?

	▲	ACCount37 2 hours ago \| parent \| next [-]
		For the same reason you don't run "4+6" on a calculator. External tool call has an overhead. It requires a round trip into an external tool. It requires an LLM to run in agentic autoregression - it can't be used in prefill. Which means that having native arithmetic capabilities is useful. Forward pass arithmetics are an LLM version of quick mental math. An LLM can read "#define SILLY_TIME_CONST (32060601000)" and have "SILLY_TIME_CONST is 60 h expressed as 216000000 ms" already cached by the end of the line, before it even emits its first token.
	▲	defrost 5 hours ago \| parent \| prev \| next [-]
		This is more how an LLM thinks about math internally - an LLM version of drilled tables being used for mental arithmetic "as humans do". When humans stall on these tasks, they reach for pen and paper, a slide rule, a calculator, etc. Mathematica is overkill for arithmetic, in addition it's licenced and can cost a bit extra. If an LLM were to reach for a light cheap arithmetic tool something like bc would be a good first stop - a CLI tool with a language that supports arbitrary precision numbers with interactive execution of statements. https://en.wikipedia.org/wiki/Bc_(programming_language)
	▲	jampekka 4 hours ago \| parent \| prev \| next [-]
		They do. I asked CharGPT for 327 x 48 and it used the "ChatGPT Instruments" calculator. Previously it used to run Python scripts, and may still do for more complex calculations.
	▲	breezybottom an hour ago \| parent \| prev [-]
		ChatGPT does, and has since 2023

▲

euroderf 6 hours ago | parent | prev | next [-]

The spirit of Rube Goldberg is alive and well.

▲

dominotw 23 minutes ago | parent | prev | next [-]

i dont like this new trend of generating html with ai to say something. i think some guy from anthropic started this trend .

now everything looks the same and i can no longer read on kindle.

▲

old_sound 2 days ago | parent | prev | next [-]

What happens inside an LLM when it tries to calculate with nothing but matrices.

▲

silvestrov 6 hours ago | parent | prev | next [-]

This is a very nice and fresh page layout.

▲

andrewstuart 4 hours ago | parent | prev [-]

I assumed it wrote Python or some sort of other code.

	▲	mavhc 3 hours ago \| parent [-]
		writing and calling an entire python setup seems massive overkill, surely just have an internal way of calling a simple calculator function would be millions of times faster