They literally cannot. They can detect the user’s intent to do math, and then use a different tool to do math, hopefully with the correct inputs. The LLM is not suited to giving deterministic answers to math problems.

Maybe it’s just a different and in some ways better way of doing mathematics? Maybe how we think and process mathematics of physics is just but one way to do it? I’m not suggesting an LLM will prove 2+2=6 because of course that’s nonsense but maybe it can invent a new calculus?

> The LLM is not suited to giving deterministic answers to math problems.

Less so with formal mathematics proofs maybe but I think in general humans don’t provide deterministic answers to math problems or questions either. Humans get it wrong all the time and when you ask a human to solve a problem they may solve it in a different way than before.

reasoning models can trivially do math (open up astra and ask it some undergraduate problems), but eventually break down (similar to how humans start to lose track if asked to do math without any assistance)

It would be more accurate to say they can do math instantaneously without even thinking, at a level far beyond what humans can do. (I assume you're talking about doing arithmetic.)

  TLDR: Astra has 8.6x better odds of doing a reasoning task without CoT than the
  next best model (Fable 5.1), and can do 7.2 serial arithmetic steps in a forward
  pass vs 4.1 for the next best model (Gemini 3.8 Flash/Fable 5.1)

Reasoning models can do math on their own without external tools.