> AI is a powerful tool because it can use tools and can also build tools for you to use or for it to use. But it is not a calculator.

It’s both, and a million other things. Of course AI can do complex calculations today. I can’t even remember the last time I saw an AI make a mistake on something like that.

The problem is that people’s mental image of what AI can do always has an inertia of 6-12 months. Like you can still see people talking about how image generation models supposedly can’t draw hands, even though today we have 2K AI-generated video with perfect hands.

I've seen AI-generated videos posted two weeks ago with six fingers on someone's hand. I don't know what model they used, but it still happens. Less often than it used to, granted, but it still happens.

The latest models can do fairly complex arithmetic without tools now, but they're definitely not calculators.

It can do basic arithmetic because it's fed millions or billions of synthetic calculations as training data.

Quick!

What's the next top-k tokens after each of these?

"999 + 999 = "

"999.0 + 999.0 = "

"999 * 2 = "

"999 * (3-2) = "

Mathematically identical. But as a token sequence, not so much.

Modern models will of course have very similar next-token distributions across all of these. But not identical!

Now add a few digits and you quickly land in territory whose synthetic training data is much sparser.

Code has determinism and runs in milliseconds. It's the better tool for the job.