Seconded on the "not cheap" argument here. I've spent $25 worth of tokens completing a one-week task in an afternoon, or rather my company spent the money. I would never have personally felt OK with throwing this much money after some prompting back and forth for a few hours, one lazy Saturday afternoon. I ran the risk of not finding the solution before the token usage would be too high for me to want to carry on, if I was my own credit card linked to the account.

Of course money in this situation is a bit of a funny measurement, right, because if I was able to take the rest of the week off as soon as I had solved the one-week problem, then I would have no problem at all throwing even $100 worth of tokens at it, so I could enjoy a nice 4-day "mini-vacation".

How cheap "cheap" is, is indeed "in the eye of the beholder".

"Lack" isn't the right word. "Lacking" is more like it.

If there was a deep fundamental inability, we wouldn't see things like newer generations of LLMs consistently improving on ARC-AGI series (heavy spatial reasoning loading) and SimpleBench (a lot of commonsense + spatial reasoning components).

In a way, it's a surprise that LLMs, notoriously lacking any sort of embodied experience, can even get this close to human baselines on tasks like this.

My takeaway is that text is a far richer modality than anyone has expected - and that high end LLMs are often sharp and flexible enough to recognize their weak points and substitute their strengths. I.e. all the LLMs implementing A* to optimally solve pathfinding in ARC-AGI-3 tasks, often unprompted.

There might still be unrealized gains there from true depth-unbounded recurrence, or maybe from finding better ways to integrate modalities in training. But clearly, a "fundamental limit" it ain't.

Is there any proof that they are not good at special reasoning? Arc agi 1 and 2 are saturated.

I generally agree, though direction, intuition, and domain knowledge are still relevant. Your breadth-first-search framing feels right, but you still need a sense of which paths are worth following, and you need to know when to trust the results.

I’ve been doing more math as a hobby in the past few weeks — working on lesser-known conjectures and exploring proofs of hard theorems — than I could have managed over the previous several years. It’s an exciting time.

You've not seen how they react to noobs asking physics questions, I think.

Even when you've got an interesting idea, if you're an enthusiastic amateur who don't yet know enough to phrase the question right but does actually know the basics, they'll put you in the same category as the people who think healing crystals can power hyperspace telepathy with Anubis: "oh no not another one".

LLMs have infinite patience, but unfortunately come (came?) with too much sycophancy, giving even more people far too much confidence.

(within limits)

Uh.... Have you ever called customer service lately?

Do we employ mathematicians in customer service roles?

I think the big names behind the AI companies already have that problem solved. A lot of people probably won't like the solution very much though.

When machines are doing all the work - we no longer have to.

I can't stop wondering myself.... I'm writing some software with AI and wondering, why am I doing this? Will anyone need this? Will anyone have money to buy this?

Best I've come up with is we'll need to be adopted by technofeudlaist overlords to be our patrons like in the roman days

That view is incredibly reductionist. It really is an efficient encoding of how nature behaves. It might be a human construct, but given how best it allows to understand nature (through principles of physics), it is uncanny to be any different from the language of nature.

Reminds me of Wigner's Unreasonable effectiveness of mathematics in natural sciences [0].

[0]: https://en.wikipedia.org/wiki/The_Unreasonable_Effectiveness...

At a very high level mathematics is basically 100% text/symbolic rewriting. You start from some set of postulate assumed true and you do your thing to get a new different set of equivalent assertions in a form that is more useful.

I don’t know if LLMs will kill the working-mathematicians but at least seem like that it doesn’t seem absurd to imagine LLMs will be good at math…

The infinite monkey theorem assumes random distribution of symbols*.

Given the tokenizers have a vocabulary in the 10k-100k range, "a million attempts" will generally still only get the first token of the answer correct.

Even really rubbish models, e.g. talkie, the "what if we only use pre-1930s data to train a model?"** model, had to be almost all the way to the right answer to reach the really low HumanEval pass@100 score of ~0.04 (I'm only eyeballing the relevant chart).

* Actual monkeys not being like this is, while amusing, irrelevant

** https://talkie-lm.com/introducing-talkie

>Ever heard of the infinite monkey theorem?

Even if every atom in the universe were a supercomputer generating a trillion trillion random characters every second since the Big Bang, the chance of producing Hamlet would still be essentially zero.