What I'm pushing back on is, apparently, that some people genuinely believe these LLM-based computer programs are human-like intelligences.

In reality, they're more like very good search engines that output relevant snippets of text. If you run them in a loop (feeding them their output as input) you can make them return even better search results.

The software developers who created these systems used sexy words like "reasoning" and "thinking" to describe this search process. They used words like these because they're trying to make money and it sounds cool, not because they've actually re-created human cognition.

That's the thing - they are more human-like intelligence than search engines. I'm gonna push back on your push-back here strongly. I'm not saying they are intelligent - but they're much more like human-like intelligences than like any kind of classical software systems, which is why it makes more sense to talk and think about them in these terms than as software system.

To do the opposite invites confused thinking like considering "lethal trifecta" a solvable programming problem.

some people genuinely believe these LLM-based computer programs are human-like intelligences

"If LLMs were human-like intelligences they would do X, but they don't". What is X?

In reality, they're more like very good search engines that output relevant snippets of text.

What are the "relevant snippets" that contained the solutions for the unit distance and Jacobian conjectures?

It seems to me that they used words like these because the LLM-based computer programs they sell can solve problems which humans apply reasoning and thinking to solve. Why do you think it's more than that?