Anecdotally, does this match anyone's actual experience?

Everyone I know who is working closely with LLMs is doing some of the most acute critical thinking of their lives these days.

I'm very skeptical of these kinds of takes; I'm reminded that Socrates railed against books on the basis that they were going to rot the brains of the youth.

I have seen a significant number of excellent engineers just seemingly turn their brain off entirely and resort to just spitting out "but AI said <blah>" or "AI reported <foo> which I found surprising but I didn't analyze or dig into that result whatsoever". So anecdotally, yes, this absolutely matches my experience.

Socrates was worried that books would prevent people from developing their memories, and his prediction was correct. How many people you know use mnemonic devices to remember entire Greek epics (or something equivalent) by heart? Probably very few.

> Anecdotally, does this match anyone's actual experience?

Yes. In contrast to you, I really think many of my coworkers seem to get dumber by the day. Not all of them yet, fortunately, but many.

I always compare it to satnav: how many people bother checking maps and figuring out the route to go anywhere nowadays, just punch it into your satnav and hit "Go!". It'll get you there, but when there are unexpected things some people do panic, or even turn into roads that are actually rivers, etc.

Or course "everyone you know" might be doing something different, but AI is offering the same delegation possibilities for critical thinking. Do you think there won't be people who'll do the equivalent of driving into the river when using AI?

One must distinguish people whose lives were spent thinking critically, and then who subsequently adopted LLMs, and those who will grow up with the technology and will never have to think critically to get a program to run. Or do whatever.

I think algebra too - it requires less understandng than geometry. You operate at a higher level, and in a sense, don't really know what you're doing.

Geometry existed for centuries before Euclid's formalization. Similar for algebra before Al'Khwarizmi. People were using logic before Boole.

"External Intelligence" is not new. But we are still in the informal stage of LLM use.

Perhaps more skeptical on certain tasks, but in general; no. Instead of reasoning with colleagues it's more common to hear "Claude said I found the best solution to task X" without questioning that. Anecdotally, the dopamine rush of validation is real.

There's people who use it as a tool, and there's people who use it as a brain surrogate. Most of the people using it as the latter aren't people who were deep thinkers to begin with.

Looks like the paper is from 2025?

I think the problem with these sorts of studies is that the industry is moving so fast, and by the time your paper gets peer reviewed and published it is no longer relevant.

In this paper, the participants are using mostly ChatGPT - well, there's a HUGE difference in free chatGPT, gpt 6, gpt 4o, gpt 3.5?

What might have been the case for say, GPT 4, is probably no longer the case with GPT 6.

My experience with Google's gemini right now makes this obvious, as I run out of Gemini 4 Astra in a conversation, and then it switches me to Gemini flash lite.

On a micro scale, yes I see this regularly at work: some devs blindly put up ai generated PRs that are fundamentally flawed or don’t actually complete the work in the ticket. It’s often trivial to check this. Similarly some tickets are written by ai that make incorrect assumptions about our product, resulting in nonsensical requirements.

You could argue that these are just examples of laziness, but I think these are both tasks that require critical thinking, and it’s easier to hand them off to an llm that at least purports to do this for you.

I’d argue that both setting requirements, and self code review are both things that need careful attention. (What _exactly_ do you need doing? Does your work correctly do the thing that needs doing?).

I have no fundamental issue with using ai, but there’s no doubt that applying critical thinking to both these tasks directly, and to the broader question of “should I use ai for this job” should result in a different outcome.

Personally I would also not agree with your take that working with llms has allowed me to do “acute critical thinking”. It’s sped up my development time, sure, but I would argue that my evaluation of, say a new sdk now using Claude is far less detailed than it would’ve been in the past when I did it by reading docs. As a side note, I’ve been rereading Douglas Hofstadter’s GEB recently in paperback, and I’ve found a lot of joy in wrestling with paragraphs and writing notes in the margins. This is way more focussed attention than I really give anything when working with llms.

> Everyone I know who is working closely with LLMs is doing some of the most acute critical thinking of their lives these days.

According to them.