Reposting a comment (and one of the replies) since it's relevant:

---

It occurred to me on my walk today that a program is not the only output of programming. The other, arguably far more important output, is the programmer.

The mental model that you, the programmer, build by writing the program.

And -- here's the million dollar question -- can we get away with removing our hands from the equation? You may know that knowledge lives deeper than "thought-level" -- much of it lives in muscle memory. You can't glance at a paragraph of a textbook, say "yeah that makes sense" and expect to do well on the exam. You need to be able to produce it.

(Many of you will remember the experience of having forgotten a phone number, i.e. not being able to speak or write it, but finding that you are able to punch it into the dialpad, because the muscle memory was still there!)

The recent trend is to increase the output called programs, but decrease the output called programmers. That doesn't exactly bode well.

See also: Preventing the Collapse of Civilization / Jonathan Blow (Thekla, Inc)

https://www.youtube.com/watch?v=ZSRHeXYDLko

---

Munksgaard 1 day ago:

Peter Naur had that realization back in 1985: https://pages.cs.wisc.edu/~remzi/Naur.pdf

I haven't thought about it this way but I have been feeling that something is off. I think you got it right. There is a HUGE difference between "I understand the concept" and "I can write down the concept on paper".

Everyone fools themselves into thinking that they understand but the illusion immediately falls apart when they try to write it down. The problem with LLM is that it is actually able to produce something that "works". But more often that not, what it produces is usually beyond what the author actually understand.

Arguably, one could ask "Why does it matter?" If there are enough tests and monitoring to capture the behavior of the program, who cares how it is implemented? To me this is extremely disappointing. I have always wanted to write software that lasts for a century but if software becomes a commodity, I see software quality mattering less and less.

If something is broken, just ask the LLM to patch it. Unlike a human, there is no limit to this. The LLM will happily fix that 50 years old Fortran code that no one understand. There is a lot less pressure to rethink at the fundamental principles.

Update: I fleshed this out into an article:

The Deliverable is You! Programming as Theory Building

https://nekolucifer.substack.com/p/the-deliverable-is-you-pr...

The open question is to what degree the actual typing is necessary. I see it as proof that the mental model is sound.

The thinking produces the mental model, the mental model produces the program. In the absence of that final step, what validates the model? (Complaints from users and colleagues? ;)

Another great thread to pull on: the overlap between "moving up levels of abstraction", i.e. AI assisted programming merely continues and accelerates a trend that has been there for the entire history of programming.