These are all important points and I love the analogy. But there is an even bigger issue with having LLMs write for you:

Writing is thinking. Thinking and deciding. There have been many times when I start out writing something substantial - could be an email, a blog post, a software design document, anything - when my own views substantially changed during the writing process. Writing forces you to serialize your thoughts - and you can't always trust the gestalt.

Reviewing gives you the chance to ensure the arguments connect solidly, that references are accurate (even informal references) and gives you the time to consider counter-arguments you aren't addressing.

None of this matters much on LinkedIn, but it matters a lot in our work. You cannot outsource your understanding to AI. They are powerful tools but they do not have any human understanding - that isn't their optimization target.

(I work at Anthropic) I agree. I use an LLM to write my code, but I do all of my writing by hand, since it helps me think.

It reminds me of the transition over the last year from AI-assisted coding to AI doing all the coding. At first the code output wasn't good enough, and humans read and iterated on the code all day, so the details of the source code mattered. Now, the code is largely high quality and it meets a large set of guardrails we've set up over the years (linters, typecheckers, security checks, LLM-assisted code quality checkers), and it's just Claude working on the code, so the details matter less and engineers think a level or two up (machine code < assembly/bytecode < source code < conversation with agent < artifact with high level design).

I wonder if long form writing will go the way of code. You and the AI agree on an outline or other high level representation, then the LLM expands it into a document. But writing and coding are different enough in a number of ways that this is far from inevitable.

> Writing is thinking. Thinking and deciding. There have been many times when I start out writing something substantial - could be an email, a blog post, a software design document, anything - when my own views substantially changed during the writing process. Writing forces you to serialize your thoughts - and you can't always trust the gestalt.

I don't disagree, but I think it's often not appreciated how much there's other work to writing too.

The biggest one is that you have to communicate non-interactively to an unknown audience. Having to (literally) put it in someone else's assumed terms does help giving different perspectives into the matter, but doesn't necessarily help one's own thinking that much. Instead you have to do some of the reader's thinking for them.

You also have to spend time on textual matters like grammar and style and a lot of "unspoken rules", which aren't really about linearizing your thinking about the contents.

Not all writing is thinking and not all thinking is writing.

I've thought exactly that "writing is thinking" before as a reason to not let a LLM write for me.

Then again, I've seen a counterargument [1] by someone who clearly heavily uses LLMs for writing (going by both their LLMy writing style and their own admission). The person I'm citing describes a process where they get a LLM to write something, they check over it and provide feedback to the LLM, the LLM rewrites, and the process repeats iteratively. So clearly he is putting thought into the process.

I think there is something valuable missing, even if it's hard to clearly express. I'll try. The threshold for what I'm willing to accept if I'm simply approving something is likely different from what I'll get if I write something myself, for instance. Saying "LGTM" is too tempting. It seems to me like he's outsourcing his selection of topics to cover as well. If you're not thinking yourself about what to cover then it would be very easy to miss a critical subject. There also an asymmetry between checking and generating something with constraints placed on it. Checks can't catch everything, and a constrained generating process can reduce the amount that needs to be checked, avoid issues that can't be checked so easily, and focus your attention on areas that you know historically have had issues with this generating process. I've thought about this quite a bit in terms of whether to write new code or use an existing library. Sometimes "the devil you know" (my code) is better than an existing library simply because I understand its flaws better.

[1] https://www.nature.com/articles/d44148-026-00236-3

I always found writing to be an unnecessarily arduous tool for communication, so I never do it until I've overthought what I want to express and approximately how. At that point I flesh out the skeleton of the message, and pile on words and structure and references and other rhetorical tricks until the packaging feels sufficient. Rarely, if ever, have I experienced writing things down altering my perception of the strength or weaknesses of the chosen arguments, or revealing new ones. That is to say, I really don't vibe with the concept of writing being thinking, rather it's a waste of humanity's resources to push it as The Tool.

That's an important point. It's the core of your life, thinking, balancing, exploring, improving. I don't believe adjusting something large you didn't do will ever benefit people.

Related, one of my favorite quotes by the computer scientist Leslie Lamport is this one:

"If you’re thinking without writing, you only think you’re thinking."

I have found this extremely relevant as a (primarily) non-verbal thinker.

I don't, generally, think in words, more in - I guess I would call it something like meta-shapes? A sense of a shape but not things I can exactly visualise.

(You might be surprised to read this and then hear I have an English degree. Surely I thought about Shakespeare in words?! Nope. Shapes, movement, structures)

For me, having to write is critical because it is the only way I practice serialising my thoughts in a way other people can understand.

If I do not then I get very "deep" into my own way of sensing ideas and it's difficult to dig myself back out.

This might also be why I have never been very enchanted by LLMs? They only seem to "think" verbally. So it is always a translation effort for me.

I never can really enter any "flow" state with an LLM. My intuition is that highly verbal thinkers can enter flow with LLMs very easily

> Writing is thinking.

This way of phrasing it is needlessly confusing. Writing is a kind of thinking--one of many--but it's not equivalent to thinking.

Dialectical thinking, for example, produces similar results.

> You cannot outsource your understanding to AI. They are powerful tools but they do not have any human understanding - that isn't their optimization target.

Understanding is the bottleneck; the way they speed things up is by letting me outsource understanding, and get back a summary. The entire advantage to AI is that it lets me skip understanding the problem, and just get a working solution.

On the other hand there is so much things that don’t require hard thinking. An example is I got an email the other day from a supplier asking if they can turn off their old email. It didn’t require ”thinking”. All I had to communicate was ”turn off graphql but don’t touch restapi”. Instead of sending off such a short and maybe unclear reply I had AI type up a concise and clear reply with exactly what can be turned off and what must be left on. Could I have done it on myself? Sure but would require more work than just a quick prompt and copy paste.

writing also implicitly has your review baked in. hand written code is "reviewed by construction", if you will.

I completely agree. And I'd say that at least in my case, it applies to both writing prose and code

This puts into words something I have been thinking about but haven't been able to articulate when people ask me if I'm using AI heavily. Usually when I write a report, I start out with a question and try to find some hints that will give me hypotheses that I can then test. I can't just prompt "Write me a report". The process of coming up with the information in the prompt is best done by writing the friendly report.

  They are powerful tools but they do not have any human understanding - that isn't their optimization target.

A) we only know of one species capable of metacognitive understanding,

B) we already tried that in the 1970s, and it was good work but often evolved into what we'd call boring ol' computing rather than AI, and

C) an alien mind wouldn't be a very good agent, for a ton of reasons relating to affect, conversational rythyms, cultural understanding, etc.

The trick is to make something that acts like a human but with the affordances of a computer (e.g. scalibility, symbolic certainty), without making it so human that it takes issue with its existential reality and/or use of its labor...

Writing can be thinking. There's a huge presumption that if someone is banging away on a keyboard they're doing work because you can hear and see them doing stuff. But that's the whole plot of The Shining -- Jack despite all his writing wasn't thinking at all.

Likewise using AI can be thoughtless, but it doesn't have to be. I don't see why a valid creation process can't be like this Simpson's meme[1], where you start with a rough object and then cut away and refine until it's done. I don't see it as lacking merit or requiring less thinking compared to starting from a blank canvas and adding more until it's done.

And either way at the end of the day the writing artifact stands on its own. It's either good or bad, taste permitting, and can be evaluated for what it is.

[1] https://media.licdn.com/dms/image/v2/D4D22AQFoqRgMxteTNg/fee...