> Here’s the thing: most of our habits at work still assume that producing something means you have to understand it.

This has only ever been true in a relative sense. Until recently, one was expected to understand the software-level code they wrote. But not the machine code, or the computer internals at a deep level. You trust that those layers work. At some point in the past you couldn’t just trust that. At some point in the near future, you will just trust that the software code works. Instead you’ll just debate the spec (the prompt). Your thinking can move one level up the system hierarchy. Everyone’s can. We’ve done it before, we’ll do it again, and this won’t even be the last time it happens. Let go of the anxiety and let it happen.

You trust machine code because it was produced by a deterministic system that was written and tested thoroughly by other engineers.

AI systems are not the same. You can't guarantee deterministic output.

Also, code is still the best language we have to convey our intentions as engineers into function. Natural language is too imprecise and ambiguous.

Understanding the code allows us to understand the intention behind the code and identify bugs, plan and architect better. Inspecting prompts gives you a blinkered view of the system meaning you're more likely to make incorrect assumptions leading to serious bugs.

You trust that those layers work

Unlike AI, those layers were traditionally compiled using deterministic tools. And generally came with a suite of other tools to trace and debug them. I don't trust AI in the same way at all.

>This has only ever been true in a relative sense. Until recently, one was expected to understand the software-level code they wrote. But not the machine code, or the computer internals at a deep level.

Just because something wasn't required at all levels or 100% it doesn't mean it wasn't required or it wasn't important though.

I trusted that the machine code worked because it was a mostly mechanical transformation, with little to zero non-determinism involved, and the design decisions for the high level code were still mine and understood. Not the same with an LLM, even when see just as an added level of abstraction.

Given some LLM produced code, I also trust the compiler will produce working machine code matching it. Doesn't mean I trust the LLM code in the same way.

Then put your money where the mouth is and start committing prompts and having your LLM “compile” them with each production release.

I don't think that's the problem the source article is describing. I've seen a number of instances now where people produce work product they don't understand at any level. "Hey Claude, design this feature and send it out for review, thx!" will often seem to work if you don't know any better.

I'd argue the trust was always relative. Witness what happens when you are hired to untangle the buggy mess that was left by the previous "senior" engineer. Sure there was trust when they were in charge, but based on the code they wrote, they didn't actually understand anything they were doing.

Mostly agreed. In a previous life, I used to write code, then got Peter Principled upwards to the point where I "trusted" the engineers under me to do their job. Did I understand all the code my team wrote? Of course not. Even if I wanted to review all the code written by 20+ developers to the point where I could stay I understood it, there simply wasn't enough hours in the day to do so.

I figure what is happening is (human) software engineering as a discipline, at least on large scale projects, is rapidly and irreversibly mutating to software project management. But instead of managing teams of people, you're managing teams of agents. Where I disagree with your comment is in the assertion that everyone "can move one level up the system hierarchy." Some folks can't, so they'll stagnate. Welcome to the Peter Principle World.

because we could trust the layer below. Now we can't