I was reflecting on this on Saturday in an unformed way, trying to trace the lineage of a decision made at work.

The code change itself doesn't specifically matter. But suffice to say, it was about an AI feature in one of our products.

The code was stamped by Claude driven by a prompt. The prompt was for a ticket generated with the Atlassian AI integration. Atlassian had digested docs made with AI. The docs came from strategy memos I'm 90% sure were written entirely by Claude.

The strategy was chosen by management at the urging of exec leadership. The execs now communicate mostly via AI written memos. I do not know how they make decisions, but they reference tech influencers, market conditions, customer expectations.

This gave me pause. Who had actually made the decision then? Arguably there has been several layers of human review, but the actual source of the decision was hard to pin down.

We were not building the feature because we wanted it. We were building it because we thought other people expected it.

Perhaps reflecting on the state of the market, I thought, could indicate who was actually in control.

Where do investor and customer expectations come from in 2026? It is very murky, at least in tech. There appears to be hype. Some hype comes from true believers, some comes from cynics. But both respond to market incentives that reward bigger and bigger claims.

Where does the market's "action" come from? What is the driver?

Investors do not really seem to understand what the tech is or its limitations. Some are passive operators. Others are just responding to the overall froth and speculation in the market - which becomes a runaway feedback cycle.

This left me lost.

Nobody in this ecosystem, I thought, is actually in control here.

Nobody is actually orienting work and action to real, concrete goals. It's all based on speculation and anxiety about the future.

So it is not only that nobody understands what the code does. It is that we cannot, or at least I cannot, explain the motivation. There doesn't seem to "be" any form of "intention" in this environment.

It has all been hollowed out, replaced either be inscrutable machines, or inscrutable incentives.

Ironically it rather resembles the kind of "misaligned" superintelligence we are supposed to be avoiding.

In many (most perhaps, but hard to know personally) companies, most people don't really understand the market they are in, the competition, their own products, the customers etc well enough to actually make good decisions. They also aren't likely to be around (or held responsible) for decisions as they play out over years. So what has historically happened is that people use proxies for good decisions and understanding - which are clever sounding documents and presentations.

This is a long winded way of saying "people made up clever/sensible sounding stuff". Now it's easier to do it with AI so the problem is worse. However, I'm not sure what you were looking for was ever really there - the "inscrutable machines" and "inscrutable incentives" were always quite inscrutable.

From my experience in big and medium sized companies or even startups I would say that this was already the state of things before AI. Most leaders and top management were following internal committees, that were following consultants recommendations, that were themselves recommending what others were doing and was hype and/or following Gartner like "studies". The motivation was equally dubious, and many developers already had problem with management and product direction, but many didn’t bother to question it. AI just makes it obvious.

To put a finger on a word: David Hume’s “is vs ought” problem.

Data can describe to you what exists. But it can’t tell you what you value.

What you describe is people who can’t tell the difference, and who let the machine (data) make the value judgments.

For me, at this company, which is struggling to develop a new value proposition, everyone is unfortunately using AI in a blatant way. It’s even worse when executives and management pressure us to move fast. Presentations, documents, and mockups all use AI, rinse and repeat, feeding it context, but somehow, nothing is moving. It’s just staying static.

Most people are trend followers and what trends they see decides what trends they follow. It sounds like the algorithms that chose which influencers to promote made the decision, which was already somewhat true before the current AI trend. This happened when there was a big push from deterministic feeds to algorithmically ranked feeds by facebook and twitter. I guess an argument could be made that the people who chose what content to consume feed the algorithm but really there is a heavy hand on the scales that tilts the algorithm in favor of various things.

The word I have for this is “commitment laundering”. People pass around AI artifacts that nobody has necessarily read or considered, and the invented assumptions and tagalong commitments just keep piling up. They look like they’ve got real provenance but nobody can say what is being done or why.

> The execs now communicate mostly via AI written memos

It bothers me to no end when I get an AI written response, especially from the executive team or any one of my co-workers

Anecdotally, I know people in the industry who tell me their job is "so easy" now because all they need to do is be a meat proxy for claude and take home the paycheck.

Question from someone written in AI? Just answer it in AI and send it back. What was it actually about - who cares? Bug comes in? Post the jira link in claude and don't even bother prompting anything else. If something critically breaks - well, eh, we'll deal with it then. FIRE (early retirement), a prediction their layoff is inevitable, and investing aggressively so you can finally escape actually working are often invoked in the same breath. Everyone feels like they're just trying to punch the drywall and grab as much copper wire out of the walls as they can until they're finally let go and/or the whole company fails.

There's a great deal of nihilism and cynicism in the industry currently, and it feels like LLMs are just greatly enabling it. Where you would've done a halfassed job previously, you'd now do an unchecked AI job.

I see this is code in a smaller level. It lands on an alternative but in the end nobody actually chose it, not even by personal taste.

Yet everyone assumes that, like before, there must be a reason.

Worse, we can't tell apart real decisions choices from the dream-machine side effects.

The invisible hand of the market is in control, and we don't understand it either.

Ultimately competitive pressure is what drive decisions.

> Ironically it rather resembles the kind of "misaligned" superintelligence we are supposed to be avoiding.

yes, and it is the exact same system that is producing the "misaligned" superintelligence. Funny how that works, and begs the question: exactly how are you supposed to avoid building "misaligned" superintelligence?

At some point deferring all decisions to AI will be the competitive thing to do, regardless if its aligned or not.

> The code was stamped by Claude driven by a prompt. The prompt was for a ticket generated with the Atlassian AI integration. Atlassian had digested docs made with AI. The docs came from strategy memos I'm 90% sure were written entirely by Claude

Was there never a developer in the loop? I hope my org won't give up control of their business to an AI like this soon, sounds like a nightmare to figure out what's going on.

This resonates, but also, I'm not convinced anyone can truly understand the workings of any society, or market economy, or what have you.

Who's in control? Everyone is, to some extent. And no one is: when you're hungry for food, are "you" in control of that? You can consciously repress your impulses to go eat something, but your mind didn't create those impulses.

Human societies develop impulses and minds of their own, emerging (weakly) from the impulses and minds that comprise them, and they make decisions in mysterious ways.

Of course, it sure is nice when we can come up with a compelling story for the motivations behind something. Easier said than done…

Thank you for taking the time to write this. Unfortunately, there are still people on this forum who argue that a turtles-all-the-way-down approach will somehow work out in the end and that the best way to verify LLM work is to add another layer of AI agents, whereas the only way to stop it is to ensure that the entire thing is built on some human verified last line of defense. Unfortunately, the higher up you go the more insecure people get when you ask for their supporting sources. They'll try to hand wave it saying they researched it or got others to research it for them instead of owning up to deferring the decision to AI. You know whats even scarier? When a C-suite doesn't try to hide the fact that they had the AI make the decision. Then it's just time to leave, or alternatively time to turn your brain off and collect your pay check until the music stops.

>Perhaps reflecting on the state of the market, I thought, could indicate who was actually in control. Where do investor and customer expectations come from in 2026?

This looks like ill-fated Gartner driven development on crack. Where does customer expectations come from in 2026? How about from your customers?

If you're doing enterprise software and talk to your users, you'll end up learning that the actual users of your software don't use 80% of your features. You might even learn they don't use your software at all, and that the software was mandated top-down from the C-suites or that the execs in charge forgot what purpose your software served but are too insecure to question it, unless and until there is a sudden pressure to cut costs.

> Nobody is actually orienting work and action to real, concrete goals. It's all based on speculation and anxiety about the future.

the simulation has become simulacra! cosmic horrors abound.

Exactly the same issues we are facing right now. And, well why wouldn't it end up like this? The promises of going 'faster' while ignoring the organisational processes capability to handle it is - and continues to be - the greatest delusion of this new age. But, even in the face of all of this, we are seeing leaders asking for more. At some point the culture will collapse in on itself, and no one will understand anything anymore.

One of the more thoughtful observation's I've seen in this "space," one which offers an unsettling counterpoint to the claim in the original post that "AI Can't Drive Itself."

To your point, AI can drive itself; it's just that the fashion in which control occurs is distributed. Which is to say, the process (e.g. a feature being implemented, in some way, or at all) is not spontaneously occurring: it's emergent from the mesh of AI automation.

Niceties aside, it's pretty clear that humans are not in the loop in a meaningful way, much of the time. What emerges may hence not well be not aligned with business or technical needs. What it is aligned to may be impossible for we humans to discern.

Who controls the way a forest grows?

Ask a tree, get an answer, but don't forget, that's not the answer.

Very good post, thanks! You remind me of the countless times I've heard "we need to be AI native", "use AI", "pass it through AI for review".

AI has become the thing you do, and what you do it with, to achieve it. It's a self-fulfilling chicken that is an egg that is a chicken.

It's like old fashioned Wordpress, edited by someone directly in production, on a shared host that probably is infested with who knows what, with little to no oversight except inscrutable emails, which in the old days were like a CEO of a small business emailing 'hey, the thing we spoke about on the phone, is it coming?' and no actual document trail.

Except for the enterprise customer and at enterprise prices.

This is why we are building Origin (originhq.com). Having a record of every prompt, tool call and model response enterprise wide allows you to answer these kinds of questions.