Unfortunately not everything will cleanly show up in PnL, which is a function of the entire company’s processes.

If developers are shipping more code (and most of the enablement is in software right now) does that show up in top line? Maybe… if sales and marketing are also selling more, and product people are building the right thing, and…

It’s not a straight line from AI -> $$$

An agent found a SQL optimization that at least a month of eng effort hadn't found.

Agents accelerated rollout of a data model change by at least 2 months of eng effort.

Those unblocked a $100k/month infra downsizing. That 2 months pays for my team's AI spend for the year.

I know a very specific use case in management consulting where hours spent by a consultant for a repetitive task (mainly report writing and source data gathering) went from on average of 40 hours to 12 hours per project. It costs the firm roughly ~$150 in token spend to save those 28 hours. They used to be roughly $200/hr (so $5k~$6k).

That's about as hard of data as you can get.

I think part of the issue is-if AI makes you twice as productive at X, that doesn’t necessarily result in any increased revenue, because now your customer expects twice as much for the same price, and if you don’t deliver that one of your competitors will.

A new technology can deliver significant increases in real economic value in terms of utility per $ expended without significantly increasing revenue for the average firm.

I actually feel like it’s really premature for a relatively new technology to be showing profit improvements. We are talking on the order of a few years of investment and “figuring out” how to use it productively in business. Most new technologies take a decade or longer to really yield tangible value.

This is the first cycle in my (relatively) long career where the demand for return felt instant. The tools and techniques are almost comically immature and the average person interfacing with the tools is largely bewildered.

For word processing for instance people spent years upon years coming up to speed on wordstar, word perfect, word. Decades later they’re a common part of life and the tangible benefit over secretarial pools is obvious - there is no such thing as a secretarial pool.

AI it feels like almost a blink of an eye and the demand is “well, it’s been a few years, where’a the post scarcity society I was promised?”

Speaking annecdotally because im an engineer, not in sales:

Our company has secured projects and executed them to a standard which we would not have otherwise were it not for AI or hiring more people. Now, these are at least 3 year projects, so we are only seeing ~3 being AI accelerated.

For us, yes, it's a multiplier, directly translated to more $$.

With the advent of the internet and email, many people surely made similar observations. I wonder what the fate is of the companies who refused to accept these productivity enhancements.

Can’t share more than that publicly. Just that the time saved on this one thing alone is saving us way more money than the subscription per employee that is needed (~100$). I don’t doubt there are companies where this is not the case. Would like to share more details, but the rough math of doing something in a day vs several days + engineering salary have to do here.

Unfortunately we're at the mercy of Goodhart's law and in an interesting twist, AI is really good with things that have an immediate feedback mechanism, eg writing code that doesn't compile or pass tests, but you're trying to grade AI o something that doesn't have an immediate feedback mechanism. AI can slop out a PowerPoint deck in a fraction of the time it used to take a human by hand. The competent human then goes in and unslopifies it, but overall, still time is saved. Unfortunately, that employee is salaried and not hourly, so they can't click out early and even if they were, what's their incentive to clock out early if it means less money in their pocket? Or they can use the hours now free on something else. My gut, as an Internet random who's used AI is that it helps. I don't know how to measure it, just like how do you measure how much better it is to live in a clean house vs a cluttered one? How would you put that in a metric? If you can't put that into a metric, why bother cleaning your house? We clean it anyway, because of a certain je ne sais quoi.

It's probably just another Red Queen's Race. The benefits of LLMs won't show up in higher profits for most companies because their competitors are also using the same tools to move faster. You can't opt out of the race or else you'll be left behind.

Internally the hard data is there. On a budget of $2000 of API-priced tokens (Claude models + GPT models) we were able to migrate ~6 teams of infrastructure in 1 month, done by 1-2 people. Without AI it would've taken 10x longer wall clock time to do, and involved far more folks, and probably would have been much bumpier of a rollout. The big lift was mostly using AI for massive information gathering, digging through huge volumes of discussions, documents, commits, PRs, alerts, dashboards, etc pretty exhaustively for the past 5 years of history, to discover all the "chestertons fences" far more thoroughly than even the folks on the team who built it were able to. Then the 1-2 folks could go to each team with a super comprehensive "here is every need, every interaction, every discussion, everything you've ever done w/r/t this infeastructure, and here's how we're going to handle each of those with the migration we're going to do. We will do all the work for you, is that acceptable, yes or no?"

The teams throughly reviewed and brought up 2 overlooked cases, but mostly we all reminisced about things forgotten that AI discovery had turned up which would have been totally overlooked otherwise. Forgotten discussions, designs, outages, odd decisions, weird edge cases prepared for but never encountered, etc. Then they were migrated, it went smoothly, all to save a low seven-figures per year dollar amount.

The fundamental approach of exhaustively researching to that degree would never have been possible without AI. It's just not feasible to analyze 5 years of even one of those discussions sources (e.g. slack) to figure that out, let alone ALL of them. Now a team of ~1.5 engineers has tackled all that by leveraging AI.

The hard data is piling up in companies, but it'll take a while to get out there.

(Numbers and details lightly greeked in the above account)

[dead]

[flagged]

> And what revenue and/or profit increases can you cite publicly as a result of the use of AI in this work?

Nobody even tries to measure this. Their CEOs and board members have AI shares, and potential investors need to see your AI story to justify your valuation. It's a prison of lies.

Tons of unstructured data that needs to be cleaned up in so many different ways that are hard to predict. Knowledge distributed in the heads of greybeards, handwritten papers und excel, sharepoint and dozens other locations like ancient proprietary software. Which you have to repeatedly ask for and discover more piece by piece. Then clean up, discover issues, make the customer gather more data/correct it, etc.

I think a lot of “boilerplate” is just to align with a company’s way of doing things/ecosystem.

In a webpage the UI design, authentication, etc. are all “boilerplate” if one just wants to do “X” instead of worry about infrastructure.

You’re not meant to ask that question.

Probably a controversial take - but in the pursuit of elegance. 'Uhh, so I need to hook this connector up to that receiver using this adapter and then uh, I pass it to the connector manager which sends it to the scheduler that... @#%! FML.' And then it's all so polymorphicized that going through the debugger you spend an eternity just trying to figure out what the types even mean and why/where they came from.

When you look at old-school code that just hacked things together, it was ugly, but also very easy to in turn further hack. There was no elegance or grand base types with 80 levels of inheritance to enable clean extension at every possible juncture of types. And no spending forever to find out the exact function you want to be able to override, only to discover that it's outside your modifiable codebase and wasn't marked virtual. Everything just worked.

https://en.wikipedia.org/wiki/OODA_loop

speed of iteration is more important than quality of iteration; high iteration speed lowers the cost of failure and compounds improvements

Not all experiments work out. If we find the right approach faster we can finish the project faster. Some projects are mostly fixed price or offer bonuses for reaching milestones in a certain time. Meaning we more often get a bonus and can take more projects. Also we can take projects with tighter deadlines that we would need to decline otherwise.

Yep. Hard to see why this increases revenue or profitability

The two needn't go hand in hand. But here we are.

There's a difference between the cost of coding and the cost of delivering and maintaining software? AI has demonstrated that these two things are almost entirely unrelated.