One idea I've heard is prototype-first design reviews. If the cost of code genuinely trends to zero, there's no reason why most technical disagreements about product functionality couldn't come with prototypes to illustrate each side of the debate. Today, that's not always practical between token costs and usage limits.

nobody know how to measure software productivity + ai is supposed to mean productivity goes up = more ai means more productivity

As best as I can tell, that's the thinking. It's one number, it's very easy to find and manage, and there is a belief that it directly measures productivity.

I disagree that it does; seems to me the throughput of useful features is a better measure, but I'm not in the drivers seat on this one

Yep - it’s impossible to separate experimental tokens vs value creating ones.

Ultimately the performance will be assessed via the income statement and cash flows of customers of the model producers.

Frankly in the window pre-IPO it’s in the best interests of OAI et al to show a line going to the top-right in relation to tokens, in their prospectus. What does that mean?

Strategic manipulation.

If they've already built themselves a loyal customer base (which is usually the point of fighting a price war) and the customers are happy with the technology they have, then if funding is tight and turning a profit is more important why wouldn't they pivot to optimizing inference by stopping further training, freezing the model versions, burning the weights into silicon and building better caching strategies and improving harnesses and tools that lower their cost and increase their margin?

If all they do is hike prices then they'll lose customers to competitors who don't or who find a way to serve a similar model cheaper.

The demand isn't going to go away purely through higher prices. Once people know something is possible they will demand it whether supply is constrained or not. That's a huge bounty for anyone who can figure out how to service that demand.

There is no evidence that labs are losing money on inference subscriptions. The labs have massive fixed costs, but as long as inference spend is higher than the datacenters they use for inference cost all they need to do to become profitable is scale up. Right now software engineers are basically the only ones actually paying for inference, the labs just need to create coding assistants for everything that are good enough that every white collar worker in the country(world?) is paying a $1000/yr subscription. Certainly theres a lot of risk, will models become commoditized and everyone switches to open models? can they actually get non software engineers to pay for inference in mass? But its not like theres no path

Only if nothing changes. Right now, people are running agent frameworks like OpenClaw on their own hardware or a VPS and the frameworks are often single person projects. This results in all sorts of problems but you can pick an easy solution from history which is to create a walled garden service for running these agents where you can provide security and standardization. If that platform also allows trusted services to integrate then they can provide end to end security guarantees. They also benefit from improvements to the models themselves making them more difficult to subvert. Creating something that is secure enough for the average person to entrust their credit card to is not an impossible task.

>Everybody is using it everywhere.

No one in our Auto shop is using AI. One of the new diagnostic tools was demo'd with AI, and none of us were having it. It's about as accurate as Googling your symptoms.

My mother had an AI powered lung scan that came back with Stage 4 Cancer. The Oncologist got called in (for a fee!) to tell us it was just early stage COPD.

In my experience people vastly overestimate the competence of doctors. Getting medical advice from LLMs could be life saving.

Personally I experienced this when a specialized doctor believed a drug interaction to be the opposite, thinking A hinders the absorption of B, when actually it hinders the clearance, tripling concentration of B.

Without AI, I would have been clueless about this and could not have spotted the mistake. I don't know if it would truly have been critical, but it did shake my confidence in doctors.

> getting medical advice

Id be careful stating this is an inappropriate use of LLMs. Im semi tapped in to the medical literature community and there is a lot of serious discussion and research going into the usage of LLMs for medical advice and most of it is showing that LLMs are barely worse than doctors, and much much cheaper/more convenient. They definitely arent ready to completely replace doctors, but it seems they can provide competent medical advice in a pinch. Look out for the literature on this in the coming year, its only the last few months that researchers seem to be taking LLMs seriously.

Right, but that just proves there's tons of pent-up demand waiting in the wings as token prices fall.