> So long as there is competition it’ll be available at marginal cost.

Most things are not perfect competition, so you get MR=MC not P=MC.

We're talking about massive capital costs. Another name for massive capital costs are "barriers to entry".

Granted that capital costs are a barrier to entry and that barriers to entry leads to non-perfect competition, but the exploitability is limited in the case of LLMs because they exist on a sub-linear utility scale. In LLMs 2x the price is not 2x as useful, this means a new entrant can enter the lower end of the market and work their way up. The only way to prevent that is for the incumbent to keep costs as close to marginal as possible.

There is a natural monopoly aspect given the ability to train and data mine on private usage data but in general improvements in the algorithms and training seem to be dominating advancements. Microsoft's search engine Bing paid an absolute fortune for access to usage data and they were unable to capitalize on it. LLMs have the unusual property that a lot of value can be extracted out of fine tuning for a specialized purposes which opens the door to a million little niches providing fertile ground for future competitors. This is one area where being a fast follower makes a lot of sense.

▲

mlyle 2 days ago | parent [-]

Almost anything has a utility scale which is diminishing. But we still see MR=MC pricing in industries with barriers to entry (IPR, capital costs). TSMC and Mercedes don't price cheap to avoid giving others a toehold.

> There is a natural monopoly aspect given the ability to train and data mine on private usage data but in general improvements in the algorithms and training seem to be dominating advancements.

There's pretty big economies of scale with inference-- the magic of how to route correctly with experts to conduct batching while keeping latency low. It's an expensive technology to create, and there's a large minimum scale where it works well.

▲

cjbgkagh a day ago | parent [-]

I’m unconvinced that the lessons learned from scaling will constitute much of a moat. There is certainly an incentive for incumbents to give such an impression.

▲

mlyle 17 hours ago | parent [-]

Probably not. But we don't tend to see P=MC where there's any differentiation or barrier to entry, and I do not believe that AI is fully commoditized or will be soon.

▲

cjbgkagh 14 hours ago | parent [-]

Perhaps my 'marginal cost' should have been taken to be 'near marginal cost' and as such I don't believe it'll be fully commoditized either, just mostly commoditized in that it becomes impossible to extract meaningful monopolistic rents. Similarly I don't believe in perfect market efficiency so nothing would be exactly at marginal cost. At the moment we have investment subsidizing use so often these APIs are available at below marginal cost - the old adage 'we lose money on every sale but make up for it in volume'. I'm not confident that these investors on average will be able to make their money back plus a required rate of return and for many it's probably not the primary point of investing in this industry. If you have a $1B dollars to spare you too can light it on fire...

▲

mlyle 14 hours ago | parent [-]

I don't believe the APIs are actually under marginal cost. Inference is cheap, but people are using a lot of it.

The problem is, the arguments you're making could be used for almost any industry, including ones that we've seen sustained excess profits.

	▲	cjbgkagh 13 hours ago \| parent [-]
		It depends on how you define sustained and define excess profits. Being an unusually effective and well managed company can certainly yield sustained above average returns but that hardly means that is a meaningful barrier to entry - the question at hand. Ozempic is marked up 200x (20,000%) because they're able to extract monopolistic rents, that's a completely different ballpark of 2x or even 5x markups. In a fully commoditized industry participants yield average ROIs, in a mature market it's industry dependent but I would consider ranges from 10% to 100% above average ROIs to be reasonably normal. It's when things get to 10x to > 100x that I would consider to be able to extract monopolistic rents. I know I'm mixing up profits (ROIs) with markups on COGs with the issue being that companies extracting monopolistic rents tend to obscure the fact with padded expenses and the financial details needed to calculate Total Production Cost per Unit are generally not available.