That seems reasonable compared to an actual developer (depending on your region), but I had hoped for these models to make simple tasks like this fast and cheap so those developers can focus on the difficult stuff.

That's not really an error, that's a fundamental feature of unit economics.

Fixed costs can't be rolled into the unit economics because the divisor is continually growing. The marginal costs of each incremental token/query don't depend on the training cost.

It would be really hard to properly account for the training, since that won't scale with more generation.

The training is already done when you make a generative query. No matter how many consumers there are, the cost for training is fixed.

The challenge with no longer developing new models is making sure your model is up to date which as of today requires an entire training run. Maybe they can do that less or they’ll come up with a way to update a model after it’s trained. Maybe we’ll move onto something other than LLMs

The training cost is a sunk cost for the current LLM, and unknown for the next-generation LLM. Seems like it would be useful information but doesn't go here?

The AI training data sets are also expensive... The cost is especially hard to estimate for data sets that are internal to businesses like Google. Especially if the model needs to be refreshed to deal with recent data.

I presume historical internal datasets remain high value, since they might be cleaner (no slop) or maybe unavailable (copyright takedowns) and companies are getting better at hiding their data from spidering.

Comparing it to running a refrigerator or the dishwasher is very relatable, as most people have at least one refrigerator without a second thought.

This is for someone using a lot of LLM tokens relative to the average customer of these companies.

No! What a disaster. Enforce the costs of externalities as close to the source as possible. If electricity costs more money charge more money for electricity. Don’t add some “regulation” to force end users to pay more based on your estimate of how much more you think electricity should cost.

> As long as it's unaccounted for by users it's at best anexternality.

Why is it an externality? Anthropic (or other model provider) pays the electricity cost, then it's passed along in the subscription or API bill. The direct cost of the energy is fully internalized in the price.

That's hardly unique to data centers.

I'm all for regulation that makes businesses pay for their externalities - I'd argue that's a key economic role that a government should play.

I mean if we really cared about this one bit we'd stop making a car based society in the US and save far more energy and pollution. That's not politically expedient and there are powerful vested interests in ensuring it doesn't happen.

That's why I think most of this data center energy use, especially over longer terms is a joke. Data center can pretty easily run on solar and wind energy if we spend even a small amount to political capital to make it happen.

I don't see how this follows. Data center operators buy energy and this is almost their only operating expense. Their products are priced to reflect this. The fact that basic AI features are free reflects the fact that they use almost no energy.

To add a bit more to what @scottcha is saying: overall GPU load has a fairly significant impact on the energy per result. Energy per result is inversely related, since the idle TDP of these servers is significant the more the energy gets spread the more efficient the system becomes. I imagine Anthropic is able to harness that efficiency since I imagine their servers are far from idle :)

I think this number is derived from the water used to create all the goods they consume. Let's say every American eats 1lb of beef per day, and nothing else, to create a simple model. Then every American has a water footprint of about 1,850 gallons.

[0] https://watercalculator.org/news/articles/beef-king-big-wate...

It does seem high, most of the estimates I find are around half that.

What number would you provide for that?

The training cost for a model is constant. The more individual use that model gets the lower the training-cost-per-inference-query gets, since that one-time training cost is shared across every inference prompt.

It is true that there are always more training runs going, and I don't think we'll ever find out how much energy was spent on experimental or failed training runs.

You underestimate the amount of inference and very much overestimate what training is.

Training is more or less the same as doing inference on an input token twice (forward and backward pass). But because its offline and predictable it can be done fully batched with very high utilization (efficiently).

Training is guestimate maybe 100 trillion total tokens but these guys apparently do inference on the quadrillion token monthly scales.

Training is pretty much irrelevant in the scheme of global energy use. The global airline industry uses the energy needed to train a frontier model, every three minutes, and unlike AI training the energy for air travel is 100% straight-into-your-lungs fossil carbon.

I'm not convinced that LLM training is at such a high energy use that it really matters in the big picture. You can train a (terrible) LLM on a laptop[1], and frankly that's less energy efficient than just training it on a rented cloud GPU.

Most of the innovation happening today is in post-training rather than pre-training, which is good for people concerned with energy use because post-training is relatively cheap (I was able to post-train a ~2b model in less than 6 hours on a rented cluster[2]).

[1]: https://github.com/lino-levan/wubus-1 [2]: https://huggingface.co/lino-levan/qwen3-1.7b-smoltalk