One thing about these numbers that's absolutely shocking to me is how low the energy use is:

> That model’s usage was well within our budget ($68, about 4kWh of energy use / 365 grams of carbon emissions).

The energy cost is literally 1% of the total cost. For context, 4kWh of energy would drive you about 15 miles in an EV, about half of the average person's daily driving miles. It's boiling 10 gallons of water.

With the talk of AI data centers' impact on the world, you'd think this would be 10x to 100x the amount of energy in order to get the effects they're using here.

My takeaway: the AI data center buildout is an overbuild probably at least as large as the fiber buildout that left us with so much dark fiber. If not even bigger. The only thing that will save the economy is the inability of NVIDIA and chip fabs to produce enough chips to match the buildout planned.

▲

ThibWeb 3 hours ago | parent | next [-]

I agree but it does worry me how fast my usage is increasing. Two months ago I was using 10x less tokens and probably not much more than 5kWh on inference. This month about 30kWh on inference. If it becomes more affordable, is there going to be another jump? Not quite sure

▲

CuriouslyC an hour ago | parent [-]

At some point the agents will be good enough that you can tell them "here's $100, go make me money" and they will, maybe not a lot and not all the time, but the EV given the cost of inference will be positive.

▲

MisterMunchkin 30 minutes ago | parent | next [-]

If I’m the AI company why would I let you do that, when I could just do it myself and get all of the money?

▲

CuriouslyC 21 minutes ago | parent [-]

Regulation. If it got proven to the point that it was scaled out en mass, it would get hit hard and fast.

	▲	monkpit 7 minutes ago \| parent [-]
		??? Then why would any AI company exist when they could just use their money to buy tokens from another AI company and make money for zero effort? There would be no incentive to be a provider. Not to mention inflation would grow to match or outpace the rate you could earn on these guaranteed AI gains.

▲

mannanj an hour ago | parent | prev | next [-]

Those rewards seem like they would be captured by the providers and AI companies though who would use them first to make themselves money. You would be left with whatever they didn’t pursue with their first movers advantage.

▲

pphysch an hour ago | parent | prev [-]

Good enough at what, fraud? Robotic Ponzi schemes would be an easy way to generate "income".

	▲	CuriouslyC an hour ago \| parent [-]
		More likely they bot will do research on how to make make money and do experiments given the resources available to it.

▲

timmmmmmay an hour ago | parent | prev | next [-]

"the talk of AI data centers' impact on the world" has been wildly exaggerated and you can see here that this is the least impact of any major new technology in the history of industrialization

▲

smartmic 2 hours ago | parent | prev | next [-]

Related to this: see the difference between China and US here: https://aidatacenterindex.com/compare/united-states-vs-china...

I have the feeling China is somehow ahead when it comes to energy (and cost) efficiency for AI usage. After all, the two are in a direct competition, and this difference is significant. Or is the "hyper" scaling of energy hungry datacenters in US part of a bubble?

▲

killingtime74 2 hours ago | parent | prev | next [-]

Your only talking about variable direct energy. Does it take into account the entire lifecycle, building the data centre, running the cooling, building the chips, the % the chips are not utilised.

	▲	epistasis an hour ago \| parent \| next [-]
		> Does it take into account the entire lifecycle, building the data centre My entire point is that 99% of the dollar cost of running these models goes to things other than the GPU power. The capex cost to building cost to GPU cost to storage/networking/chasses/wiring plus the other operation costs dwarf the electricity. Even the other electricity costs, lets say double it for all the supporting compute, plus another 25% for a 1.25 PUE, and you're at 2.5% of all-in cost of running these models is from electricity. The non-electricity costs are massive and the constraints on fabs, etc. will drive the amount of the AI build far more than energy availability.
	▲	ThibWeb 2 hours ago \| parent \| prev [-]
		No those numbers are GPU only. See https://cleerdash.sustainableaigroup.com/ for a fuller model

▲

jjcm an hour ago | parent | prev | next [-]

> AI datacenter buildout is an overbuild

a.) we’re supply constrained

b.) only 3% of households pay for AI

Inference amounts will continue to grow heavily.

▲

api 10 minutes ago | parent | prev | next [-]

Yeah the whole data center environmental panic seemed astroturfed to me. I took a look at the numbers and it’s not that bad. If you telework one day instead of commuting you make up for over a week of heavy AI use, and the water use is on par with an average golf course.

There are noise issues in some places. But the panic is excessive. Like nuts.

Maybe environmental panics are to the left what moral panics about stuff like trans people are to the right.

▲

eli 2 hours ago | parent | prev [-]

It would be a pretty big deal if the average person's car became 50% less energy efficient, no?