I did a little investigation. Turns out that GPT-4's training consumes as much energy as 300 cars in their lifetime, which comes about 50 GWh. Not really that much, could be just families on a short street burning that kind of energy. As for inference, GPT-4 usage for an hour consumes less than watching Netflix for an hour.

If you compare datacenter energy usage to the rest, it amounts to 5%. Making great economies on LLMs won't save the planet.

▲

lelanthran 3 days ago | parent [-]

> As for inference, GPT-4 usage for an hour consumes less than watching Netflix for an hour.

This can't be correct, I'd like to see how this was measured.

Running a GPU at full throttle for one hour uses less power than serving data for one hour?

I'm very sceptical.

▲

visarga 3 days ago | parent [-]

An hour of Netflix streaming consumes approximately 77 Wh according to IEA analysis showing streaming a Netflix video in 2019 typically consumed around 0.077 kWh of electricity per hour [1], while an hour of active GPT-4 chatting (assuming 20 queries at 0.3 Wh each) consumes roughly 6 Wh based on Epoch AI's estimate that a single query to GPT-4o consumes approximately 0.3 watt-hours per query [2]. That makes Netflix about 13 times more energy-intensive than LLM usage.

[1] https://www.iea.org/commentaries/the-carbon-footprint-of-str...

[2] https://epoch.ai/gradient-updates/how-much-energy-does-chatg...

▲

lelanthran 3 days ago | parent | next [-]

Jesus Christ, what a poor take on those numbers! It's possible to have a more wrong interpretation, but not by much.

The Netflix consumption takes into account everything[1], the numbers for AI are only the GPU power consumption, not including the user's phone/laptop.

IOW, you are comparing the power cost of using a datacenter + global network + 55" TV to the cost of a single 1shot query (i.e. a tiny prompt) on the GPU only

Once again, I am going to say that the power cost of serving up a stored chunk of data is going to be less than the power cost of first running a GPU and then serving up that chunk.

==================

[1] Which (in addition to the consumption by netflix data centers) includes the network equipment in between, the computer/TV on the user's end. Consider that the user is watching netflix on a TV (min 100w, but more for a 60" large screen).

	▲	blharr 3 days ago \| parent [-]
		If you look at their figure (0.0377 kW hour) for a phone using 4G, the device power consumption is minimal and mostly made up by the network usage. The data center +network usage will be the main cost factor for streaming. For an LLM, you are not sending or receiving nearly as much data, so while I wouldn't know the numbers, it should be nominal

▲

dns_snek 3 days ago | parent | prev [-]

> while an hour of active GPT-4 chatting (assuming 20 queries at 0.3 Wh each)

We're not talking about a human occasionally chatting with ChatGPT, that's not who the article and earlier comments are about.

People creating this sort of AI slop are running agents that provide huge contexts and apply multiple layers of brute-force, like "reasoning" and dozens/hundreds of iterations until the desired output is achieved. They end up using hundreds (or even thousands) of dollars worth of inference per month on their $200 plans, currently sponsored by the AI bubble.