Remix.run Logo
lelanthran 3 days ago

Jesus Christ, what a poor take on those numbers! It's possible to have a more wrong interpretation, but not by much.

The Netflix consumption takes into account everything[1], the numbers for AI are only the GPU power consumption, not including the user's phone/laptop.

IOW, you are comparing the power cost of using a datacenter + global network + 55" TV to the cost of a single 1shot query (i.e. a tiny prompt) on the GPU only

Once again, I am going to say that the power cost of serving up a stored chunk of data is going to be less than the power cost of first running a GPU and then serving up that chunk.

==================

[1] Which (in addition to the consumption by netflix data centers) includes the network equipment in between, the computer/TV on the user's end. Consider that the user is watching netflix on a TV (min 100w, but more for a 60" large screen).

blharr 3 days ago | parent [-]

If you look at their figure (0.0377 kW hour) for a phone using 4G, the device power consumption is minimal and mostly made up by the network usage.

The data center +network usage will be the main cost factor for streaming. For an LLM, you are not sending or receiving nearly as much data, so while I wouldn't know the numbers, it should be nominal