▲ | lelanthran 3 days ago | |
Jesus Christ, what a poor take on those numbers! It's possible to have a more wrong interpretation, but not by much. The Netflix consumption takes into account everything[1], the numbers for AI are only the GPU power consumption, not including the user's phone/laptop. IOW, you are comparing the power cost of using a datacenter + global network + 55" TV to the cost of a single 1shot query (i.e. a tiny prompt) on the GPU only Once again, I am going to say that the power cost of serving up a stored chunk of data is going to be less than the power cost of first running a GPU and then serving up that chunk. ================== [1] Which (in addition to the consumption by netflix data centers) includes the network equipment in between, the computer/TV on the user's end. Consider that the user is watching netflix on a TV (min 100w, but more for a 60" large screen). | ||
▲ | blharr 3 days ago | parent [-] | |
If you look at their figure (0.0377 kW hour) for a phone using 4G, the device power consumption is minimal and mostly made up by the network usage. The data center +network usage will be the main cost factor for streaming. For an LLM, you are not sending or receiving nearly as much data, so while I wouldn't know the numbers, it should be nominal |