Hopefully in this world, someone figures out how to deliver the performance equivalent of a B300 GPU for about 1/100th the power of a current B300 (which can be up to 1400 watts), or the world will bake.

▲

tossandthrow a day ago | parent | next [-]

Compared to 20-30kw spend on cruising the highway in a car (considerably higher for older ICEs) 1.4kw does not really seem to dent the energy consumption.

Especially if cognitive technologies mean that we need to travel less (eg communiting to work, or ineffecient supply chains).

▲

eleventen a day ago | parent [-]

What percentage of your time is spent cruising at highway speed? Presumably that one GPu would be saturated all the time.

	▲	paimapi a day ago \| parent [-]
		2-3 hours at 20kW daily (assuming commute, etc) vs 20ish hours always on tasks really is close to equivalent which is to say that the last thing our planet needs is another universalized technology that outputs as much total emissions as cars in an ideal world, we'd keep LLMs/CNNs/etc specialized and academic until we are hitting diminishing returns on optimizing fundamental microprocessor tech like GAA. but the pursuit of market dominance and mass adoption is our current operating philosophy, and so we have things like this top graph: https://hai.stanford.edu/news/inside-the-ai-index-12-takeawa... >Grok 4's estimated training emissions reached 72,816 tons of CO2 equivalent, or roughly the same amount of greenhouse gas emissions created from driving 17,000 cars for one year

▲

i5heu a day ago | parent | prev | next [-]

8 biillion * 1400 watts = 11.2 terawatts

current global average electricity production = approximately 3.6 terawatts

But then again these systems do not use 1400 watts all the time. We would probably have much more then 11.2 terawatts demand if all humans turn on all their electrical consumers at the same time.

	▲	not2b a day ago \| parent [-]
		So demand quadruples if they are always on, and doubles if they are on 1/3 of the time. Either way, too much.

▲

keysersoze33 a day ago | parent | prev | next [-]

Most of the energy consumption comes from moving data between memory and compute units rather than the math itself. While HBM stacked on a silicon interposer was a signification efficiency improvement over traditional DDR, there's still room for improvement. The high energy costs will drive this further, e.g. near-memory/in-memory compute (PIM) or tighter 3D packaging and possibly optical interconnects. (And of course training/model optimisations)

▲

avaer a day ago | parent | prev | next [-]

Analog AI/approximate computing? There's research into it, worth looking at if you're interested.

The idea is you tolerate some loss/degradation (which neural networks do) but gain orders of magnitude power efficiency.

▲

threetonesun a day ago | parent | prev | next [-]

Right, if everyone in the world has a GPU we would have solved so many problems with power generation and... I'm not going to do the math but I think we'd be mining asteroids too? I'd probably use it as a bookend at the point.

▲

tomaskafka a day ago | parent | prev | next [-]

I think personalized suriveillance and manipulation (thought control, manipulating people through filtering of content they receive) on a level any dictator would drool over can be gotten for a fraction of 1 GPU per person.

▲

sriniwasx a day ago | parent | prev [-]

Specialized chips like what cereberus is building combined with small models can give great result in the long run, but at the pace AI is growing it's not very practical yet.