The thermal design is irrelevant, and people saying they want insane power density are, in my personal view, deluded ridiculous individuals who understand very very little.

Overclocking long ago was an amazing saintly act, milking a lot of extra performance that was just there waiting, without major downsides to take. But these days, chips are usually already well tuned. You can feed double or tripple the power into the chip with adequate cooling, but the gain is so unremarkable. +10% +15% +20% is almost never going to be a make or break difference for your work, and doing so at double or triple the power budget is an egregious waste.

So many of the chips about are already delivered at way higher than optimum efficiency, largely for bragging rights. The exponential decay of efficiency you keep pushing for is an anti-quest, is against good. The absolute performance wins are ridiculous to seek. In almost all cases.

If your problem will not scale and dumping a ton of power into one GPU or one cpu socket is all you got, fine, your problem is bad and you have to deal with that. But for 90% of people, begging for more power proces you don't actually know jack & my personal recommendation is that all such points of view deserve massive down voting by anyone with half a brain.

Go back to 2018 and look at Matthew Dillon on DragobflyBSD underpowering the heck out of their 2990wx ThreadRipper. Efficiency just soars as you tell the chip to take less power. The situation has not improved! Efficiency skyrockets today at least as much as it did then by telling chips not to go all out. Good chips behave & reward. I believe Apple competent enough to thoroughly disabuse this position that this chip would be far better if we could dump 2x 3x more power into it. Just a fools position, beyond a joke, imo. https://apollo.backplane.com/DFlyMisc/threadripper.txt

▲

vablings 10 minutes ago | parent | next [-]

It's been funny to see people move from overclocking to underclocking. Especially for the older AMD gpus. On the RX480 a slight underclock would cut the power usage almost in half!

▲

nottorp an hour ago | parent | prev | next [-]

> Overclocking long ago was an amazing saintly act, milking a lot of extra performance that was just there waiting, without major downsides to take.

Back when you bought a 233 Mhz chip with ram at 66 Mhz, ran the bus at 100 Mhz which also increased your ram speed if it could handle it, and everything was faster.

> But these days, chips are usually already well tuned. You can feed double or tripple the power into the chip with adequate cooling, but the gain is so unremarkable. +10% +15% +20% is almost never going to be a make or break difference for your work

20% in synthetic benchmarks maybe, or very particular loads. Because you only overclock the CPU these days so anything hitting the ram won't even go to 20%.

	▲	mapt an hour ago \| parent [-]
		Initially, thermal throttling was a safety valve for a failure condition. A way to cripple performance briefly so as not to let the magic blue smoke out. Only a terrible PC would be thermal throttling out of the box; Only neglectful owners who failed to clean filters, had thermal throttling happening routinely. That's not how it works any more. Many of these CPUs both at the high end and even a few tiers down from the top, are thermal throttling whenever they hit 100% utilization. I'm thinking of Intel's last couple generations particularly. They're shipped with pretty good heatsinks, but not nearly good enough to run stock clocks on all cores at once. Instead, smarter grades of thermal throttling are designed for for routine use to balance loads. Better heatsinks (and watercooling) help a bit, but not enough, you end up hitting a wall; Only the risky process of delidding seems to push further. We're running into limitations on how well a conventional heatsink can transfer the heat from a limited contact patch. GPUs seem to have more effective heatsinks, and are bottlenecked mostly by power requirements. The 600 watt monsters are already melting cables that aren't in perfect condition.

▲

Marsymars 10 hours ago | parent | prev | next [-]

Oh, we're largely on the same page there.

I was actually looking for benchmarks earlier this week along those lines - ideally covering the whole slate of Arrow Lake processors running at various TDPs. Not much available on the web though.

▲

ssl-3 9 hours ago | parent | prev | next [-]

I learned a lot about underclocking, undervolting, and computational power efficiency during my brief time in the ethereum mining[1] shenanigans. The best ROI was with the most-numerous stable computations at the lowest energy expense.

I'd tweak individual GPUs' various clocks and volts to optimize this. I'd even go so far as to tweak fan speed ramps on the cards themselves (those fans don't power themselves! There's whole Watts to save there!).

I worked to optimize the efficiency of even the power from the wall.

But that was a system that ran, balls-out, 24/7/365.

Or at least it ran that way until it got warmer outside, and warmer inside, and I started to think about ways to scale mining eth in the basement vs. cooling the living space of the house to optimize returns. (And I never quite got that sorted before they pulled the rug on mining.)

And that story is about power efficiency, but: Power efficiency isn't always the most-sensible goal. Sometimes, maximum performance is a better goal. We aren't always mining Ethereum.

Jeff's (quite lovely) video and associated article is a story about just one man using a stack of consumer-oriented-ish hardware in amusing -- to him -- ways, with local LLM bots.

That stack of gear is a personal computer. (A mighty-expensive one on any inflation-adjusted timeline, but what was constructed was definitely used as a personal computer.)

Like most of our personal computers (almost certainly including the one you're reading this on), it doesn't need to be optimized for a 24/7 100% workload. It spends a huge portion of its time waiting for the next human input. And unlike mining Eth in the winter in Ohio: Its compute cycles are bursty, not constant, and are ultimately limited by the input of one human.

So sure: I, like Jeff, would also like to see how it would work when running with the balls[2] running further out. For as long as he gets to keep it, the whole rig is going to spend most of its time either idling or off, anyway. So it might as well get some work done when a human is in front of it, even if each token costs more in that configuration than it does OOTB.

It theoretically can even clock up when being actively-used (and suck all the power), and clock back down when idle (and resume being all sleepy and stuff).

That's a well-established concept that [eg] Intel has variously called SpeedStep and/or Turbo Boost -- and those things work for bursty workloads, and have worked in that way for a very long time now.

[1]: Y'all can hate me for being a small part of that problem. It's allowed.

[2]: https://en.wikipedia.org/wiki/Centrifugal_governor

	▲	jermaustin1 2 hours ago \| parent [-]
		I did Crypto Mining as an alternative to heating. In my centrally cool apartment my office was the den which had the air return. So my mining rig ran RIGHT in front of that, it sucked the heat out and pushed it all over the house. Then summer came, and in Texas the AC can barely keep up to begin with. So then my GPUs became part of a render farm instead.

▲

sandworm101 7 hours ago | parent | prev [-]

>> people saying they want insane power density are, in my personal view, deluded ridiculous individuals who understand very very little.

Or they are simply not-rich people who cannot afford to purchase extra hardware to run in parallel. Electricity is cheap. GPUs are not. So i want to get every ounce of power out of the precious few GPUs i can afford to own.

(And dont point at clouds. Running AI on someone else's cloud is like telling a shadetree mechanic to rent a car instead of fixing his owm.)