I read it as the inefficient part isn't the compute efficiency, the inefficient part is dumping all the resulting heat into the environment without capturing it and using it in some way to generate electricity or do work.

On a related/side note, when there's talk about seti and dyson spheres, and detecting them via infrared waste heat, I also don't understand that. Such an alien civilization is seemingly capable of building massive space structures/projects, but then lets the waste heat just pour out into the universe in such insane quantities that we could see it tens/hundreds of light years away? What a waste. Why wouldn't they recover that heat and make use of it instead? And repeat the recovering until the final waste output is too small to bother recovering, at which point we would no longer be able to detect it.

> but then lets the waste heat just pour out

There is no other alternative! If I build a perfect Dyson sphere and capture the energy output of a star, all of that energy will become heat. The average surface temperature of my Dyson sphere will be (IIRC) the ratio of the surface area of the sphere to that of the contained star, multiplied by the star's effective surface temperature.

"Recovering heat and making use of it" requires a heat differential. You need a cold side and a hot side to use energy. Using that energy causes the cold side to heat and the hot side to cool, until they reach equilibrium. The further the difference, the more usable work you can do. The closer the two sides are, the less work you can do.

Someone else here said it best: waste heat is the graveyard of energy. Once you have used energy, it will become high-entropy, low-grade, diffuse heat which is difficult-to-impossible to extract further work from.

All energy inevitably changes into heat eventually, and in the steady state, power in = power out.

There is no way to get rid of heat. It has to go somewhere; otherwise, the temperature of the system will increase without bound.