Remix.run Logo
petra 3 hours ago

I wonder what perf. this cpu can get in bit/ternary inference, and whether the 512AVX units(or whats it's called on AMD), get throttled.

wincy an hour ago | parent [-]

I’ve heard that AMD chips generate a ton of heat with AVX512 operations, overclockers turn off the instruction set for more stability.

adrian_b 44 minutes ago | parent [-]

Any CPU generates more heat when it does more work, e.g. when more cores are active, or when using vector instructions.

Using the AVX-512 operations is much more efficient, i.e. the same amount of work can be done by consuming less energy. For instance, if you have a program running an AVX computation on 4 cores, you could double the throughput either by using AVX-512 or by using 8 cores, but the former variant will produce less heat.

For an overclocked system, where the normal power limits of the CPU are disabled, so it can consume a power great enough to exceed the capability of the cooler, it may make sense to disable AVX-512, for fear that one could run accidentally some program that is aware of AVX-512 and which would achieve a greater speed by using it, while also exceeding the power consumption that can be handled by the existing cooler.

On the other hand, on a non-overclocked system or in an overclocked system with a very capable cooler, disabling AVX-512 is always a mistake, which can only increase the average power consumption.