Most likely something wrong with the cooler, maybe it's not making proper contact with the CPU. Should be repairable if you give it a try, but yes, the used thin clients are higher quality.

I did clean the thermal paste between the CPU and the heat sink, and re-applied new thermal gel. That didn't help. It was the raised voltage preset in the BIOS. If I lowered the voltage, the temperature stayed low and it would run fine, until the next power down reverting the BIOS changes. The box was too small, the heat sink was too cheap, and the fan was too lacking to be a proper overclocking setup. It would run fine out of the box for couple months until the CPU and hardware worn out due to stress and heat.