Careful engineering can help here.

Reflect back shortwave IR sunlight below the bandgap of the solar panels.

Make the solar panels "black" at longer IR wavelengths corresponding to the temperature of the panels and to IR windows in the atmosphere, to maximize emission of heat back into space.