> 2 m2 panel makes about 500w

It receives around 2.5kW[0] of energy (in orbit), of which it converts 500W to electric energy, some small amount is reflected and the rest ends up as heat, so use 1kW/m^2 as your input value.

> If we run the radiators at 80C (a reasonable temp for silicon), that's about 350K, assuming the outside is 0K which makes the radiator be able to radiate away about 1500W, so roughly double.

1500W for 2m^2 is less than 2000kW, so your panel will heat up.

[0] https://www.sciencedirect.com/topics/engineering/solar-radia...

You can't just omit the 500 W of electric. That ultimately ends up as heat too.