Making balls ever rounder clearly has diminishing returns, as there's a limit to roundness.

Is there a similar limit to intelligence though?

Yes, absolutely. Even human intelligence stops being useful at a certain point and becomes 'enough, provided the human is motivated and tenacious'. Intention is a better metric, but it's a serious AI weak point if not actively an achilles' heel. 'Warios' come to mind.