Disabling non-actionable alerts actually lowered the error rate in my experience, because people would start paying attention to the alerts. Even if they were being lazy, they'd be able to see a pattern after getting rid of the noise.

Exactly! Cut the noise, boost the signal. Every alert outside business hours should mean "drop everything and investigate this". Otherwise it can wait until the morning.