This isn't being hard on abuse though, this is being lazy and incompetent.

I'm fairly sure that Safe Browsing's false-positive rate is extremely low otherwise it'd be unusable in Chrome. Which also means that acting on positive results is very likely a correct approach.