The ones you hear about are caught quickly, I’m more worried about the non obvious ones. So far none of these have been as simple as changing a true to a false and bypassing all auth for all products or something, and would that be caught by an automated scanner?

There are definitely levels to this. Yes I think it can be caught by automated scanners in theory. Either commit by commit scanning and reproducible builds or fuzzing and getting the behavioral differences between versions