I was an adjunct math professor at a University for a few years and I completely agree with this. There is this propaganda circulating in (and out of) education that testing is not a good measure of mastery, but it 100% is. It's a hard sell for public schools because, as you said, people will fail more.

Math is an even tougher sell because it's largely a service department. It's hard to justify holding non-majors up because they can't pass calc 2. For upper division math though tests are the way.