> At Google, for example, 16% of tests exhibited flakiness

This really surprised me. In my experience, usually a flaky test indicates some kind of race condition, and often a difficult-to-reproduce bug.

In the past year, we had a flaky unit test that caused about 1-2% of builds to fail. Upon fixing it, we learned it was what caused a deadlock in a production service every 5-6 months. As a result of fixing this one "flaky" test, we eliminated our biggest cause of manual intervention in our production environments.

At scale, every test is flaky.

> > At Google, for example, 16% of tests exhibited flakiness

> This really surprised me.

It doesn't surprise me. Close to 80% of Google code is actually quite terrible.

> In my experience, usually a flaky test indicates some kind of race condition, and often a difficult-to-reproduce bug.

Yup, that's my experience too. I flip shit when I start seeing flaky tests because it means a lot of actual work. Luckily there's generally a lot of tools available (sanitizers are a godsend) to fix it, and a lot of experience I rely on to look for smells when those tools come up empty.

It makes me really sad when I work on a project with few (or, gasp, zero) tests. I've spent the last 5 months just adding tests to my current project at $employer and... those tests have revealed all kinds of problems in dependencies created by other teams at $employer. Ugh.