Yup. Working on a 10 year old codebase, I always wondered whether a test failing was "a long-standing bug was accidentally fixed" or "this behavior was added on purpose and customers rely on it". It can be about 50/50 but you're always surprised.

Change detector tests add to the noise here. No, this wasn't a feature customers care about, some AI added a test to make sure foo.go line 42 contained less than 80 characters.

▲

baq 5 days ago | parent | next [-]

I like calling out behavioral vs normative tests. The difference is optics, mostly, but the mere fact that somebody took the time to add a line of comment to ten or hundred lines of mostly boilerplate tests is usually more than enough to raise an eyebrow and I honestly don’t need more than just a pinch of surprise to make the developer pause.

▲

groestl 5 days ago | parent | prev [-]

> a long-standing bug was accidentally fixed

In some cases (e.g. in our case) long standing bugs become part of the API that customers rely on.

▲

strbean 5 days ago | parent [-]

It's nearly guaranteed, even if it is just because customers had to work around the bug in such a way that their flow now breaks when the bug is gone.

Obligatory: https://xkcd.com/1172/

	▲	thaumasiotes 4 days ago \| parent \| next [-]
		That comic doesn't show someone working around a bug in such a way that their flow breaks when the bug is gone. It shows them incorporating a bug into their workflow for purposes of causing the bug to occur. It isn't actually possible for fixing a bug to break a workaround. The point of a workaround is that you're not doing the thing that's broken; when that thing is fixed, your flow won't be affected, because you weren't doing it anyway.
	▲	giaour 5 days ago \| parent \| prev [-]
		Also known as Hyrum's Law (https://www.hyrumslaw.com/), but more people know the XKCD at this point :)