Remix.run Logo
27183 a day ago

If a tree falls in the forest and nobody hears it...

[edit] Testing your tests, like testing your backups, is a good idea

fragmede a day ago | parent [-]

Yes, test the negative case as well. eg if you get the system setup so you can log in, also make sure you get permission denied for bad login info.

27183 15 hours ago | parent [-]

Yeah, negative tests are more important than the "test your tests" thing. Negative tests are themselves regression tests. Testing that a test works is something you do at dev time, it doesn't really live in CI. Negative tests, as well as positive tests, establish invariants on the code under test. They're effectively permanent, immortalized in the CI suite. Testing that the test actually works is a one time thing at the time the test is written. Permute the code under test, check that the test fails in the expected way, done.

You just have to trust your colleagues won't edit the tests in such a way that vandalizes the invariant. That's why you can never trust an LLM to edit a test. They're notorious for tweaking tests such that they pass.