| |
| ▲ | chasd00 a day ago | parent | next [-] | | why would a test setting unit to Bytes fail and not MB, KB, or GB, and so on? That's like trying create a unit test for email opt-in, both true and false are valid values. It's up to the user to select the right one. | | |
| ▲ | hsbauauvhabzb 12 hours ago | parent | next [-] | | Why have a default at all? Explicit is better than implicit. | |
| ▲ | 27183 a day ago | parent | prev [-] | | I'm not quite following your objection.. I'd expect a test that checks the multiplier is correct would detect orders of magnitude discrepancy. So if you're billing $x/byte you'd write a test for the billing thing that checks that, given y bytes, the bill is x*y. [edit] This may need to be an integration test to be effective, there is a certain peril to mocking that could bite you here. But that's fine, we have the technology. | | |
| ▲ | soniclettuce 12 hours ago | parent [-] | | I think they're saying that their understanding of the hypothetical error is that somebody more or less configured a price as 5¢/byte instead of 5¢/GB. And if that's the case, that's not really a thing testing will catch. 5¢/byte is as "valid" a price as 5¢/GB. There's a lot of hypotheticals in there, so who knows if testing could have caught this specific problem. But I see what they're saying in principle. |
|
| |
| ▲ | nullorempty a day ago | parent | prev | next [-] | | Technically, there could be a test. It could just be wrong! | | |
| ▲ | 27183 a day ago | parent [-] | | If a tree falls in the forest and nobody hears it... [edit] Testing your tests, like testing your backups, is a good idea | | |
| ▲ | fragmede a day ago | parent [-] | | Yes, test the negative case as well. eg if you get the system setup so you can log in, also make sure you get permission denied for bad login info. | | |
| ▲ | 27183 15 hours ago | parent [-] | | Yeah, negative tests are more important than the "test your tests" thing. Negative tests are themselves regression tests. Testing that a test works is something you do at dev time, it doesn't really live in CI. Negative tests, as well as positive tests, establish invariants on the code under test. They're effectively permanent, immortalized in the CI suite. Testing that the test actually works is a one time thing at the time the test is written. Permute the code under test, check that the test fails in the expected way, done. You just have to trust your colleagues won't edit the tests in such a way that vandalizes the invariant. That's why you can never trust an LLM to edit a test. They're notorious for tweaking tests such that they pass. |
|
|
| |
| ▲ | 8note 21 hours ago | parent | prev [-] | | missing canaries more likely? insufficient tests that dont assert on the right things? the existence of a test doesnt mean it catches the right thing based on the description, id bet the COE action item will be to do a migration that enforces units are passed at the billing service level theres no good reason for the billing service to make up its own units. | | |
| ▲ | 27183 21 hours ago | parent [-] | | Yeah types and APIs that are difficult to misuse are way better than any regression test, canary, or what have you. Systems that are correct by construction always beat "you're holding it wrong". |
|
|
