Remix.run Logo
crabbone 2 days ago

1. I'm not sure what CodeRabbit has to do with the article... It's obviously an advertisement, but it's merged into the text of the article, which I find bizarre. As an aside, I used CodeRabbit at work, and I have mixed feelings about it. I'm not entirely against using it or a similar tool, but people often treat the comments from CodeRabbit as a gospel, and they can harm their code as a result. Also, I've never seen CodeRabbit being anything more than a superficial reviewer: fixing typos, other unintended errors, but it never comments on the substance of the change, which implicitly validates it for the author.

2. On test composition. Unit tests are called "unit" because they are supposed to test one thing. If a test is testing more than one thing, it's an integration test. Ideally, people writing unit tests are the developers themselves and people writing integration tests are test (automation) people. This matters for administrative reasons: in the development cycle, the unit test is the basic check that validates a particular piece of code, probably, submitted for review or for merging. Passing such a test might be a necessary condition to progress the changeset along the designed workflow path. Integration tests, on the other hand, are more of a retrospective tool that is meant for detection of problems in the entire product. A failure of an integration test should, normally, schedule new task for the developers, not reject the one being worked on. Integration tests, typically, will require a more elaborate system under test setup and a more elaborate, perhaps involving multiple teams, investigation of the failure. They can be also a lot more expensive to run in terms of equipment used.

Finally, the author touched on a contentious subject a.k.a. the number of assertions in a test. A lot of people believe (me included) that the number of assertions in the unit test should be exactly one. This is often inconvenient because it requires implementation of equality for possibly ad hoc created set of results. Even so, I believe it's still worth it.

When it comes to integration tests, I don't believe assertions are at all the way to go. The system under test should be monitored continuously and every reading should be compared against the desired state of the system that the test modifies simultaneously with the change effected to the system. This is because, in practice, it's rarely just two features that are tested together. If the test waits until the final step to compare the desired and the actual state of the system, the error as well as the context in which it happened might be long gone.

As a side bonus: the monitoring+alerts system could well be part of the product itself, or, if not, it can be used in long-running tests intended to collect mileage (i.e. tests intended to prove that the system performance doesn't degrade over substantially long periods of time).

senderista 2 days ago | parent [-]

That's why you pepper your code with assertions and rely on depth of test coverage (including generative testing) to hit them.