Remix.run Logo
somewhereoutth 5 hours ago

If your product is used by humans, then it needs to be tested by humans - this cannot be automated. Those humans can be your QA people, or your customers. Perhaps your customers are happy to be testers, perhaps not.

Unit tests are very expensive and return little value. Conversely, a (manual?) 'smoke test' is very cheap and returns great value - the first thing you do when updating a server for example is to check it still responds (and nothing has gone wrong in the deployment process), takes 2 seconds to do, prevents highly embarrassing downtime due to a misconfigured docker pull or whatever.

ivan_gammel 4 hours ago | parent | next [-]

Automated tests are no longer expensive to write and nowadays less expensive to execute.

baal80spam 4 hours ago | parent | prev [-]

> Unit tests are very expensive and return little value

Why are unit tests very expensive? This goes against everything I know.

somewhereoutth 3 hours ago | parent [-]

Unit tests very roughly double the amount of effort required to make any meaningful change to your codebase. They are also require maintenance same as ordinary code - but the customer does not care in the slightest whether or not they pass. On the other side, they can only really tell you about low level bugs that you already expected, they cannot surface system level bugs - the actual hard bugs that cause problems for you and your customers.

Then there is the danger of thinking that green=all good, an example of 'automation bias' where we learn to trust the automation even as things go wrong.

As makers, it is also tempting to believe that [all] problems can be solved by making something (i.e. code), but actually many problems are not of that nature, and cannot be solved in that way.