I am fascinated by the prevalence of wanting "tests" from hacker news comments. Most of the code I have worked on in the past 20 years did not have tests. Most of it was shopping carts, custom data transformation code, orchestrating servers, plugin code functionality to change some aspect of a website.

Now, I have had to do some salesforce apex coding and the framework requires tests. So I write up some dummy data of a user and a lead and pass it through the code, but it feels of limited value, almost like just additional ceremony. Most of the bugs I see are from a misconception of different users about what a flag means. I can not think of a time a test caught something.

The organization is huge and people do not go and run all the code every time some other area of the system is changed. Maybe they should? But I doubt that would ever happen given the politics of the organization.

So I am curious, what are the kinds of tests do people write in other areas of the industry?

what are the kinds of tests do people write in other areas of the industry?

Aerospace here. Roughly this would be typical:

- comprehensive requirements on the software behavior, with tests to verify those requirements. Tests are automated as much as possible (e.g., scripts rather than manual testing)

- tests are generally run first in a test suite in a completely virtual software environment

- structural coverage analysis (depending on level of criticality) to show that all code in the subsystem was executed by the testing (or adequately explain why the testing can't hit that code)

- then once that passes, run the same tests in a hardware lab environment, testing the software as it runs on the the actual physical component that will be installed on the plane

- then test that actually on a plane, through a series of flight tests. (The flight testing would likely not be as entirely comprehensive as the previous steps)

A full round of testing is very time-consuming and expensive, and as much as possible should be caught and fixed in the virtual software tests before it even gets to the hardware lab, much less to the plane.

The interesting bit is this: how much of what you wrote over all those years actually did what you wanted it to do, no more, no less?

This is where testing gets interesting: I took some old code I wrote 30 years ago or so and decided to put it literally to the test. A couple of hundred lines from a library that has been in production without every showing a single bug over all that time. And yet: I'm almost ashamed at how many subtle little bugs I found. Things you'd most likely never see in practice, but still, they were there. And then I put a couple of those bugs together and suddenly realized that that particular chain must have happened in practice in some program built on top of this. And sure enough: fixing the bugs made the application built on top of this more robust.

After a couple of weeks of this I became convinced: testing is not optional, even for stuff that works. Ever since I've done my best to stop assuming that what I'm writing actually does what I want it to. It usually does, for the happy path. But there are so many other paths that with code of any complexity, even if you religiously avoid side effects you can still end up with issues that you overlook.

How do you know you haven't unknowingly broken something when you made a change?

I think if: - the code base implements many code paths depending on options and user inputs and options such that a fix for code path A may break code path B - it takes a great deal of time to run in production such that issues may only be caught weeks or months down the line when it becomes difficult to pinpoint their cause (not all software is real-time or web) - any given developer does not have it all in their head such that they can anticipate issues codebase wide

then it becomes useful to have (automated) testing that checks a change in function A didn't break functionality in function B that relies on A in some way(s), that are just thorough enough that they catch edge cases, but don't take prod levels of resources to run.

Now I agree some things might not need testing beyond implementation. Things that don't depend on other program behavior, or that check their inputs thoroughly, and are never touched again once merged, don't really justify keeping unit tests around. But I'm not sure these are ever guarantees (especially the never touched again).

The value of tests is when the fail they show you of something you broke that you didn't realize. 80% (likely more, but I don't know how to measure) of the tests I write could safely be thrown away because they fail again - but I don't know which tests will fail and thus inform me that I broke things.

The system I'm working on has been in production for 12 years - we have added a lot of new features over those years. Many of those needed us to hook into existing code, tests help us know that we didn't break something that used to work.

Maybe that helps answer the question of why they are important to me. They might not be to your problems.

I think the whole concept of testing confuses a lot of people. I know I was (and still sometimes am) confused about the various "best practices" and opinions around testing. As as well as how/what/when to test.

For my projects, I mainly want to Get Shit Done. So I write tests for the major functional areas of the business logic, mainly because I want to know ASAP when I accidentally break something important. When a bug is found that a test didn't catch, that's usually an indicator that I forgot a test, or need to beef up that area of functional testing.

I do not bother with TDD, or tests that would only catch cosmetic issues, and I avoid writing tests that only actually test some major dependency (like an ORM).

If the organization you are in does not value testing, you are probably not going to change their mind. But if you have the freedom to write worthwhile tests for your contributions to the code, doing so will probably make you a better developer.

Follow-up questions: Do you test manually? Why? Do you debug manually? Why?

You wanted examples: https://github.com/openjdk/jdk/tree/master/test/jdk/java/uti...