The benefit is certainty that the system you are building and delivering to people works. If that benefit is little, then I don’t quite understand the point of testing.

> it's also easy to manually test while developing during QA and UAT.

As I said in the original comment, e2e tests can definitely be manual. Invoke your CLI, curl your API, click around in GUI. That said, comprehensively testing it that way quickly becomes infeasible as your software grows.

▲

dsego 6 days ago | parent [-]

> The benefit is certainty that the system you are building and delivering to people works.

I'd say that works and works correctly and covers all edge cases are different scenarios in my mind. Looking at an exaggerated example, if I build tax calculator or something that crunches numbers, I'd have more confidence with a few unit tests matching the output of the main method that does the calculation part than a whole end-to-end test suite. It seems wasteful to run end to end (login, click buttons, check that a UI element appears, etc) to cover the logical output of one part that does the serious business logic. A simple e2e suite could be useful to check for regressions, as a smoke test, but it still needs to be kept less specific, otherwise it will break on minor UX changes, which makes it a pain to maintain.

▲

strogonoff 6 days ago | parent [-]

An e2e test shows that it works. If your tax calculator’s business logic perfectly calculates the tax, but the app fails with a blank screen and a TypeError in console because a function from some UI widget lib dependency changed its signature, your calculator is as good as useless for all intents and purposes. A good unit test will not catch this, because you are not testing third-party code. An integration test that catches it approaches the complexity of an e2e.

Sure, you wouldn’t have all possible datasets and scenarios, but you can easily have a few, so that e2e test fails if results don’t make sense.

Of course, unit tests for your business logic make sense in this case. Ideally, you would express tax calculation rules as a declarative dataset and take care of that one function that applies these rules to data; if the rules are wrong, that is now a concern for the legal subject matter experts, not a bug in the app that you would need to bother writing unit tests for.

However, your unit test passing is just not a signal you can use for “ship it”. It is a development aid (hence the requirement for them to be fast). Meanwhile, an e2e test is that signal. It is not meant to be fast, but then when it comes to a release things can wait a few minutes.

▲

dsego 6 days ago | parent [-]

What's more likely to fail or cause issues? Dependencies failing and parsing errors are usually handled by the build system (type checkers and linters). In the case where they are triggered in production, it can be easily caught by monitoring services like Sentry. Ideally any changes are manually tested before releasing, and a bug in one part of the app that's being worked on is not likely to affect a different section, e.g. not necessary to retest the password reset flow if you're working on the home dashboard. Having a suit of usually flaky end-to-end tests seems like the most sloppy and cumbersome way to ensure the application runs fine, especially for a small team.

	▲	strogonoff 6 days ago \| parent [-]
		That sounds suspiciously like “don’t need to test if I use static typing and monitoring”. > Ideally any changes are manually tested before releasing, and a bug in one part of the app that's being worked on is not likely to affect a different section, e.g. not necessary to retest the password reset flow if you're working on the home dashboard That is one can of worms. First, during normal development work it is very common to modify some part that affects multiple parts of the app. In fact, it is inhuman to know exactly what it affects in a big app (ergo, testing). Second, while manual testing is a kind of e2e testing, it is not feasible in a bigger application. > usually flaky end-to-end tests Then make them not flaky. It’s amazing what can happen if something stops being treated as an afterthought!