I posted this comment in 2013 [1] and I stand by it 12 years later:

Cucumber tries to solve the problem of turning customer requirements into 'real code'. In exchange for that worthwhile benefit, it asks you to implement the most terrible, reg-ex based spaghetti code imaginable.

The problem is that it doesn't solve the original problem AT ALL. And then you are left with terrible reg-ex driven spaghetti code. Like the Jamie Zawinski saying, "now you have two problems".

The lesson here is that software development processes have to pass the 'human nature' test.

The software industry has largely abandoned waterfall development because it just doesn't work well in practice. It doesn't work because people don't know perfectly what they want before they build it. Agile processes usually are much more efficient because they are more closely aligned to how humans solve problems in the real world.

Cucumber suffers from the same issue of being disconnected with reality. In theory, you can find a customer who can give you perfectly written use cases and you can cut-and-paste those into your cukes. In practice, that never, ever works. So let's all stop wasting our time pretending it was a good idea now that it has been shown to not work.

[1] https://news.ycombinator.com/item?id=6411787#6412391

▲

blablabla123 4 days ago | parent | next [-]

Nobody is going to read these test cases any way, except when troubleshooting or adding new code. But that is never any of the stakeholders

It's quite funny that now with AI-based no/low code the same thing is attempted again. Even if the regexes might disappear, it's even more text, with even less structure (assuming anyone checked those prompts into git in the first place)

▲

hn_throwaway_99 4 days ago | parent | next [-]

But, as a software engineer, writing tests is probably the primary place AI gives me value. I can simply write tests much faster than writing them manually, and as a consequence, I'm a lot more likely to write tests that cover more cases. I still review all the code output, but the consequence of me missing a bug is much less than missing a bug in production code.

▲

MoreQARespect 4 days ago | parent | prev [-]

I use executable specs quite a lot with stakeholders.

I'd never ask them to write them, but I will often write a spec/test based upon their two sentence jira and then screenshare and walk them through my interpretation to get feedback early (i.e. before ive wasted time building the wrong thing).

Cucumber/gherkin is awful at this of course, and the regex thing was a terrible idea but it's not the only tool.

The idea that tests should be split into a specification layer and execution layer is a good one that should have taken off by now.

▲

jonahx 4 days ago | parent | next [-]

> The idea that tests should be split into a specification layer and execution layer is a good one that should have taken off by now.

There is a fundamental reason it hasn't:

An actual specification layer isn't any simpler than the execution layer. That's a programmer's fallacy.

What has taken off, and is part of virtually every software project, is a loose, natural language specification, which hints at "more or less" what the stakeholders are imagining. The idea that you can close the gap between this and a complete specification in a way that all stakeholders can still digest is the fantasy of cucumber. Or any other tool that attempts it.

You can't solve the problem in that way. Because, from a high-level stakeholder's perspective, the whole point of the people below them (whether programmers or UX designers or anyone else) is to fill in those details in a way that matches their own hazy expectations, at least well enough.

▲

MoreQARespect 4 days ago | parent [-]

>An actual specification layer isn't any simpler than the execution layer.

The point of separation of concerns isnt to keep the simple layer separate from the complex one. It's to simplify the whole thing by only addressing one concern per layer.

Unit tests are often a pain in the ass to read because they are a mess of implementation and specification details. No separation.

>You can't solve the problem in that way. Because, from a high-level stakeholder's perspective, the whole point of the people below them (whether programmers or UX designers or anyone else) is to fill in those details in a way that matches their own hazy expectations

If you crystallize the hazy expectations on their behalf and skip feeding it back to them then you will often find out that "thats not what i meant" after the code is complete.

Those mistakes are expensive and avoidable.

▲

jonahx 4 days ago | parent [-]

Oh I understand the standard arguments. I just don't agree with them.

> The point of separation of concerns isnt to keep the simple layer separate from the complex one. It's to simplify the whole thing by only addressing one concern per layer.

With some caveats, I think this is just a fiction. There can be some value in having high-level tests and low-level tests, but not because it removes complexity. It can help with focus and priority. Which is a problem that can be solved in many ways.

> If you crystallize the hazy expectations on their behalf and skip feeding it back to them then you will often find out that "thats not what i meant" after the code is complete.

But this is exactly what they want you to do, and do well enough that "that's not what i meant" is not a big problem. They certainly don't want to read cucumber tests as a way of ensuring you're on the same page. They will tolerate rough, incremental prototypes, per the old agile advice, and this is probably still the best way of solving the communication gap problem.

	▲	MoreQARespect 4 days ago \| parent [-]
		>Oh I understand the standard arguments. I think the problem here is that you are substituting standard arguments for my arguments. >With some caveats, I think this is just a fiction. There can be some value in having high-level tests and low-level tests, but not because it removes complexity. Separation of concerns isn't about having "high level" and "low level" tests. You don't have "high level" views and "low level" views in MVC. You have views and controllers and you keep them separate because otherwise you have an old-skool PHP-like mix of SQL and HTML all over the place. That PHP style mess happens to most tests because the code that is intended to describe behavior is mushed up thoroughly with the code that attaches to the app and executes it. If you separate the concerns, the test becomes readable and even serves as a spec. >But this is exactly what they want you to do, and do well enough that "that's not what i meant" is not a big problem. The problem is rarely that they didn't say what they mean. The problem is that once you've built the thing or walked them through a crystallized user story it makes them rethink what was actually needed. In many cases I've walked stakeholders through user stories and they suddenly realized no new feature was necessary. >They certainly don't want to read cucumber tests Nobody does, but that's because cucumber tests have abysmal syntax and UX, not because the idea of readable spectests was intrinsically bad. Throwing away spectests as a concept because cucumber is bad is like throwing away programming as a concept because COBOL is bad. >They will tolerate rough, incremental prototypes Of course, which is not as expensive as a fully fledged app but way more expensive to build than crystallized user stories. I prefer the cheaper option.

▲

exe34 4 days ago | parent | prev [-]

Hey, could you please share what you use for writing your specs? Pure python? Or some DSL?

▲

MoreQARespect 4 days ago | parent [-]

hitchstory. typed YAML over python.

	▲	exe34 4 days ago \| parent [-]
		thanks! looks interesting.

▲

Xss3 3 days ago | parent | prev | next [-]

My manager and our qa team like that they can read what we are testing step by step in plain English and reason about it the same way. If they want a new step they often request it in a similar format to the ones we have written, or directly suggest reusing a step from another test.

You dont get that for free, but i disagree with it being a 'mess' of regex...Regex for test steps is usually as simple as a basic string or number capture pattern. It's 100% a skill issue if it ends up 'messy'.

If you need something more complicated than regex you use a data source file for parameters with named columns....Which brings me nicely round to how easy it is for qa or management to look at the data being tested and again, reason about it, make suggestions, spot missing cases, etc.

▲

Xss3 3 days ago | parent | next [-]

Oh and one more thing. I get a bit of a 'hur durr regex bad because its regex' vibe from your comment. Regex is absolutely the right tool for the job here. The alternative would be a custom DSL.

▲

Xss3 3 days ago | parent | prev [-]

Just to pre-empt anyone saying 'but nobody will ever read them'...that absolutely isn't true in my company.

Qa read every test we write. Management reads ones of high importance, sometimes.

	▲	Xss3 3 days ago \| parent [-]
		Ps: everyone is very happy with this arrangement, quality is very important, bugs are unacceptable for our products, and auditors wouldnt pass us if they saw inadequate testing

▲

burnt-resistor 3 days ago | parent | prev [-]

Everything makes sense, except:

> The software industry has largely abandoned waterfall development

Variants of waterfall are essential for large, high-risk safety systems. There is no 1SFA dev process.