Remix.run Logo
imiric 13 hours ago

> Since it came up in another thread (yes, it's trivial), a function `add` is no easier or harder to test with examples than with PBT

Come on, that example is practically useless for comparing both approaches.

Take a look at the article linked above. The amount of non-trivial code required to setup a PBT should raise an eyebrow, at the very least.

It's quite possible that the value of such a test outweighs the complexity overhead, and that implementing all the test variations with EBT would be infeasible, but choosing one strategy over the other should be a conscious decision made by the team.

So as much as you're painting PBT in a positive light, I don't see it that clearly. I think that PBT covers certain scenarios better than EBT, while EBT can be sufficient for a wide variety of tests, and be simpler overall.

But again, I haven't actually written PBTs myself. I'm just going by the docs and articles mentioned here.

Jtsummers 9 hours ago | parent [-]

> Come on, that example is practically useless for comparing both approaches.

Come on, I admitted it was trivial. It was a quick example that fit into a comment block. Did you expect a dissertation?

> that implementing all the test variations with EBT would be infeasible

That's kind of the point to my previous comment. PBTs will generate many more examples than you would create by hand. If you have EBTs already, you're one step away from PBTs (the generators, I never said this was trivial just to preempt another annoying "Come on"). And then you'll have more comprehensive testing than you would have had sticking to just your carefully handcrafted examples. This isn't the end of property-based testing, but it's a really good start and the easiest way to bring it into an existing project because you can mostly reuse the existing tests.

Extending this, once you get used to it, to stateful testing (which many PBT libraries support, including Hypothesis) you can generate a lot of very useful end-to-end tests that would be even harder to come up with by hand. And again, if you have any example-based end-to-end tests or integration tests, you need to come up with generators and you can start converting them into property-based tests.

> but choosing one strategy over the other should be a conscious decision made by the team.

Ok. What prompted this? I never said otherwise. It's also not an either/or situation, which you seem to want to make it. As I wrote in that previous comment, you can use both and use the property-based tests to bolster the example-based tests, and convert counterexamples into more example-based tests for your regression suite.

> I haven't actually written PBTs myself.

Huh.