The simplest practical property-based tests are where you serialize some randomly generated data of a particular shape to JSON, then deserialize it, and ensure that the output is the same.

A more complex kind of PBT is if you have two implementations of an algorithm or data structure, one that's fast but tricky and the other one slow but easy to verify. (Say, quick sort vs bubble sort.) Generate data or operations randomly and ensure the results are the same.

▲

eru a day ago | parent [-]

> The simplest practical property-based tests are where you serialize some randomly generated data of a particular shape to JSON, then deserialize it, and ensure that the output is the same.

Testing that f(g(x)) == x for all x and some f and g that are supposed to be inverses of each other is a good test, but it's probably not the simplest.

The absolute simplest I can think of is just running your functionality on some randomly generated input and seeing that it doesn't crash unexpectedly.

For things like sorting, testing against an oracle is great. But even when you don't have an oracle, there's lots of other possibilities:

* Test that sorting twice has the same effect as sorting once.

* Start with a known already in-order input like [1, 2, 3, ..., n]; shuffle it, and then check that your sorting algorithm re-creates the original.

* Check that the output of your sorting algorithm is in-order.

* Check that input and output of your sorting algorithm have the same elements in the same multiplicity. (If you don't already have a datastructure / algorithm that does this efficiently, you can probe it with more randomness: create a random input (say a list of numbers), pick a random number X, count how many times X appears in your list (via a linear scan); then check that you get the same count after sorting.

* Check that permuting your input doesn't make a difference.

* Etc.

▲

sunshowers a day ago | parent | next [-]

Speaking for myself — those are definitely all simpler cases, but for me I never found them compelling enough (beyond the "it doesn't crash" property). For me, the simplest case that truly motivated PBT for me was roundtrip serialization. Now I use PBT quite a lot, and most of them are either serialization roundtrip or oracle/model-based tests.

	▲	eru a day ago \| parent [-]
		Oh, yes, I was just listing simple examples. I wasn't trying to find a use case that's compelling enough to make you want to get started. I got started out of curiosity, and because writing property based tests is a lot more fun than writing example based tests.

▲

jgalt212 a day ago | parent | prev [-]

> The absolute simplest I can think of is just running your functionality on some randomly generated input and seeing that it doesn't crash unexpectedly.

For this use case, we've found it best to just use a fuzzer, and work off the tracebacks.

That being said, we have used hypothesis to test data validation and normalizing code to decent success. We use on a one-off basis, when starting something new or making a big change. We don't run these tests everyday.

Also, I don't like how hypothesis integrates much better with pytest than unittest.

	▲	eru 11 hours ago \| parent [-]
		> For this use case, we've found it best to just use a fuzzer, and work off the tracebacks. Yes, if that was the only property you want to test, a dedicated fuzzer is probably better. But if you are writing a bunch of property based tests, you can add a basic "this doesn't crash in unexpected ways" property nearly for free.