Is there a demo of what Antithesis does? I have seen it on HN a few times and I like the idea of monkey typing a system. But how does it work in practice? Does it call my APIs, does it introduce memory corruptions, does it bring down my containers...what does it do?

▲

vlovich123 12 hours ago | parent [-]

It arbitrarily reorders events across the entire “universe” and injects reasonable kinds of faults (eg dropping or reordering packets). It does so by running all events for all threads across all machines in a deterministic “random” order by serializing on a single thread and the randomness is initialized by the seed for that run. It also runs the universe in faster than real time since there’s no actual network delay or time elapsing (that too is simulated).

You generate the workload by defining your test case the same as property tests or traditional example tests. You cannot call arbitrary network services.

▲

typpilol 11 hours ago | parent [-]

Is it like Stryker basically? Mutation testing?

Or it like a super set of mutation testing?

	▲	vlovich123 2 hours ago \| parent [-]
		Mutation testing is poorly named in some sense. Mutation testing is probabilistic measurement of code coverage. It doesn’t test anything, instead it tells you how good your existing test suite provides coverage of non-existent bugs - it changes your code to generate a “mutant” (eg changes an addition to a subtraction) and sees if your test suite still passes - if it does that counts as a failure. Traditional alternative approaches are things like codecov that measure line coverage or branch coverage which famously don’t actually give you an actually accurate estimate of quality whereas mutation testing does a bit better job. However, none of this actually tests anything and is more a meta-metric of how good your test coverage is. Antithesis is more like property testing - it tests your code under random scenarios and sees if the tests still pass. Unlike property testing, rather than just random inputs, it also randomly reorders events at a very deep level to make sure your distributed system behaves correctly. It can even be used for simple things like helping you deterministically reproduce a flaky test in a non-distributed system.