From unit tests to whole universe tests (with will wilson of antithesis) [video]

I found helpful this explanation of what Antithesis isn't:

> Property-based testing vs. Antithesis

> Property-based testing (PBT) uses random inputs to check individual data structures, procedures, or occasionally whole programs for high-level invariants or properties. Property-based testing has much in common with fuzzing—the main differences are heritage (PBT comes from the functional programming world, while fuzzing comes from the security/systems programming world) and focus (program functionality vs. security issues). Like fuzzing, PBT is generally only applicable to self-contained libraries and processes.

> Antithesis is analogous to applying PBT to an entire interacting software system—including systems that are concurrent, stateful, and interactive. Antithesis can randomly vary the inputs to a software program, and also the environment within which it runs. Like a PBT system, Antithesis is designed to check high-level properties and invariants of the system under test, but it can do so with many more types of software.

I've scrubbed through the video, and it seems to be 100% talking-head filler except for an outro still image—no actual video information content at all unless you want to analyze Wilson's facial expressions or think he's hot.

Regular reminder that yt-dlp (--write-sub --write-auto-sub --sub-lang en) can download subtitles that you can read, grep, and excerpt, so you don't have to watch videos like this unless you like to.

▲

stogot 12 hours ago | parent [-]

Thanks for the auto sub I didn’t know it is a feature.

How did you get ytdlp to work? It used to work for me and I just did a fresh install a week ango and now youtube is giving me auto/cookie/sign in errors (captcha I presume?) when it didn’t before

	▲	tczMUFlmoNk 12 hours ago \| parent [-]
		As a general rule, you should update yt-dlp before using it. They release new versions very frequently to work around new walls on YouTube and other platforms. An update usually solves this kind of issue for me, even if I've updated just a few days ago. (I haven't tried it today so can't speak to whether this is a complete solution in this particular case.)

▲

jatins 10 hours ago | parent | prev | next [-]

Is there a demo of what Antithesis does? I have seen it on HN a few times and I like the idea of monkey typing a system. But how does it work in practice? Does it call my APIs, does it introduce memory corruptions, does it bring down my containers...what does it do?

▲

vlovich123 10 hours ago | parent [-]

It arbitrarily reorders events across the entire “universe” and injects reasonable kinds of faults (eg dropping or reordering packets). It does so by running all events for all threads across all machines in a deterministic “random” order by serializing on a single thread and the randomness is initialized by the seed for that run. It also runs the universe in faster than real time since there’s no actual network delay or time elapsing (that too is simulated).

You generate the workload by defining your test case the same as property tests or traditional example tests. You cannot call arbitrary network services.

	▲	typpilol 10 hours ago \| parent [-]
		Is it like Stryker basically? Mutation testing? Or it like a super set of mutation testing?

▲

narsa123 3 days ago | parent | prev | next [-]

Any tools we could use to test mobile apps automation testing using AI (Like MCPs for mobile app testing)??

	▲	fitzn 13 hours ago \| parent [-]
		Reflect tests mobile apps by converting plain text instructions into appium commands at runtime using AI. Your tests are just the text steps. https://reflect.run/mobile-testing/ disclaimer: I co-founded Reflect.

▲

dlahoda 11 hours ago | parent | prev [-]

do they have sustained at least one prominent rust first and rust core customer? i doubt, rust has a lot of tooling and catches at compile time what their product does in runtime.

also not sure about antithesis biz practices. you pay them for integration, you spend time educating them and improving their product. and in the end get vendor locking on their compute with arbitrary non transparent pricing.

if your are not in rust - sure it can be price efficient.

▲

vlovich123 10 hours ago | parent [-]

I think you are misunderstanding. Rust does not solve or prevent distributed systems bugs, just memory safety and certain kinds of thread safety problems. For that you’d need to use a formal proof system like Coq.

There’s a reason you should still be writing unit tests and hypothesis/property tests in Rust, to catch issues the compiler can’t catch at runtime which is a huge surface area.

▲

lmm 7 hours ago | parent | next [-]

> There’s a reason you should still be writing unit tests and hypothesis/property tests in Rust, to catch issues the compiler can’t catch at runtime which is a huge surface area.

It would be irresponsible to suggest that Rust eliminates a large enough proportion of common errors that you can YOLO anything that compiles and achieve an acceptable defect rate... but it does happen to be true in my experience.

	▲	dlahoda 6 hours ago \| parent [-]
		yes. tests test what? logic. logic = types (proven). so stronger type system less test needs to be written. more, if proc macro or build.rs can execute logic, based on parsed """types"""(partial info only), we can extend type systems with custom logic (and panic at compile time and/or startup time if usage violation detected). on top of that, add fail fast (fail at compile time, build time or start up time) and newtype and errors-part-of-api culture; and lack of late binding (dyn is very limited use, no runtime reflection), and we get even less reasons to write tests. some examples of industrial """typing"""(eDSLs, construction time) solutions in rust : - https://github.com/elastio/bon - https://github.com/contextgeneric/cgp - https://github.com/paritytech/orchestra - https://git.pipapo.org/cehteh/linear_type.git sure we need write tests, and tests like antithesis helps with. but list of tools helping with tests exactly as antithesis does(and more of others) is huge. that is built on top of absolutely strong supply chain audit, quality and security. there is even """levels""" of determinism tooling to decide how much to pay for cloud compute.

▲

dlahoda 8 hours ago | parent | prev [-]

i guess most of issues anthitesis finds are preventable by simple or more evolved rust (patterns).

in rust i just have more time for other things you mentioned.

also it is clear you are misunderstand rust. rust type and macro system allow to write adhoc partial proves of things around in my code with no extra tooling. that is easy bits what rust adds on top of thread and mem.

and definetely i do need to run for help of rocq right away, rust ecosystem has a lot of options.

also not only lang itself matters, but also cargo which goes along.

▲

Smaug123 8 hours ago | parent [-]

You still seem to be completely misunderstanding, as is evident from the fact that your argument "proves" that in Rust you don't even need to write any tests. Again, Antithesis is designed to test distributed systems, deterministically.

	▲	dlahoda 8 hours ago \| parent [-]
		sorry, where exactly i stated no need to write tests? i argue, overall, that antithesis less likely will be adopted in rust because language itself (in sense extended to typelevel patterns and in macro simulations) and its ecosystem (by ecosystem i mean available libraries and tools and integrations which cover a lot of antithesis agenda). i did not expanded ecosystem argument so, because there was objection to that yet.