Remix.run Logo
skybrian 4 hours ago

It isn't used by anyone besides me, but I wrote a property-testing library for Deno [1] that has a form of "sometimes" assertions (inspired by Antithesis) and uses "internal shrinking" (inspired by Hypothesis).

But it's still a "blind" fuzzer and it would be nice to write one that gets feedback from code coverage somehow. Instead, you have to run code coverage yourself and figure out how to change test data generation to improve it.

[1] https://jsr.io/@skybrian/repeat-test

chriswarbo 2 hours ago | parent [-]

> But it's still a "blind" fuzzer and it would be nice to write one that gets feedback from code coverage somehow

There have been simplistic attempts at this, e.g. instead of performing 100 tests, just keep going as long as coverage increases.

The Choice Gradient Sampling algorithm from https://arxiv.org/pdf/2203.00652 feels like a nice way to steer generators in a more nuanced way. That paper uses it to avoid discards when rejection-sampling; but I have a feeling it could be repurposed to "reward" based on new coverage instead/as-well.