Hey! I'm Henry, one of the engineers working on Mo. You're looking at the right numbers, but the takeaway is slightly different from ours.

The main thing we're trying to show is that Mo finds over 3x as many correct bugs in the same amount of time as the next-best QA agent. This is the main thing we optimize for since the worst thing you can do in QA is miss a real bug and ship broken code to prod. Currently, the tradeoff you make for that better recall is higher cost. I would say the precision is more or less comparable, but marginally lower.

Obviously we'd love to be beating every other agent on all of these numbers, and we're actively working on it, but for now this is an honest and transparent look at how the tool is doing.

Feel free to take a look at the full writeup for more detail!

I feel like your writeup could be a little stronger if you made it clear how important is hitting those recall targets.

I also do believe search quality is recall dominated, but the link between the metrics and results are not as clear as they could have been. I think you have a stronger sell here than it looks at first glance.

Congratulations to the team tho! Those are very nice results