Remix.run Logo
▲ staindk 3 hours ago

Congrats on the launch!

Some questions are top of mind after having done POCs and trials with other companies that have products in this space (excuse me if they are answered somewhere obvious - I skimmed the site and your post before drafting this):

1. What is vendor lock-in like with Momentic? In my mind there's space for a mature QA platform that agentically generates and maintains playwright/similar tests, but perhaps I'm wrong.

1.b. will Momentic code or testing standard or some aspect be open enough that it is feasible to move away from it if needed, or continue using it if Momentic goes under/deploys major breaking changes?

2. How deterministic are Momentic tests?

3. Who is your target user? Product owner / developer / SDET / ?

----

Also - is this saying Codex is better and cheaper than Mo?

"... though Codex is a bit more precise at 86% vs 84%. The tradeoff is cost: Mo costs $2.67 per bug vs $0.80 for Codex, less than Claude Code and Devin."

▲jeffan 2 hours ago | parent | next [-]

Thanks! I'm Jeff, one of the co-founders.

1. Mo generates its own test cases in natural language, but you can ask it to write Playwright and maintain it in your repository if you really want. We actually do have a regression testing product as well where tests are just natural language in your repository. For that product yes there is some vendor lock-in since this is a proprietary format that only our CLI translates into browser actions. But ultimately since it's natural language it's pretty portable :)

2. Mo is focused on exploratory testing and so is inherently non-deterministic. But the regression testing product has a ton of caching built into it that we've written about in the past (not just a single css selector). https://momentic.ai/blog/teaching-browser-agents-user-intent This is a big part of reliability and latency optimization, otherwise your CI would just hang for hours on any reasonable suite.

3. I would say the regression testing product was very much SDET/developer focused since it required you to have a codebase and use Git. Mo is meant to be super easy for anyone to onboard onto and lower the barrier to entry, so we're hoping we'll see more product owners, PMs, QA and even leadership use the product. But devs are always going to be our bread and butter, we ultimately tried to build the product we wanted to use ourselves.

4. We're optimized for recall (# of bugs you find in a given amount of time). The bench shows we find ~3-5X more bugs compared to Codex and Claude Code, with approx. the same precision. The service costs more than the raw harnesses for sure, but that's the all-in cost including our infra, parallel agents, etc.

▲ 2 hours ago | parent | prev [-]
[deleted]