Remix.run Logo
▲ AstroBen 2 hours ago

This is just pure vibe coding and trying to convince yourself it's quality work. AI can't be trusted to go from an English written spec to tests

Here[0] we have benchmarks for AI writing tests for code that already has the correct behavior. Best case scenario is that they're at 67% with Fable, and that's with the highest quality specification we could possibly have (code). English is far more ambiguous

[0]: https://labs.scale.com/leaderboard/sweatlas-tw

▲skeledrew an hour ago | parent [-]

Maybe you don't trust them to, but they can go from spec to tests; I've made them (even if it initially took extra prompting and scrutiny). And ultimately tests are just a proxy to checking that X works, written by devs for devs; actually trying X is something to be done by QA and/or project/product manager before shipping.