Remix.run Logo
▲ dahart 2 hours ago

This is a nice idea, but I’m in a big enough and low-level enough code base that using other languages or stacks isn’t very viable; we build the stack, so it’s software and hardware and compiler and driver. We do sometimes use multiple agents and adversarial review. The agents tend to come up with more complete corner-case testing than humans, the most common issue is that they don’t fully understand all design assumptions and parameters and they over-test for things that can’t happen using very complicated test setups, and they don’t test enough for simple mainline usage; they generally don’t know what the common workflows are. I had the same complaint about humans before LLMs too. :P We often try to be clever about testing and design all this logical coverage while ignoring the boring & easy stuff, we think like engineers and not users.