Remix.run Logo
esafak 6 hours ago

I agree that not having to read the code is the future but I struggle to get there because the AI, like humans, can make mistakes (not follow instructions). For example, I might ask the AI to render a scene using a physical model, and we might come to an agreement at the plan stage, but then it will go off and cut some corner in implementation. So the question is how to reliably verify compliance. If you merely tell another AI to check it properly implemented the plan it too can miss bugs, so you need to put all your effort in making the acceptance tests foolproof, and this is easier said than done. You need a defense in depth approach, using the type system, unit tests, formal verification (if you can), linting, etc. It is an open question how to optimally allocate your correctness efforts for a given budget but, as models get smarter, it is clear to see that the amount of human diligence needed to achieve likely correctness for a given task (say, in LOC) is going to tend to zero (https://metr.org/time-horizons/)