Remix.run Logo
▲ frostiness 31 minutes ago

Neither of these prompts effectively address the problem though. Fixing the root cause and doing so with the least amount of code often is contradictory. Including instructions to make suggestions to improve code maintainability will cause the AI to do it every single time it makes a change, even if it's completely unnecessary, drowning out useful suggestions in noise. When a bug is caused by a requirement elsewhere in the application, current models are more likely to break that functionality and implement a fix anyway rather than pause and get confirmation on the requirement. They may leave a note about the broken functionality at the end, but by that point the AI has already been iterating with an incorrect assumption. Any code the agent wrote in the same turn is going to be tainted by that assumption and it often does not remove or rewrite that code when given clarified requirements.

It's tempting to blame all AI issues on prompting because it's the only thing that we're able to control, but if a task falls into the long tail of work unable to be satisfactorily completed with AI, there's no prompt that will work. When there's always the alternative of doing the work yourself, it's not worthwhile to spend the tokens or time to get an AI model to do something after the first two or three prompts.