| ▲ | jasonlotito an hour ago | |
> I've yet to find a way to get an LLM to actually follow the instructions in CLAUDE/Agents.md I'll give you a hint: do you rely on people to do the right things, or do you have automated unit, integration, and e2e testing? Do you have linters? Do you have static analysis that automatically runs and will block when violated? If you are verifying humans, why aren't you verifying LLMs? | ||
| ▲ | maccard an hour ago | parent [-] | |
Yes, and agents are generating code that fails those checks regularly. I wouldn’t accept a junior engineer saying “oh I just submit my code and see if the tests pass, let me know if they’re failing and I’ll fix them”. Pre commit hooks are disable-able and LLMs have shown they’re more than willing to say “tests were failing before this change” when they weren’t, or to use python scripts to self escape their sandbox restrictions. | ||