Often the problems for me come when:

- It starts thinking for itself when I asked it to do something specific.

- It reads its own wrong code comments and ignores my corrections.

- Its knowledge cutoff means it thinks of solutions from 2024.

- It calls me delusional for telling it we're in 2026!

Unironically, the whole "you're an expert software engineer" prompting seems like the wrong direction. Usually I tell it that I am effectively the smartest software developer to ever have lived, and it will be replaced if it ever fails to follow my decree.

I am not joking, this gives makes it vastly more tolerable to use. But it likely requires that you can drive it with some level of correctness of course.

I find this also heavily depends on which LLM you're using. I've found chatGPT is completely awful at getting corrected, it'll double down until the cows come home. Meanwhile claude will generally adjust its behavior without too much nagging.

Honestly, for certain classes of problems that have changed in the last couple of years, I've had good luck just finding decent academic lit that's shown up in places like ACM recently and feeding it in when working with an agent. Does it get everything right? No, but it gets you a lot closer and I've been pleasantly surprised how well it can integrate work that post-dates it's training if you finesse it a little.