Additionally, if you are asking agents to write test, be very careful. They will write tautology tests.

They will mock things to no end. They will flat out REMOVE assertions (saying it's not needed). They can also write test to assert the wrong result.

You have to always review it, it's exhausting.

That is my experience as well. I write tests by hand because they really still can't do it properly.

And I think I know why - neither do most people.

This was my experience with some of the stupider models, but I haven't had Claude do this type of thing since I started using it for work about 6mo ago.

Claude will do some boneheaded things for sure, but it's pretty good about writing tests that are useful, and not removing or modifying tests just because they're in the way.

Claude is pretty bad about assuming that it couldn't have broken a test it didn't know about, as it has often told me "this is already broken on main" which is definitely NOT true.