> You can interrogate the LLM. You can ask it all sorts of things. Tell it to justify its decisions. Tell it to give you supporting evidence. Tell it to give you options. Tell it to explain things in detail.

Hopefully you realize that it's not actually introspecting into its previous thought process, and instead it is just making up a plausible after-the-fact explanation that matches the format of similar explanations found in its training set?

I too often find these explanations enlightening, but please don't forget that you need to read all such responses as those of a pathologically lying psychopath desparate to trick you, not as a trusted expert who actually understands its internals.

Of course. But again this all comes down to how you use it.

If you start blindly accepting everything it says, good luck to you.

But you can do things like get one agent to do something. Spin up another agent to review and critique. Use different LLMs to critique. I’ve had the same agent write piece of code and then come back and say “there are bugs x, y and z”.

For most code, it’s going to do a fairly good job and tell you best practices, give you things to research by yourself.