> How to "understand" what to do?

How did you ever know? It's not like everyone always wrote perfect code up until now.

Nothing has changed, except now you have a "partner" to help you along with your understanding.

Well, I have a whole blog post of an answer for you: https://www.evalapply.org/posts/tools-for-thought/

Who "knows"?

It's who has a world-model. It's who can evaluate input signal against said world-model. Which requires an ability to generate questions, probe the nature of reality, and do experiments to figure out what's what. And it's who can alter their world-model using experiences collected from the back-and-forth.