Yes I'm confused as well, it feels like it's still all prompting which isn't new or different in the LLM space.

It’s all just loading data into the context/conversation. Sometimes as part of the chat response the LLM will request for the client do something - read a file, call a tool, etc. The results of which end up back in the context as well.