Llms only work in one direction, they produce the next token only. It can't go back and edit. They would need to be able to back track and edit in place somehow

▲

thunky 2 days ago | parent | next [-]

Loops.

Plus, the entire session/task history goes into every LLM prompt, not just the last message. So for every turn of the loop the LLM has the entire context with everything that previously happened in it, along with added "memories" and instructions.

▲

DANmode 2 days ago | parent | prev [-]

"Somehow", like caching multiple layers of context, like all the free tools are now doing?

▲

tayo42 2 days ago | parent [-]

That's different then seeing if it's current output made a mistake or not. It's not editing in place. Your just rolling the dice again with a different prompt

▲

thunky 2 days ago | parent [-]

No, the session history is all in the prompt, including the LLM's previous responses.

▲

tayo42 2 days ago | parent [-]

Appending more context to the existing prompt means it's a different prompt still... The text isn't the same

▲

thunky 2 days ago | parent [-]

I'm not sure what your point is?

Think of it like an append only journal. To correct an entry you add a new one with the correction. The LLM sees the mistake and the correction. That's no worse then mutating the history.

▲

tayo42 a day ago | parent [-]

thats not how it works.

You put in its context window some more information, then roll the dice again. And it produces text again token by token. Its still not planning ahead and its not looking back at what was just generated. There's no guarantee everything stays the same except the mistake. This is different then editing in place. you are rolling the dice again.

	▲	thunky a day ago \| parent [-]
		> its not looking back at what was just generated It is, though. The LLM gets the full history in every prompt until you start a new session. That's why it gets slower as the conversation/context gets big. The developer could choose to rewrite or edit the history before sending it back to the LLM but the user typically can't. > There's no guarantee everything stays the same except the mistake Sure, but there's no guarantee about anything it will generate. But that's a separate issue.