| ▲ | lubujackson 4 hours ago | |
Reading the prompt is very interesting. I always wonder how they make these long-running prompts and I guess they literally just tell it to "keep going". After working with LLMs day-in, day-out an SWE for months, I feel like this could be greatly improved with something like a state machine of progress and proper orchestration. Instead of spinning up a ton of subagents to follow different paths, whip up some Markdown (or LaTex or whatever math-equivalent) to store summaries of attempted paths, and have the agent augment those docs. Leave a paper trail of what has been tried. Iterate on that paper trail and repeatedly examine it for untried alternatives. LLMs can construct, navigate and summarize exceptionally well. Why is anyone trying to make them "hold the whole thing in your head"? I may be completely off the mark here since I have no math background, but my intuition for how LLMs are able to build on understanding through an external context store makes me feel like this isn't much different than someone trying to one shot a 3D game with Fable Max for $10,000 when they could get the same, or better, result with more human intention. | ||
| ▲ | steveklabnik 2 hours ago | parent | next [-] | |
> I always wonder how they make these long-running prompts and I guess they literally just tell it to "keep going". Many harnesses support a /goal as well. When the agent thinks it's done, another LLM compares its results to the goal, and if not, tells it to keep going. It's quite easy to have agents working on something for hours this way. | ||
| ▲ | Miraste 3 hours ago | parent | prev | next [-] | |
What you're describing is similar to how the copilot harness in vs code tracks state and previous work. These systems are being implemented, bit by bit. | ||
| ▲ | perching_aix 3 hours ago | parent | prev [-] | |
I mean you can just ask them to do exactly that. Especially with GPT (5.5), I've been having a lot of issues with it just repeatedly stalling out. I had to build a quota monitoring skill so that it'd keep plowing forward until either the task was finished (in some way) or the quota budget was exhausted. I also had issues with the compaction. Codex seems to compact... weirdly, resulting in the agent becoming a newborn after each compaction event. Telling it to use a notes file is basically essential and self-evident. Now that I mention, I should probably refine this skill to monitor the context window fill as well, to work around this. | ||