I wonder scaffolding synthesis is the way to go. Namely the LLM itself first reasons about the problem and creates scaffolding for a second agent that will do the actual solving. All inside a feedback loop to adjust the scaffolding based on results.

▲

modeless 5 days ago | parent | next [-]

In general I think the more of the scaffolding that can be folded into the model, the better. The model should learn problem solving strategies like this and be able to manage them internally.

▲

sixo 5 days ago | parent | prev | next [-]

I toyed around with the idea of using an LLM to "compile" user instructions into a kind of AST of scaffolding, which can then be run by another LLM. It worked fairly wellbfor the kind of semi-structured tasks LLMs choke on like "for each of 100 things, do...", but I haven't taken it beyond a minimal impl.

	▲	harshitaneja 5 days ago \| parent [-]
		I am working on something similar but with an AST for legal documents. So far, it seems promising but still rudimentary.

▲

plantain 5 days ago | parent | prev [-]

If you've ever used Claude Code + Plan mode - you know that exactly this is true.