Theres a deep insight in this post about the value of looping for throw away code to explore a problem space, rather than brute force a problem by just applying more tokens and hoping.

The more I play in this space, the more I’m drawn to the idea that some kind of back tracking constraint solver is a better solution than then the current naive while loop / brute force approach here.

The results I see are similar to what you get from a greedy brute force constraint solver; solves trivial problems, sometimes solves harder problems after a long time, takes too long to solve really hard problems; solutions are increasingly non optimal on average as complexity goes up.

We have so much existing knowledge about building good constraint solvers, if we could just figure out how to apply it here somehow.

> back tracking constraint solver

We'd need a LLM back tracking constraint solver and I think the bottleneck will be good grading rules on whether a direction is actually architecturally clean or extensible. I'm not sure how one would approach that problem