Problem is, no matter how many sub-agents you break the LLM into it's still stochastic parrots all the way down. I've seen no evidence that compartmentalised harnesses can reach solutions that a monolithic harnesses cannot, so I'm thinking the compartmentalisation is just an excuse to throw more tokens into the token furnace - it doesn't unlock a step change in capability. If there is evidence to the contrary, I'd be interested in seeing it.

There have been some successful attempts to do science like this (https://github.com/AstroPilot-AI/Denario). Denario generated 1k+ "solutions" on its way to winning the FAIR Universe challenge, and they all had to be fed through a fitness function of some kind to assess them. It also got stuck on a local maximum and had to be kicked in the ass by a human to get it off that.

When writing software, that fitness function is relatively closed (when I run it, does it do what I want?). With something as open-ended as managing a business... might as well roll dice, IMO.

My belief is that 20's LLMs have a lot in common with 90's GAs: the more clearly we can define "success" for a given task (the fitness function) the more successful they can be. Open-ended problems are somewhat beyond them right now, and, I suspect, forever.

(The comment that's currently at the top of the discussion, about making an "AI Boss" that remembered everything about the company, and fed the user three tasks a day? That's not an AI Boss, that's an AI Assistant with different framing.)