I don’t mean to pick on you in particular here, but this approach has been bothering me a lot lately, and it seems like it’s super common.

I get that it’s an early prototype and not all the design choices are made yet, but I struggle with “I can’t afford human-readable documentation yet”. Isn’t human readable documentation important for efficiently planning and deciding what you want to build? I feel like I can’t afford not to have human readable docs and plans while the project is taking shape

Related, if a user can prompt an agent to translate to a human-readable summary, wouldn’t it be better to just do this in place? Sure, models can deal with noisy LLM outputs but shouldn’t a document that’s easier for humans also be easier for bots?

No worries, I don't feel picked on at all.

Yes, it would probably be better to have something like that if you're struggling to plan and decide. In my particular case this has been a project that I have dreamt about and tinkered with for at least 15 years now. Most of the work I'm doing is just refinement of existing ideas what were already codified in other projects I've built.

Why don't I do this in place? The amount of churn is utterly incredible as ideas are refined and taken to the limit. These agents are getting quite good, but they regularly make the silliest of mistakes. You actually have to be extremely cautious about what you document because it often has a way of becoming "doctrine" to the agents. In fact, one of the most difficult parts of working on this project has been that the foundational properties cause the agents to lock down behavior WAY too early.