How do you view harness engineering as an organic development that emerges from its use within a specific domain? Basically the meta-loop that allows an agent to tailor its harness to improve outcomes based on performance feedback. I use Pi a lot and I'm very interested in "self-assembling software".

One concrete example might be maintaining a conventions document per-project that covers how to name things semantically from a list of nouns and verbs. The idea is that LLMs are often not very globally aware, but it's important to maintain coherence across a code base in order for it to scale (in size and over time). Sometimes an LLM might call the same concept a Materialization, sometimes a Projection, and its not useful if its using two terms interchangeably without purpose.

Basically, how are you maintaining coherence when there isn't a human steering the code beyond providing requirements and validation directives?

I see you have relevant context in the repo like https://github.com/lopopolo/harness-engineering/tree/trunk/d... but I'm curious what exists beyond context. Do you use any tooling to steer this type of thing more consistently?

Your example is super amenable to vibing some tests. As an example, I’ve been able to ban `number` from representing a duration by walking the AST in a linter to fail if var or param names that look like the end in millis or ms or sec appear. This is largely good enough. If you see that “drifting” behavior appear more than once, you have enough to stop and force the agent to write some static verifiers that reject all but the option you want. For a closer example, we did this with zod schemes and their corresponding inferred types to be universally ZPascalCase and PascalCase instead of camelCaseSchema and CamelCase