Yes, I'm doing my own research on AI augmented pipelines https://blog.officefloor.net . I actually found most code quality tools look for bugs and complexity, but nothing much about cohesive erosion. The nice thing about this metric, is that it determine the files where the erosion is occurring. I turned it into a GitHub action to make it easier to access to get wider feedback on the metric. The GitHub action triggers on your merge request and tells you the files where erosion is occurring to refactor. This stops erosion before it gets too expensive to change (big refactors or rewrite). Yes, happy to work with others to get the metric into other tools.

▲

visarga 3 hours ago | parent | next [-]

I just dump all user messages from all sessions in a project into a flat .md file and have agents synthesize the user's intent. Then, using that extracted intent, the agents review code and tests. I call this a retro/reflection pass. It checks whether the code matches the intent and whether the tests match the code.

Compactly formatted user messages are something an agent can ingest in a few minutes, even if they are thousands of lines long. And the quality of those messages is great: they don't track what the agent does well, only what changes and what breaks.

Having this top-down view helps a lot. Usually, within a session and deep into a task, the agent loses the global perspective and optimizes for local success. I find it weird there is no harness that treats user messages as high value signal (except my own, of course, I have it, https://github.com/horiacristescu/playbook-harness).

▲

sagenschneider 2 hours ago | parent [-]

Keeping all the specifications and user discussion does create more context, which is useful for AI.

However, I'd bring in Brooks discussion on essential and accidental complexity. In other words, there being No Silver Bullet https://www.cs.unc.edu/techreports/86-020.pdf

The problem with specification and user discussion is they still have errors that code has. But unlike code, there are no tests to confirm correctness.

So now we have a definition of the system in a non-exact language with no ability to test to confirm it's correctness. The code holds the essential complexity and now we are adding accidental complexity on top to manage.

Again agree the specifications and user discussion provides context for the AI. However, a well written test suite provides similar context that can actually confirm correctness of the system.

However, saying all the above. Focus of ImpactGate ( https://impactgate.officefloor.net ) is about erosion of the code, not correctness.

	▲	visarga a few seconds ago \| parent [-]
		You usually don't know what you want upfront, in real life it is a stream of specification and steering.

▲

catlifeonmars 4 hours ago | parent | prev | next [-]

What exactly is “cohesive erosion”?

	▲	sagenschneider 4 hours ago \| parent [-]
		Comes from the basic Computer Science principals of High Cohesion and Low Coupling. High cohesion means the functionality of a component are closely related and focused on performing a single well defined task. Basically single classes for single purposes. Erosion of this is when classes start doing to many things, in the case of God classes. The Change Impact formula looks at a way of detecting when the cohesion is eroding and flagging it on a change (as the pull/merge request itself should generally be single focus cohesive change)

▲

apercu 4 hours ago | parent | prev [-]

I've never encountered that term before (cohesive erosion) but I like it, if I'm interpreting it correctly.

Do you mean like the hyper focus an LLM puts on the task in front of it so you end up with drift (duplicated concepts/multiple ways of doing things, terminology drift (e.g., now we have "customer" and "client"). That sort of thing?

	▲	sagenschneider 4 hours ago \| parent [-]
		When you think about a god class or god method, it occurs over time by adding more than a single responsibility. Yes, there are generally complex algorithms but they usually are not things developers write (imported from libraries). What is usually going on in the god class/method is that things keep getting added to it. These things should be separated out. So the cohesiveness of the class/method erodes into doing too many things. The idea of the Change Impact formula is to catch this early so you start refactoring to separate out into classes with single cohesive purposes. The problem with AI is it handles complexity really well and will happily keep piling changes into god classes/methods reaching ridiculous CC levels (have see over 200). Previously developers would get annoyed and do the refactor. But with AI these days, changes are happening faster. So Change Impact is to try to monitor the cohesive erosion.