Remix.run Logo
Remix clone Hacker News
new | show | ask | jobsGithub
▲ aDyslecticCrow a day ago

You've seen test-rot, specification rot, and documentation rot; we now introduce; prompt rot!

Cluttering the repo with out-dated, very wordy and quickly aging prompts will just confuse any agent tasked with looking at the repo in the future. Keeping context windows down is a real limitation to good LLM output, and this workflow may work completely against it.

- A plan.md describing the project, main abstraction idea, end costumer, and so on is great; but it should be kept minimal and up-to-date with the repo.

- Block comments on top of source-files and functions are great, and already very useful to coding agents. I don't see a value to anything more than what is already typical best practice.

▲xg15 a day ago | parent | next [-]

I wonder if instead of checking the prompts into the repo as files, a better idea would be to store them inside the commit messages.

If prompts are specifications for a change of the system's behavior, then it seems natural to manage them as changes and not as resources.

This would also keep them in the right "historical context" of the repo and avoid the "prompt rot" you were talking about.

▲aDyslecticCrow a day ago | parent | next [-]

Git history is a bit annoying to navigate, but that may just be a tooling issue. I've long been bothered by the loss of the review history when merging a PR. Would actually be pretty cool to click on a row of code and see the commit messages that formed that row of code in a little sidebar, and the technical discussions that were behind it.

Functional safety development processes often demand code-review, technical design decisions, changes of plans, or intentional compromises; to be linked together with reference IDs in the code they effect. But the workflow for this is usually extremely manual and absolute misery. But a codebase made like this is like magic to read later.

▲ffsm8 a day ago | parent | next [-]

Gits data structure already allows for that, as you can add additional data to a commit after it was made.

You'd consequently only need to implement your custom git gui and extension to visualize this information.

Maybe a good project for the next weekend? Just eg make a prototype tui in golang and see where you end up

▲jamietanna 19 hours ago | parent [-]

I've definitely seen at least one tool in the past - maybe via simonw - to use `git notes` to track that

▲michaelmure 19 hours ago | parent | prev | next [-]

I want to do something like that with git-bug (keeping track of prompt/conversation/review alongside normal commit), but unsure what the DX would be. Insights welcome :)

▲chapterjason 19 hours ago | parent | prev | next [-]

You literally described git blame usage in almost every editor which supports git properly.

▲aDyslecticCrow 9 hours ago | parent | next [-]

Yes and no. Git blame is great yes, but it's still rather crude. If three commits changed one condition; it only shows the top commit. Getting a full sense of the history of a function over time is far less ergonomic.

The tech and data-structure is there; but the common UX is not quite expressing the data in a sophisticated manner. It doesn't help that the default diff algorithm is rather crude as-well.

▲mschuster91 17 hours ago | parent | prev [-]

git blame falls apart on someone running a new linter config across the project or doing some refactoring work.

▲bulatb 12 hours ago | parent | next [-]

Some tools will automatically ignore commits in a .git-blame-ignore-revs file. Git itself can "blame --ignore-rev <hash>" or "blame --ignore-revs-file <file>" since version 2.23.

  git commit -m "Autoformat source files"
  git log --format="# %s%n%H" -1 >> .git-blame-ignore-revs
  git add .git-blame-ignore-revs
  git commit -m "Ignore autoformat commit in blames"

  cat .git-blame-ignore-revs
  # Autoformat source files
  0a1b2c3d...

  git blame --ignore-revs-file .git-blame-ignore-revs code.src
Git can use it automatically too:

  git config blame.ignoreRevsFile .git-blame-ignore-revs
  git blame code.src
Though git-blame may fail or crash if blame.ignoreRevsFile is set but the file is missing (hence "config", not "config --global"). Since version 2.53, a configured ignoreRevsFile can be missing if the value starts with ":(optional)".
▲ 13 hours ago | parent | prev | next [-]
[deleted]
▲matijsvzuijlen 15 hours ago | parent | prev [-]

In GitHub and vim at least you can dig past those commits to see earlier relevant commits. I'm sure there are more tools that allow this.

▲ a day ago | parent | prev [-]
[deleted]
▲boomlinde 10 hours ago | parent | prev [-]

I am skeptical of adding anything but a brief description of the change, the reason for the change and possibly some explanation of non-obvious implementation choices to the commit message.

The reason is that the commit message log serves as an overview of the changes commited. That's what humans use it for, anyway: to get an idea of what happened since they last pulled, to help give an idea of where a regression might have been introduced and so on, at a glance. To that end, brevity is very useful.

My understanding is that chatbots used to perform such tasks will also benefit from brevity.

▲xg15 9 hours ago | parent [-]

Well, at least in Git, the commit message already has two sections.

AFAIK, the first line is supposed to be the a very brief summary, while the other lines may contain additional information and can sometimes make up a pretty long text. Lots of tools make use of this convention and only show the first line if no detail information is needed.

Most tools that show the history only show the first line of each commit.

But I agree with you, this still assumes that long commit messages are rare and not that almost every commit has a huge message. Also, "long" doesn't mean you should put a novel in there.

▲boomlinde 5 hours ago | parent [-]

I think long is fine if that's necessary to get a rough idea of what the change is about across. Discerning in the choice of what information to include is perhaps a better way to put it. In those terms we should consider how likely it is for prompts to be immediately useful information when browsing the commit history, and whether it can't instead be reduced to an informative summary.

As far as specifications for changes go, the diff itself is as good a spec as it gets. It unambiguously describes the exact change that was made. The prompt you give a chatbot to make that change is IMO something else. Maybe it's more fair to call that a specification of the work you wanted it to perform.

▲nothrows 21 hours ago | parent | prev | next [-]

Couldn't agree more. Rot is rot. People hate reading giant verbose AI generated PRs. Imagine mandating your codebase requires people now also read AI generated markdown files to compare the AI intent with the AI outcome. What a laugh and this is the guy giving out CS degrees and opposing React.

▲jsw97 11 hours ago | parent | prev | next [-]

You could in theory require consistency of the spec with the product, but this is only valuable as a “source” if you ensure some kind of reproducibility. E.g., 9/10 times this spec + opus produces code that is equivalent, measured appropriately.

Ok I already talked myself out of this idea.

▲sajithdilshan a day ago | parent | prev | next [-]

I agree, because at the end of the day the source code is the source of truth and that's what the LLM should look at instead of some months old markdown file.

▲jve 20 hours ago | parent | prev | next [-]

My agent, whenever changes are made, happily keeps those .md files up-to-date with newest changes.

▲krab 20 hours ago | parent | next [-]

Mine doesn't. It mostly adds and keeps referencing previous decisions. Correctly, referring to them as being overridden. Even tweaking the original text or occasionally deleting a part. But text would look differently had it been written from scratch.

Unless a LLM works truly as a compiler, there will be a drift between the code and the spec.

▲aDyslecticCrow 19 hours ago | parent | prev [-]

For me they mostly append the end of the file with more sections or details. Heck it often adds very specific implementation details that it worked on recently that have no business being written in a repo wide summary.

▲jve 18 hours ago | parent [-]

Weird. I changed/amended model/integration and it updated table, provided explanation.

But yeah, I like to reference the docs so it can refresh his memory. But I may have had the .md files updated even if I didn't explicitly provide/ask for it, but a refactor that touched particular areas.

But yeah, my .md references others, it also maybe helps that my dir is named docs/ and it lists it somewhere in .md files... maybe its just the way it is structured helps infer agent what to do.

▲viccis a day ago | parent | prev | next [-]

Already happens in my experience. I'll try to figure out why it keeps doing this one thing and it turns out it's from some poorly advised info it put in a markdown file 20 commits ago that CLAUDE.md or AGENTS.md tell it to treat as gospel.

▲aDyslecticCrow a day ago | parent [-]

Weekly reminder to delete your memories folder that claude code loves creating over the most silly of information.

"yes claude i prefered the blue graph two months ago, how does thar help us with this json parsing bug?"

▲viccis 21 hours ago | parent [-]

Remember that 20 commits ago is like 12 hours ago

▲matchagaucho a day ago | parent | prev | next [-]

I think the OP is referring to less transient use cases, like the actual source code to agent system instructions.

▲ 5 hours ago | parent | prev [-]
[deleted]