Remix.run Logo
synalx 8 hours ago

I have this problem even in codebases. Claude will work on something, and add detailed comments in which it extrapolates the from the design and confidently states intentions and decisions which aren't actually grounded in reality. Then, later sessions suffer when it reads back those hallucinations and treats them as canonical.

bartread 5 hours ago | parent | next [-]

Delete the comments: I’ve gone on a tear with these recently because they’re nothing but trouble.

The absolute least worst outcome is they chew up your token budget. But what tends to happen, and this is much more serious, is they poison future work and make further modification of the codebase more burdensome and error prone.

LoganDark 5 hours ago | parent | next [-]

Absolutely. IMO, comments can explain historical reasoning for the code, but refactorings can benefit heavily from rethinking ideas from scratch, as opposed to trying to follow the same reasons. LLMs tend to be misguided by comments, probably by treating them as instructions. Ergo, get rid of them.

acedTrex 4 hours ago | parent | prev [-]

I completely block all LLM comments via pi extension, it makes using them significantly more enjoyable. If the LLM wants to add a comment it must ASK me explicitly to do so.

Seeing LLM comments in other peoples code is very upsetting because theres just so much meaningless noise.

furst-blumier 44 minutes ago | parent | next [-]

Which extension is it?

copperx 4 hours ago | parent | prev [-]

What's the extension, please?

post-it 6 hours ago | parent | prev | next [-]

I've been whittling my way towards having Claude write zero comments in code. They're pretty much never helpful.

sdeframond 6 hours ago | parent [-]

How do you do it?

ch4s3 6 hours ago | parent [-]

You could probably add linting rules to your tool of choice and tell claude that your lint tool has to pass.

demibabs 5 hours ago | parent [-]

but then no human generated comments allowed either, right?

ch4s3 5 hours ago | parent [-]

I'd do this by only running the linter locally and using this as a custom rule for the LLM, but I need to think about it a bit harder.

devmor 5 hours ago | parent [-]

There is probably something you can do to only apply some rules to the actual changed lines in a git diff.

You may have to write your own linter for that specifically.

notsirius 4 hours ago | parent [-]

you'd probs want a commit hook for that

unleaded 2 hours ago | parent [-]

Does "don't write comments" not work?

jaggederest 5 minutes ago | parent [-]

about as effective as don't make mistakes... For anthropic's models at least

rco8786 5 hours ago | parent | prev | next [-]

We have this in our harness markdown:

```

## Comments

Use comments extremely sparingly. Most comments should be at the request of the user. When something warrants a comment, keep it to one or two lines: what the code does and why it's necessary. No background narrative, no replaying the investigation or failure mode, nothing a test name or the commit message already says. Applies to specs too. If a comment needs a paragraph, make the code clearer instead.

```

The comments Claude was leaving got absolutely out of control. Just lines and lines of LLM drivel that was barely intelligible and not remotely relevant to what a code comment should be used for.

preommr 4 hours ago | parent | prev | next [-]

I've had codex delete useful (albeit not directly relevant or perhaps messy wip notes) comments, even though I explicitly have it in my agents.md not to delete comments, and ask for permission if it thinks it should.

It deleted the comments, and when I asked why it did that even though I expressedly asked it not to, it responded that me prompting it in the first place explicit permission. I have no idea if that's the actual reason or just some post-hoc explanation.

But I genuinely don't think it's possible to just have these things be completely, 100%, indpenedent and also solve deep problems that need to also be understood by people in a people-based organization context.

recursive 3 hours ago | parent [-]

There might not be an "actual" reason that's expressible in human-comprehensible language.

coder-pm 4 hours ago | parent | prev | next [-]

My approach is to keep the comments only with a references. My LLM based projects always have the decision log where are my decisions while working on the features are stored with the date. I found this useful to actually trace why something is in the codebase. It's much easier for both me and the LLM to navigate through the big projects where I spent months and dozens of full night sessions executing my plans with --dangerously-skip-permissions. In the morning I was answering all the model questions and iterating like that. Honestly, try that.

7 hours ago | parent | prev | next [-]
[deleted]
mrtesthah 8 hours ago | parent | prev | next [-]

Probably good to establish canonical spec documents up front that it can use and maintain as an independent reference?

rendaw 6 hours ago | parent | next [-]

That doesn't solve the problem of it hallucinating more design details and putting them in comments. The solution to too much authoritative documentation isn't more authoritative documentation.

pc86 6 hours ago | parent [-]

Comments exist for humans to understand the why of the code, including perhaps external dependencies/assumptions/requirements.

LLMs inserting code comments makes zero sense by definition.

pavlov 5 hours ago | parent [-]

That’s not true because LLMs clearly benefit from having multiple statements representing aspects of the same thing.

That’s why they’re always doing the “it’s not X, it’s Y” kind of patterns. It’s reinforcement.

lupire 8 hours ago | parent | prev [-]

Doesn't need to be "up front", but you do need to curate the context.

AI can't read your mind, and it will read what you (or anyone else) wrote.

Sharlin 7 hours ago | parent | next [-]

Humans can’t read minds either, but most humans aren’t so dumb as to forget what is a normative spec and what is just their own current, evolving model of the problem space. (Though some humans certainly are…)

pydry 8 hours ago | parent | prev [-]

It can't but it will try and getting it to stop is often infeasible.

throwaway613746 8 hours ago | parent | prev | next [-]

Anthropic hopes you just eventually give up and become completely dependent on Claude. Toss in 10 (or more) other developers with way less discipline and you will become dissillusioned with Claude's overengineered, incomprehensible, technosalad and your demise is pretty much guaranteed.

recursive 3 hours ago | parent [-]

Retirement feels farther away than ever.

dan_gggggg 8 hours ago | parent | prev | next [-]

[dead]

bsjdhdudj 8 hours ago | parent | prev [-]

[flagged]

dan_gggggg 7 hours ago | parent [-]

> what the fuck is wrong with all of you?

They spent 3 years pretending that their magic robot made them 10x harder/better/faster/stronger than the lazy peons rubbing shoulders with them and now the bill has come due.