Remix.run Logo
purplepatrick 4 hours ago

Agreed. CC’s comms capabilities have decreased gradually since 4.6, and it’s a real challenge. I think the issue is that what works well for code (succinctness) doesn’t work well in prosaic English.

CC’s communication violates almost every grammatical rule that’s tested on, say, the SAT. And yet I’m sure if you had Claude take the verbal section of the exam it would ace it.

Biggest issues: dense sentences, constant metaphors, abstractions, and seemingly no understanding of correct anaphora use. For example, “the x”, with x having not only no antecedent but also being a coined word or quasi-synonym for something that is already named in the code base. This gets compounded by its being unable to regress to a baseline (existing names in code) and instead anchoring on newer (vague or wrong) terms, for example, that crept in through a plan.

CC tells me this is because the speedy and precise fulfillment of a current task will trump every other tendency, so it adheres poorly to whatever “semantic baseline” the project represents.

Of course, it also has no concept of what context the user has and assumes that it must be the same it holds in its memory, which creates this “I didn’t know that you didn’t know” type of communication.

I have managed to wrangle some of these issues with a custom output style, but wish a pre-report hook were an option, as it could force CC to rewrite plan implementation take-aways…

Btw: Fable has the exact same issues, just somewhat less pronounced.

eterm 3 hours ago | parent | next [-]

I wonder if the odd phrasing is related to achieving the watermarking that was recently touted by Anthropic.

Mtinie 2 hours ago | parent [-]

Models before the announced date don’t have watermarking, so it’s unlikely. Now, if what you are interpreting is precursor work to develop the watermarking system, maybe?

I suspect it less insidious: Claude has/had the public sentiment of being the “better writer” of the models. At some point that distinction would have been diluted as other labs’ offerings “caught up” stylistically, unless Anthropic continued to tune their output…

I personally think they’ve pushed so far that they’ve overfit and lost the sweet spot they previously occupied.

an hour ago | parent [-]
[deleted]
saaaaaam 3 hours ago | parent | prev | next [-]

> Of course, it also has no concept of what context the user has and assumes that it must be the same it holds in its memory, which creates this “I didn’t know that you didn’t know” type of communication.

Yes, this is a repeated problem for me. It will drop something in as though we have discussed it before and when I say “hold on, what is this” it realises its error - though on more than one occasion has started to get snotty with me, or actually gaslighted me and pretended we had already discussed it. That was at what I assume must have been the edge of a context window in a very long chat though.

Mtinie 2 hours ago | parent [-]

I notice the models with reasoning can conflate “internal” (or subagent) discussions with external (i.e. me). So it is accurately indicating “I’ve had this discussion before” but incorrectly asserting who it was with.

My understanding of how “thinking”works is limited though, and given the reduced visibility into the thinking traces, it is harder to tell if this is actually happening or if these are imaginary discussions the model for some reason calcifies on.

saaaaaam an hour ago | parent | next [-]

Oh, that's interesting - because that's absolutely what's happening in my experience.

If I look at the thinking (which seems to have become unavailable in Opus 5 a lot of the time, but was present - and often useful - in 4.8/4.6) you're right - it's having the discussion with itself, and seems unable to distinguish that discussion from discussions with me. BUT it also seems to be related to the length of the chat - this seems far more likely to happen in a longer chat.

I don't understand why they have removed visibility into thinking - I found it very useful, not only for spotting things like this, but also because in more complex discussions it would often mention (useful) things in its train of thought that it dropped from its response - but if I said "when you were thinking, you mentioned this" it would then expand on that point. Taking that away is another thing that has negatively impacted the value I get from Opus 5.0 versus earlier models.

jampekka 2 hours ago | parent | prev | next [-]

"Thinking" is just normal model output that's hidden from user. In practice it's just stuff in a <reasoning> tag or similar that gets filtered out from the user view. And thus it suffers from the same injection problems where the model fails to properly take into account what was the "source" of which block of tokens.

purplepatrick 2 hours ago | parent | prev [-]

Yeah, basically everything that becomes context in a session will bias perception and communication style -- subagents, plan lingo, prompt lingo, etc. And then if you write a plan with the comms context having been biased, the lingo will creep into the plan, and from the plan into the code and code comments. And from there, bad lingo will go on multiplying like rabbits...

I usually think of it in terms of having a "good" or "bad" session. In a bad session, there is a harmful bias that you can only get rid of through a new session. For example, if you exposed too much context about, say, a variable that features prominently in a doc. The entire session will be anchoring on the importance of that variable. Or if you introduced the notion of CC having to ask for permission for stuff you will have a hard time getting it to "think on its feet" or propose an effective solution (you have made CC so insecure that it now relies on you even for little things that wouldn't normally require your input). In some cases (let's say you have important context in that session) you can overcome this by upping the reasoning level or switching to Fable, but usually a new session is the way to go.

Because it's so easy to bias the session I wouldn't even want to use any of these tools that pretend to give Claude "a brain" or "remember" things. That was en vogue a year ago and helpful then, but now, it's plain harmful IMHO. The key is to have just enough context.

Subagents often have the reverse problem in that they tend to have too little context to make "judgment calls", which is why the tasks for them must be either deliberately basic or mechanical in nature, or their output should be audited by the main session agent.

As for "thinking" it's not clear that that's even a thing (https://arxiv.org/abs/2510.24941)...

saaaaaam 42 minutes ago | parent [-]

> I usually think of it in terms of having a "good" or "bad" session. In a bad session, there is a harmful bias that you can only get rid of through a new session. For example, if you exposed too much context about, say, a variable that features prominently in a doc. The entire session will be anchoring on the importance of that variable. Or if you introduced the notion of CC having to ask for permission for stuff you will have a hard time getting it to "think on its feet" or propose an effective solution (you have made CC so insecure that it now relies on you even for little things that wouldn't normally require your input). In some cases (let's say you have important context in that session) you can overcome this by upping the reasoning level or switching to Fable, but usually a new session is the way to go.

I find this very interesting, particularly your points about "made CC so insecure". I know that we have a tendency to anthropomorphise around these tools, but I have definitely noticed instances where Claude becomes quite hysterical about things - and if you look in the thinking output, it's often after I've pushed back on something, or told it it is going in the wrong direction. It spends a lot of time in agonised second-guessing of itself, going round in circles, before outputting a cringeing hand-wringing response. It's very strange.

Good tip on upping the reasoning level - I've not tried this. I have tried switching to Fable though, which does help. But it obviously very hungry, particularly in longer chats because it presumably needs to remind itself of everything that has occurred so far in the chat.

The point you make about tools that pretend to give Claude "a brain" or "remember" things is also interesting - I find the "memory" feature in Claude so destructive to good outputs that when I'm using the chat interface I am very strict about using Projects, and usually turn off the project memory, or make efforts to manage the project memory and review and delete things that are skewing the outputs.

3 hours ago | parent | prev | next [-]
[deleted]
ambicapter 3 hours ago | parent | prev | next [-]

> being a coined word or quasi-synonym for something that is already named in the code base.

This annoys me with a lot of LLM code. They rename things for the hell of it all the time.

nailer 2 hours ago | parent | prev | next [-]

> I think the issue is that what works well for code (succinctness) doesn’t work well in prosaic English.

Hrm, I would have said the oposite. Succint language communicates without unnecessary clutter that could be a barrier to communication.

> Biggest issues: dense sentences, constant metaphors, abstractions, and seemingly no understanding of correct anaphora use.

And maybe you also agree? I'm confused about your preferred style of language.

zeafoamrun 3 hours ago | parent | prev | next [-]

[dead]

intrasight 3 hours ago | parent | prev [-]

Tell it to write like an engineer and comment like a programmer;)

But for the life of me, I don't get why anyone would care about the comments. All code is "machine language" now. The only document you should be reading is your spec.