Remix.run Logo
breakingcups 7 hours ago

My pet theory without any strong foundation is because OpenAI and Anthropic have trained their models really hard to fit the sycophantic mold of:

    ===============================
    Got it — *compliment on the info you've shared*, *informal summary of task*. *Another compliment*, but *downside of question*.
    ----------
    (relevant emoji) Bla bla bla
    1. Aspect 1
    2. Aspect 2
    ----------

    *Actual answer*

    -----------
    (checkmark emoji) *Reassuring you about its answer because:*

    * Summary point 1
    * Summary point 2
    * Summary point 3

    Would you like me to *verb* a ready-made *noun* that will *something that's helpful to you 40% of the time*?
    ===============================
It's gotta reduce the quality of the answers.
kridsdale1 6 hours ago | parent | next [-]

I suspect this has emerged organically from the user given RLHF via thumb voting in the apps. People LIKE being treated this way so the model converges in that direction.

Same as social media converging to rage bait. The user base LIKES it subconsciously. Nobody at the companies explicitly added that to content recommendation model training. I know, for the latter, as I was there.

m_mueller 6 hours ago | parent | prev | next [-]

Not the case with GPT-5 I’d say. Sonnet 4 feels a lot like this, but the coding and agency of it is still quite solid and overall IMO the best coder. Gemini2.5 to me is most helpful as a research assistant. It’s quite good together with google search based grounding.

Twirrim 3 hours ago | parent | prev | next [-]

Gemini does the sycophantic thing too, so I'm not sure that holds water. I keep having to remind it to stop with the praise whenever my previous instruction slips out of context window.

typpilol 6 hours ago | parent | prev | next [-]

Anthropic also injects these long conversation reminders that are paragraph upon paragraphs about safety and what not to do.

People have said it destroys the intelligence mid convo

kridsdale1 6 hours ago | parent [-]

Yes, but that’s their brand.

porridgeraisin 6 hours ago | parent | prev | next [-]

Oh god I _hate_ this. Does anyone have any custom instructions to shut this thing off. The only thing that worked for me is to ask the model to be terse. But that causes the main answer part to be terse too, which sucks sometimes.

typpilol 6 hours ago | parent [-]

Chatgpt has a setting where you can set the tone to robotic

viraptor 5 hours ago | parent | prev [-]

Not really. Any prefix before the content you want is basically "thinking time". The text itself doesn't even have to reflect it, it happens internally. Even if you don't go for the thinking model explicitly, that task summary and other details can actually improve the quality, not reduce it.