> And research articles indicate that when the model computes that it should employ sycophantism it becomes less useful in every other way, just like a real sycophant.

The end goal of a sycophant is to gain advantage with their flattery. If sycophant behavior gets Claude's users to favour Claude over other competing LLM services, they prove to be more useful to the service provider.

▲

AstralStorm 3 days ago | parent [-]

Until users find out it's less useful to the user because of that.

Or it causes some tragedies...

▲

pera 3 days ago | parent | next [-]

The problem is that the majority of user interaction doesn't need to be "useful" (as in increasing productivity): the majority of users are looking for entertainment, so turning up the sycophancy knob makes sense from a commercial point of view.

It's just like adding sugar in foods and drinks.

▲

vintermann 3 days ago | parent | next [-]

You're ... Wait, never mind.

I'm not so sure sycophancy is best for entertainment, though. Some of the most memorable outputs of AI dungeon (an early GPT-2 based dialog system tuned to mimic a vaguely Zork-like RPG) was when the bot gave the impression of being fed up with the player's antics.

	▲	motorest 3 days ago \| parent [-]
		> I'm not so sure sycophancy is best for entertainment, though. I don't think "entertainment" is the right concept. Perhaps the right concept is "engagement". Would you prefer to interact with a chatbot that hallucinated or was adamant you were wrong, or would you prefer to engage with a chatbot that built upon your input and outputted constructive messages that were in line with your reasoning and train of thought?

▲

astrange 3 days ago | parent | prev | next [-]

Not sure anyone's entertained by Claude. It's not really an entertaining model. Smart and enthusiastic, yes.

▲

pitched 3 days ago | parent | prev [-]

Some of the open models like kimi k2 do a better job of pushing back. It does feel a bit annoying to use them when they don’t just immediately do what you tell them. Sugar-free is a good analogy!

▲

kruffalon 3 days ago | parent | prev | next [-]

Well, aren't we at the stage where the service providers are fighting for verbs and brand recognition, rather than technological advances.

If there is no web-search, only googling, it doesn't matter how bad the results are for the user as long as the customer gets what they paid for.

▲

AznHisoka 3 days ago | parent | prev [-]

I doubt humanity will figure that out, but maybe I’m too cynical