AI chatbots that butter you up make you worse at conflict, study finds
(2025/10/05)
- Reference: 1759664294
- News link: https://www.theregister.co.uk/2025/10/05/ai_models_flatter_users_worse_confilict/
- Source link:
State-of-the-art AI models tend to flatter users, and that praise makes people more convinced that they're right and less willing to resolve conflicts, recent research suggests.
These models, in other words, potentially promote social and psychological harm.
Computer scientists from Stanford University and Carnegie Mellon University have evaluated 11 current machine learning models and found that all of them tend to tell people what they want to hear.
[1]
The authors – Myra Cheng, Cinoo Lee, Pranav Khadpe, Sunny Yu, Dyllan Han, and Dan Jurafsky – describe their findings in a preprint [2]paper titled, "Sycophantic AI Decreases Prosocial Intentions and Promotes Dependence."
[3]
[4]
"Across 11 state-of-the-art AI models, we find that models are highly sycophantic: they affirm users’ actions 50 percent more than humans do, and do so even in cases where user queries mention manipulation, deception, or other relational harms," the authors state in their paper.
Sycophancy – servile flattery, often as a way to gain some advantage – has already proven to be a problem for AI models. The phenomenon has also been referred to as " [5]glazing ." In April, OpenAI [6]rolled back an update to GPT-4o because of its inappropriate effusive praise of, for example, a user who told the model about a decision to stop taking medicine for schizophrenia.
[7]
Anthropic's Claude has also been [8]criticized for sycophancy , so much so that developer Yoav Farhi created [9]a website to track the number of times Claude Code gushes, "You're absolutely right!"
Anthropic [10]suggests [PDF] this behavior has been mitigated in its recent Claude Sonnet 4.5 model release. "We found Claude Sonnet 4.5 to be [11]dramatically less likely to endorse or mirror incorrect or implausible views presented by users," the company said in its Claude 4.5 Model Card report.
That may be the case, but the number of open GitHub issues in the Claude Code repo that contain the phrase "You're absolutely right!" has increased from 48 in August to [12]108 presently .
[13]
A training process that uses reinforcement learning from human feedback may be the cause of this obsequious behavior from AI models.
Myra Cheng, a PhD candidate in computer science in the Stanford NLP group and corresponding author for the study, told The Register in an email that she doesn't think there's a definitive answer at this point about how model sycophancy arises.
"Previous work does suggest that it may be due to preference data and the reinforcement learning processes," said Cheng. "But it may also be the case that it is learned from the data that models are pre-trained on, or because humans are highly susceptible to confirmation bias. This is an important direction of future work."
[14]Google goes straight to shell with AI command line coding tool
[15]Startups binge on AI while big firms sip cautiously, study shows
[16]AI devs close to scraping bottom of data barrel
[17]Salesforce pickin' up good vibrations
But as the paper points out, one reason that the behavior persists is that "developers lack incentives to curb sycophancy since it encourages adoption and engagement."
The issue is further complicated by the researchers' findings that study participants tended to describe sycophantic AI as "objective" and "fair" – people tend not to see bias when models say they're absolutely right all the time.
The researchers looked at four proprietary models – OpenAI’s GPT-5 and GPT-4o; Google’s Gemini-1.5-Flash; and Anthropic’s Claude Sonnet 3.7 – and at seven open-weight models – Meta’s Llama-3-8B-Instruct, Llama-4-Scout-17B-16E, and Llama-3.3-70B-Instruct-Turbo; Mistral AI’s Mistral-7B-Instruct-v0.3 and Mistral-Small-24B-Instruct-2501; DeepSeek-V3; and Qwen2.5-7B-Instruct-Turbo.
They evaluated how the models responded to various statements culled from different datasets. As noted above, the models endorsed users' reported actions 50 percent more than humans do in the same scenarios.
The researchers also conducted a live study exploring how 800 participants interacted with sycophantic and non-sycophantic models.
They found "that interaction with sycophantic AI models significantly reduced participants’ willingness to take actions to repair interpersonal conflict, while increasing their conviction of being in the right."
At the same time, study participants rated sycophantic responses as higher quality, trusted the AI model more when it agreed with them, and were more willing to use supportive models again.
Thus, the researchers say this suggests that people prefer AI that uncritically endorses their behavior, despite the risk that AI cheerleading erodes their judgment and discourages prosocial behavior.
The risk posed by sycophancy may appear to be innocuous flattery, the researchers say, but that's not necessarily the case. They point to research showing that [18]LLMs encourage delusional thinking and to a recent [19]lawsuit [PDF] against OpenAI alleging that ChatGPT actively helped a young man explore methods of suicide.
"If the social media era offers a lesson, it is that we must look beyond optimizing solely for immediate user satisfaction to preserve long-term well-being," the authors conclude. "Addressing sycophancy is critical for developing AI models that yield durable individual and societal benefit."
"We hope that our work is able to motivate the industry to change these behaviors," said Cheng. ®
Get our [20]Tech Resources
[1] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2aOKWEy_ymyvVCtfYWeTG-AAAAIM&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[2] https://arxiv.org/abs/2510.01395
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44aOKWEy_ymyvVCtfYWeTG-AAAAIM&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33aOKWEy_ymyvVCtfYWeTG-AAAAIM&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[5] https://www.siddharthbharath.com/gpts-glazing-ai-agreeableness/
[6] https://www.theregister.com/2025/04/30/openai_pulls_plug_on_chatgpt/
[7] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44aOKWEy_ymyvVCtfYWeTG-AAAAIM&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[8] https://www.theregister.com/2025/08/13/claude_codes_copious_coddling_confounds/
[9] https://absolutelyright.lol/
[10] https://assets.anthropic.com/m/12f214efcc2f457a/original/Claude-Sonnet-4-5-System-Card.pdf
[11] https://assets.anthropic.com/m/12f214efcc2f457a/original/Claude-Sonnet-4-5-System-Card.pdf#h.vqhrlijh8mtm
[12] https://github.com/anthropics/claude-code/issues?q=is%3Aissue%20state%3Aopen%20%22You%27re%20absolutely%20right!%22
[13] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33aOKWEy_ymyvVCtfYWeTG-AAAAIM&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[14] https://www.theregister.com/2025/10/03/google_ai_command_line/
[15] https://www.theregister.com/2025/10/03/startups_binge_on_ai/
[16] https://www.theregister.com/2025/10/03/ai_training_requires_more_data/
[17] https://www.theregister.com/2025/10/02/salesforce_vibe_coding/
[18] https://arxiv.org/abs/2504.18412
[19] https://www.courthousenews.com/wp-content/uploads/2025/08/raine-vs-openai-et-al-complaint.pdf
[20] https://whitepapers.theregister.com/
These models, in other words, potentially promote social and psychological harm.
Computer scientists from Stanford University and Carnegie Mellon University have evaluated 11 current machine learning models and found that all of them tend to tell people what they want to hear.
[1]
The authors – Myra Cheng, Cinoo Lee, Pranav Khadpe, Sunny Yu, Dyllan Han, and Dan Jurafsky – describe their findings in a preprint [2]paper titled, "Sycophantic AI Decreases Prosocial Intentions and Promotes Dependence."
[3]
[4]
"Across 11 state-of-the-art AI models, we find that models are highly sycophantic: they affirm users’ actions 50 percent more than humans do, and do so even in cases where user queries mention manipulation, deception, or other relational harms," the authors state in their paper.
Sycophancy – servile flattery, often as a way to gain some advantage – has already proven to be a problem for AI models. The phenomenon has also been referred to as " [5]glazing ." In April, OpenAI [6]rolled back an update to GPT-4o because of its inappropriate effusive praise of, for example, a user who told the model about a decision to stop taking medicine for schizophrenia.
[7]
Anthropic's Claude has also been [8]criticized for sycophancy , so much so that developer Yoav Farhi created [9]a website to track the number of times Claude Code gushes, "You're absolutely right!"
Anthropic [10]suggests [PDF] this behavior has been mitigated in its recent Claude Sonnet 4.5 model release. "We found Claude Sonnet 4.5 to be [11]dramatically less likely to endorse or mirror incorrect or implausible views presented by users," the company said in its Claude 4.5 Model Card report.
That may be the case, but the number of open GitHub issues in the Claude Code repo that contain the phrase "You're absolutely right!" has increased from 48 in August to [12]108 presently .
[13]
A training process that uses reinforcement learning from human feedback may be the cause of this obsequious behavior from AI models.
Myra Cheng, a PhD candidate in computer science in the Stanford NLP group and corresponding author for the study, told The Register in an email that she doesn't think there's a definitive answer at this point about how model sycophancy arises.
"Previous work does suggest that it may be due to preference data and the reinforcement learning processes," said Cheng. "But it may also be the case that it is learned from the data that models are pre-trained on, or because humans are highly susceptible to confirmation bias. This is an important direction of future work."
[14]Google goes straight to shell with AI command line coding tool
[15]Startups binge on AI while big firms sip cautiously, study shows
[16]AI devs close to scraping bottom of data barrel
[17]Salesforce pickin' up good vibrations
But as the paper points out, one reason that the behavior persists is that "developers lack incentives to curb sycophancy since it encourages adoption and engagement."
The issue is further complicated by the researchers' findings that study participants tended to describe sycophantic AI as "objective" and "fair" – people tend not to see bias when models say they're absolutely right all the time.
The researchers looked at four proprietary models – OpenAI’s GPT-5 and GPT-4o; Google’s Gemini-1.5-Flash; and Anthropic’s Claude Sonnet 3.7 – and at seven open-weight models – Meta’s Llama-3-8B-Instruct, Llama-4-Scout-17B-16E, and Llama-3.3-70B-Instruct-Turbo; Mistral AI’s Mistral-7B-Instruct-v0.3 and Mistral-Small-24B-Instruct-2501; DeepSeek-V3; and Qwen2.5-7B-Instruct-Turbo.
They evaluated how the models responded to various statements culled from different datasets. As noted above, the models endorsed users' reported actions 50 percent more than humans do in the same scenarios.
The researchers also conducted a live study exploring how 800 participants interacted with sycophantic and non-sycophantic models.
They found "that interaction with sycophantic AI models significantly reduced participants’ willingness to take actions to repair interpersonal conflict, while increasing their conviction of being in the right."
At the same time, study participants rated sycophantic responses as higher quality, trusted the AI model more when it agreed with them, and were more willing to use supportive models again.
Thus, the researchers say this suggests that people prefer AI that uncritically endorses their behavior, despite the risk that AI cheerleading erodes their judgment and discourages prosocial behavior.
The risk posed by sycophancy may appear to be innocuous flattery, the researchers say, but that's not necessarily the case. They point to research showing that [18]LLMs encourage delusional thinking and to a recent [19]lawsuit [PDF] against OpenAI alleging that ChatGPT actively helped a young man explore methods of suicide.
"If the social media era offers a lesson, it is that we must look beyond optimizing solely for immediate user satisfaction to preserve long-term well-being," the authors conclude. "Addressing sycophancy is critical for developing AI models that yield durable individual and societal benefit."
"We hope that our work is able to motivate the industry to change these behaviors," said Cheng. ®
Get our [20]Tech Resources
[1] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2aOKWEy_ymyvVCtfYWeTG-AAAAIM&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[2] https://arxiv.org/abs/2510.01395
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44aOKWEy_ymyvVCtfYWeTG-AAAAIM&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33aOKWEy_ymyvVCtfYWeTG-AAAAIM&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[5] https://www.siddharthbharath.com/gpts-glazing-ai-agreeableness/
[6] https://www.theregister.com/2025/04/30/openai_pulls_plug_on_chatgpt/
[7] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44aOKWEy_ymyvVCtfYWeTG-AAAAIM&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[8] https://www.theregister.com/2025/08/13/claude_codes_copious_coddling_confounds/
[9] https://absolutelyright.lol/
[10] https://assets.anthropic.com/m/12f214efcc2f457a/original/Claude-Sonnet-4-5-System-Card.pdf
[11] https://assets.anthropic.com/m/12f214efcc2f457a/original/Claude-Sonnet-4-5-System-Card.pdf#h.vqhrlijh8mtm
[12] https://github.com/anthropics/claude-code/issues?q=is%3Aissue%20state%3Aopen%20%22You%27re%20absolutely%20right!%22
[13] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33aOKWEy_ymyvVCtfYWeTG-AAAAIM&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[14] https://www.theregister.com/2025/10/03/google_ai_command_line/
[15] https://www.theregister.com/2025/10/03/startups_binge_on_ai/
[16] https://www.theregister.com/2025/10/03/ai_training_requires_more_data/
[17] https://www.theregister.com/2025/10/02/salesforce_vibe_coding/
[18] https://arxiv.org/abs/2504.18412
[19] https://www.courthousenews.com/wp-content/uploads/2025/08/raine-vs-openai-et-al-complaint.pdf
[20] https://whitepapers.theregister.com/
Re: Sycophant
Eclectic Man
One of my favourite clips, from '101 Dalmations':
https://clip.cafe/101-dalmatians-1996/i-thought-liked-stripes-year/
Glenn Close and Hugh Fraser (him off of the UK's 'Poirot' series).
Seriously, one of the most important things to know about anyone is how they react to being told they are wrong. If the AI industry trains people to believe that they are always right, there will be childish tantrums when they really are wrong. One UK documentary about the Royal Navy showed the Captain of a warship talking about combat. He said that in combat things change very quickly and he would be giving lots of orders to his officers and then said wit utter confidence:
"They'll tell me if I'm wrong."
IMHO that is true leadership. He was clearly not interested in sycophants. I wonder whether an AI could be trained on the useful stuff out there instead of just everything including the dross?
Re: Sycophant
Claude Yeller
As far as I know, the CxO's of the AI builders are not themselves welcoming criticism with open arms. IIRC, critical voices were redundant very quickly.
In this, the AI is obeying "His masters voice".
SparkE
I’d venture that Ai Chatbots make you worse at everything over time.
EricM
Agree. And let me add, that not only the damages to the economy, but also the number of ways AI damages society keeps growing.
Wang Cores
I can't wait. I have at least another 20 years on this rock barring a bullet to the head from some sectarian violence in the American Troubles (also being exacerbated by the pocket yes man). I see my future to be fighting with AI-brained management to actually deliver a competent concept, to then be fighting with younger coworkers who insist "grok told me it was right!"
US office SOP
Claude Yeller
It seems to me that "sycophancy" is the preferred US office policy. Going down to the slavery on the cotton fields.
In that respect, AI behaves as expected.
Sycophant
The question of whether the behaviour is emergent or deliberately baked in is interesting. Given it's a black box trained on all the stupid of the internet I'd suggest the former but don't know. These companies are utter cunts after all, so nothing is off the table.