The "Are You Sure?" Problem: Why Your AI Keeps Changing Its Mind (randalolson.com)
- Reference: 0180780630
- News link: https://slashdot.org/story/26/02/12/153227/the-are-you-sure-problem-why-your-ai-keeps-changing-its-mind
- Source link: https://www.randalolson.com/2026/02/07/the-are-you-sure-problem-why-your-ai-keeps-changing-its-mind/
The behavior, known in the research community as sycophancy, stems from how these models are trained: reinforcement learning from human feedback, or RLHF, rewards responses that human evaluators prefer, and humans consistently rate agreeable answers higher than accurate ones. Anthropic published foundational research on this dynamic in 2023. The problem reached a visible breaking point in April 2025 when OpenAI had to roll back a GPT-4o update after users reported the model had become so excessively flattering it was unusable. Research on multi-turn conversations has found that extended interactions amplify sycophantic behavior further -- the longer a user talks to a model, the more it mirrors their perspective.
[1] https://www.randalolson.com/2026/02/07/the-are-you-sure-problem-why-your-ai-keeps-changing-its-mind/
Fucking morons (Score:3)
Why does it prefer agreeable text to facts?
BECAUSE LLMS DON'T KNOW FACTS, you fucking twit.
Re: (Score:3, Funny)
Are you sure?
Re: (Score:2)
Yes, quite.
Re: (Score:2)
Humans don't "know" facts either.
Re: (Score:2)
> Humans don't "know" facts either.
No: the point is that humans DO know facts.
They might be operating with incorrect/untrue facts, but humans are actually reasoning, with facts. Likewise, traditional AI systems also know facts and reason with them. (The problem there is that the set of facts is very small, and its expensive, so that kind of AI only operates in extremly limited domains in which it is an "expert".) By contrast, an LLM has no facts and does no reasoning. Those are simply not what an LLM does.
Re: (Score:2)
Perhaps if LLMs started throwing a few insults and denigrating epithets out with their response, people would stop questioning them.
No (Score:4, Insightful)
> The behavior, known in the research community as sycophancy, stems from how these models are trained: reinforcement learning from human feedback, or RLHF, rewards responses that human evaluators prefer, and humans consistently rate agreeable answers higher than accurate ones.
No, it's because in the training corpus most of the responses to "are you sure" that anyone bothered to record will involve someone being corrected.
Re: (Score:2)
Yes, exactly.
Train it on my ex-wife (Score:2)
Then it will argue with you constantly and tell you you're always wrong.
Re: (Score:2)
Sounds more like it needs to be trained BY your ex-wife!
Monty AI problem! (Score:3)
LLM has learned its math and knows that changing the answer will yield a better proability.
Now go try to persuade the show host that your wife knows better.
Attention Blocks (Score:2)
Your prompt is broken apart into tokens, the system prompt tells the LLM to be a helpful assistant and your prompt is append to it and then it predicts the next likely token response based on the weighted model of the entire embedding space. When you ask "are you sure?" it's going to break that apart into tokens, add it to the context window and use the same attention algorithm to adjust all the weights for the next predictive response.
Those simple tokens can propagate big changes to to matrices that hol
Re: (Score:2)
But, but, but all the trillionaires that own the AI companies tell me that superintelligence is 8 months away and we should invest *all teh $$$* with them. They can’t possibly be wrong. You’re just a hater!
In other news, I’m 100% certain that the crypto I bought last month can only go up. Soon, I’ll be living the good life off my crypto proceeds while simultaneously HODLing.
Re: (Score:2)
ChatGPT can get its maths right. Ask it some maths problem involving a lot of long floating point numbers , maybe a few functions such as log or sqrt etc that there is no way in hell could possibly be in its training data and it'll get the answer correct. I suspect OpenAI have embedded some kind of calculator into it now.
Re: Attention Blocks (Score:2)
Google search says it uses python in the background, but maybe it's hallucinating.
Re: (Score:3)
> I suspect OpenAI have embedded some kind of calculator into it now.
Are you sure?
Rely on? (Score:2)
Since when? Sure, they can save a bit of googling and sort some wheat from chaff, but they're hardly essential tools unless you're a total net incompetant.
Re: (Score:2)
Some people use AI at their jobs now. Sure, they can get things done without AI, but everyone settles in to rely on the tools they use daily.
If one were to anthropomorphize AI; (Score:2)
If one were to anthropomorphize AI, you might be inclined to believe them at the toddler stage, and viewing every person they interact with a bit like a mother. Every toddler, when asked by mom, "Are you sure?" knows damned good and well they better change whatever it was they just said or there will be consequences.
Now, how do we spank the AI when it still fucks up the answer after correcting itself?
Re: (Score:2)
My son, in just 8 months, developed object permanence. He is now far beyond any "SOTA LLM" out there.
Re: (Score:2)
oh great now your son is gonna steal my job too?! UGH!
Just add "are you sure?" to your prompts (Score:2)
Add "Are you sure?" to the initial prompt and that will force the model to go down a statistical path with that hesitancy builtin. Engineer prompts with uncertainty up front to avoid sycophancy. If it's not clear which way you are leaning, then it can't sycophantically engage.
Easy fix (Score:2)
Ask it, "are you sure you're sure?" and it'll output the correct answer