AI Assistants Misrepresent News Content 45% of the Time (bbc.co.uk)
- Reference: 0179852542
- News link: https://news.slashdot.org/story/25/10/22/2011239/ai-assistants-misrepresent-news-content-45-of-the-time
- Source link: https://www.bbc.co.uk/mediacentre/2025/new-ebu-research-ai-assistants-news-content
> New research coordinated by the European Broadcasting Union (EBU) and led by the BBC has found that AI assistants -- already a [1]daily information gateway for millions of people -- [2]routinely misrepresent news content no matter which language, territory, or AI platform is tested . The intensive international study of unprecedented scope and scale was launched at the EBU News Assembly, in Naples. Involving 22 public service media (PSM) organizations in 18 countries working in 14 languages, it identified multiple systemic issues across four leading AI tools. Professional journalists from participating PSM evaluated more than 3,000 responses from ChatGPT, Copilot, Gemini, and Perplexity against key criteria, including accuracy, sourcing, distinguishing opinion from fact, and providing context.
>
> Key findings:
> - 45% of all AI answers had at least one significant issue.
> - 31% of responses showed serious sourcing problems - missing, misleading, or incorrect attributions.
> - 20% contained major accuracy issues, including hallucinated details and outdated information.
> - Gemini performed worst with significant issues in 76% of responses, more than double the other assistants, largely due to its poor sourcing performance.
> - Comparison between the BBC's results earlier this year and this study show some improvements but still high levels of errors.
The team has released a [3]News Integrity in AI Assistants Toolkit to help develop solutions to these problems and boost users' media literacy. They're also urging regulators to enforce laws on information integrity and continue independent monitoring of AI assistants.
[1] https://slashdot.org/story/25/10/06/1848254/chatgpt-now-has-800-million-weekly-active-users
[2] https://www.bbc.co.uk/mediacentre/2025/new-ebu-research-ai-assistants-news-content
[3] https://www.bbc.co.uk/aboutthebbc/documents/news-integrity-in-ai-assistants-toolkit.pdf%20
That's a step up! (Score:1)
The average human fouls it up 46%.
AI replacing thought (Score:2)
AI: It's a dangerous way of not thinking.
Re: (Score:2)
To be fair, we basically outsource everything, so why not the news & truth. 1984 (the book) was so prescient.
Re: (Score:3)
It was. It coined the term doublethink. I think that the MAGAs are full of it now days. I listened to a podcast yesterday on how they use the term "common sense" to justify all of the nonsensical things that they are doing. People believe it, and the phrase lost all meaning now. When MAGAs are doing something horrible like controlling the DOJ to prosecute anybody who disagrees with them, and pardons criminals who agree with them, they claim that the "other side" did it too. I see no evidence at all
Re: (Score:1)
> When liberals are doing something horrible like controlling the DOJ to prosecute anybody who disagrees with them, and pardons criminals who agree with them, they claim that the "other side" did it too.
FTFY.
You won't find evidence that Biden personally directed the DOJ to prosecute anyone because he spent his dementia years hiding from duty, but if you want evidence of his office doing it, that's an easy google search.
Re:AI replacing thought (Score:5, Insightful)
No I won't find evidence because Biden was a decent person. He was not "hiding from duty", he just was not a shit poster on social media.
Only 45% of answers contain any errors? (Score:2)
They should get AI to write the Slashdot summaries.
It seems like every criticism I hear about AI could also be applied to humans. Sometimes more so.
AI confidently gives an answer when it doesn't know? check!
Lack of transparency for the process of coming to a conclusion? check!
Rationalisation - explaining the reasoning for a conclusion retrospectively. check!
AI output is only bad if you go in expecting it to be perfect, and not checking the results.
These are amazing tools when used correctly. Complaining a
Re: Only 45% of answers contain any errors? (Score:2)
Was thinking same. This sounds exactly like a person. Honestly, I'm not sure why anyone is surprised at this point. It's pretty clear at this point that the flaws aka hallucinations in the the outputs are not mistakes, but looks like the normal functioning of the model.
Not just AI (Score:2)
> against key criteria, including accuracy, sourcing, distinguishing opinion from fact, and providing context. It would be interesting if you applied the same criteria to all news media. I suspect in the unlikely event the evaluation was unbiased you would get even worse results. Not because AI is reliable, but because a very large proportion of news media is not reliable. I doubt AI is even toward the bottom of the scale overall.
Re: (Score:2)
> against key criteria, including accuracy, sourcing, distinguishing opinion from fact, and providing context.
It would be interesting if you applied the same criteria to all news media. I suspect in the unlikely event the evaluation was unbiased you would get even worse results. Not because AI is reliable, but because a very large proportion of news media is not reliable. I doubt AI is even toward the bottom of the scale overall.
Re: (Score:3)
The difference is that news media skew intentionally, while "AI" is simply dumb.
Re: (Score:3)
I prefer the news that is broadcast and regulated by the pre-trumpian FCC. If they told provable, blatant lies, then they would be fined large amounts of money. Unfortunately outlets like FOX cable can tell provable, blatant lies only on cable, became popular to some groups. If you compare broadcast FOX news to cable FOX news, there is a large difference. I think that if CNN were broadcast, I don't think there would be any fines.
Hey, it is below 50%! (Score:2)
Totally usable and helpful, right?
Re: Hey, it is below 50%! (Score:2)
Monkeys throwing darts comes to mind...
What kinds of questions? (Score:2)
The details matter.
Were they asking:
- What is today's most important news?
- What is news from my country?
Or were the questions more specific, like:
- What caused the AWS outage Monday?
- Whatever happened to the couple caught on the jumbotron at the Cold Play concert?
I would expect AI to do much better with the latter, than the former.
Attributions vs. other kinds of errors (Score:2)
So of the 45% that had problems:
- 31% had attribution errors. Yeah, we know, AI is terrible at attributions.
- 20% had accuracy issues, including outdated information and hallucinated details. The proportion of these two types of errors is important. "Outdated information" is everywhere on the internet, AI or not. I wouldn't blame AI for that problem. Hallucinated details are a lot worse. What portion of the 20% was hallucinated? I'd say that something less than 20% having hallucinated details isn't as bad a
It's right 55%? (Score:2)
This is pretty amazing.
How does this compare to human readers?
Are the problems with AI or are the source articles the issue?
Here is the explaination: (Score:1)
Trump misrepresents the news 45% of the time, and his sycophants parrot him. AI is a reflection of what we all see.
Re: (Score:1)
But he's still popular. Humans are not Vulcans, they care about "owning the enemy", not logic.
Re: (Score:1)
I don't consider anybody who simply calls me names as: "owning me", however, I totally get your point. They are feel good voters who voted away their food, jobs, and healthcare. I disagree that he is popular considering that the percentage of total Trump voters are less than 33% of the total population. What do the other 67% think? Who would they vote for?
Re: (Score:3)
> the percentage of total Trump voters are less than 33% of the total population. What do the other 67% think? Who would they vote for?
I think every ballot should have a "none of the above" option. If that wins, they have to redo the election with all new candidates.
Re: (Score:2)
> I think every ballot should have a "none of the above" option. If that wins, they have to redo the election with all new candidates.
It does. You can choose not to vote and that is usually the largest proportion of eligible voters.
> If that wins, they have to redo the election with all new candidates.
Almost no one would ever get elected if we required a majority to win.
Re: (Score:3)
Yes, but not voting only empowers the fanatics who do vote, and I think people would still get elected. Australia requires all eligible voters to vote. They do have the option to cast a blank ballot, but they do have to show up. There is a fine for failing to vote. At least that would require a majority of those you had even a small tendency one way or the other. I also would like to know, why election day (at least the ones with federal elections) are not automatic holidays in the US.
Re: (Score:3)
Actually everyone voting would have a huge effect on campaigns, if not the outcome. The primary goal of political campaigns is to turn out supporters because most elections are decided by who turns out. That means motivating people to action, not persuading them to make a particular choice.
If everyone voted, the campaigns would have to persuade people with reasoned arguments rather than activate them with emotional appeals. Those emotional appeals are usually based on invoking negative emotions of fear and
Re: (Score:2)
I love the idea, but it will never happen, of course. It is not in the interest of the political caste.
Re: (Score:2)
I would like to point out that the Democratic party tends to pass laws and encourage more people to vote, and the Republican Party tends to pass laws and fear monger to get fewer (usually white racists) people to vote. I personally just "feel better" when laws expand my and others freedoms, both politically, and economically. I encourage each individual to look at the laws each party passes, and think: "does this expand my rights? or make my living situation better?". Don't listen to what they say, l
So about 33% think Trump is a God (Score:2)
I mean that literally. They think he is the second coming of Jesus Christ.
Another third of the country thinks he's just a fucking monster and a fascist. We call these people right.
The remaining 1/3 just want somebody to make the price of eggs and rent go down and they literally do not give a shit what happens to anyone or anything as long as that happens. They are just barely making it day by day and they're very close to collapsing.
It's like the old saying, you are always closer to being homele
Re: (Score:2)
Perhaps a simpler explanation is that AI is poisoning AI datasets.
PS, a party focused on jailing its opponents, removing their opponents (or even same party candidates) from the ballots, censoring speech, and installing a non-democratically elected queen into the presidency, and resorting to assassinating - and celebrating the assassination of rivals - is the closest thing to facism that the U.S. has come.
While the gaslighting from the fascists that everyone ELSE is ACTUALLY THE FASCIST is always both iron
Re: (Score:2)
I think that is BS. A large proportion of Americans don't think it matters who wins. They don't much care what bathroom transvestites use. They want a secure retirement, quality affordable health care, a tax system that doesn't reward the rich at their expense, a higher education system that leaves graduates in debt without the skills to get a job to pay it off, a judicial system that isn't only usable by the rich and powerful at their expense, a plan for reducing climate emissions that doesn't put the burd
It's not just owning the libs (Score:2)
That is definitely part of it though.
But there is a fuck ton of propaganda out there.
Also a lot of people cannot tell the difference between reality and television.
I know this first hand. My mother couldn't. She was bizarrely influenced by television. I think the most ridiculous example of that was she watched an old religious movie called brother son sister Moon and decided she was a peasant. But she would do it in all sorts of other contexts.
Probably the most embarrassing was she would wat
Re: (Score:2)
Which is simply utterly dumb if you have to continue sharing a country. Or a planet.