Open source maintainers are drowning in junk bug reports written by AI
- Reference: 1733819410
- News link: https://www.theregister.co.uk/2024/12/10/ai_slop_bug_reports/
- Source link:
Seth Larson, security developer-in-residence at the Python Software Foundation, raised the issue in a [1]blog post last week, urging those reporting bugs not to use AI systems for bug hunting.
"Recently I've noticed an uptick in extremely low-quality, spammy, and LLM-hallucinated security reports to open source projects," he wrote, pointing to [2]similar findings from the Curl project in January. "These reports appear at first glance to be potentially legitimate and thus require time to refute."
[3]
Larson argued that low-quality reports should be treated as if they're malicious.
[4]
[5]
As if to underscore the persistence of these concerns, a Curl project [6]bug report posted on December 8 shows that nearly a year after maintainer Daniel Stenberg raised the issue, he's still confronted by "AI slop" – and wasting his time arguing with a bug submitter who may be partially or entirely automated.
In response to the bug report, Stenberg [7]wrote :
We receive AI slop like this regularly and at volume. You contribute to [the] unnecessary load of Curl maintainers and I refuse to take that lightly and I am determined to act swiftly against it. Now and going forward.
You submitted what seems to be an obvious AI slop 'report' where you say there is a security problem, probably because an AI tricked you into believing this. You then waste our time by not telling us that an AI did this for you and you then continue the discussion with even more crap responses – seemingly also generated by AI.
Spammy, low-grade online content existed long before chatbots, but generative AI models have made it easier to produce the stuff. The result is pollution in [8]journalism , [9]web search , and of course [10]social media .
For open source projects, AI-assisted bug reports are particularly pernicious because they require consideration and evaluation from security engineers – many of them volunteers – who are already pressed for time.
[11]AI-generated bug reports are seriously annoying for developers
[12]Microsoft dangles $10K for hackers to hijack LLM email service
[13]Open source maintainers underpaid, swamped by security, going gray
[14]The graying open source community needs fresh blood
Larson told The Register that while he sees relatively few low-quality AI bug reports – fewer than ten each month – they represent the proverbial canary in the coal mine.
"Whatever happens to Python or pip is likely to eventually happen to more projects or more frequently," he warned. "I am concerned mostly about maintainers that are handling this in isolation. If they don't know that AI-generated reports are commonplace, they might not be able to recognize what's happening before wasting tons of time on a false report. Wasting precious volunteer time doing something you don't love and in the end for nothing is the surest way to burn out maintainers or drive them away from security work."
[15]
Larson argued that the open source community needs to get ahead of this trend to mitigate potential damage.
"I am hesitant to say that 'more tech' is what will solve the problem," he said. "I think open source security needs some fundamental changes. It can't keep falling onto a small number of maintainers to do the work, and we need more normalization and visibility into these types of open source contributions.
"We should be answering the question: 'how do we get more trusted individuals involved in open source?' Funding for staffing is one answer – such as my own grant through [16]Alpha-Omega – and involvement from donated employment time is another."
[17]
While the open source community mulls how to respond, Larson asks that bug submitters not submit reports unless they've been verified by a human – and don't use AI, because "these systems today cannot understand code." He also urges platforms that accept vulnerability reports on behalf of maintainers to take steps to limit automated or abusive security report creation. ®
Get our [18]Tech Resources
[1] https://sethmlarson.dev/slop-security-reports
[2] https://www.theregister.com/2024/01/04/aiassisted_bug_reports_make_developers/
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/devops&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2Z1gfVNFJjItPH3TcefDsOQAAAMA&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/devops&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Z1gfVNFJjItPH3TcefDsOQAAAMA&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/devops&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33Z1gfVNFJjItPH3TcefDsOQAAAMA&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[6] https://hackerone.com/reports/2887487
[7] https://hackerone.com/reports/2887487#activity-31380566
[8] https://reutersinstitute.politics.ox.ac.uk/news/ai-generated-slop-quietly-conquering-internet-it-threat-journalism-or-problem-will-fix-itself
[9] https://www.nytimes.com/2024/10/11/podcasts/ai-slop-bitcoin-hot-mess.html
[10] https://www.bloomberg.com/news/articles/2024-11-08/how-social-media-became-overrun-with-ai-slop
[11] https://www.theregister.com/2024/01/04/aiassisted_bug_reports_make_developers/
[12] https://www.theregister.com/2024/12/09/microsoft_llm_prompt_injection_challenge/
[13] https://www.theregister.com/2024/09/18/open_source_maintainers_underpaid/
[14] https://www.theregister.com/2024/07/15/opinion_open_source_attract_devs/
[15] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/devops&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Z1gfVNFJjItPH3TcefDsOQAAAMA&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[16] https://alpha-omega.dev/
[17] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/devops&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33Z1gfVNFJjItPH3TcefDsOQAAAMA&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[18] https://whitepapers.theregister.com/
re: Just like social media.
The difference being you can ignore social media with very little down side. Ignoring a bug report could be a lot riskier.
Re: re: Just like social media.
Some people can't ignore social media:
[1]Romanian court annuls first round of presidential election
Ruling follows revelation of declassified intelligence alleging Russia ran online campaign to promote far-right outsider
[1] https://www.theguardian.com/world/2024/dec/06/romanian-court-annuls-first-round-of-presidential-election
Re: re: Just like social media.
"Ignoring a bug report could be a lot riskier."
Not really. The dangerous bugs don't usually show up as singleton reports ... and the ones that DO show up as singletons are always reported by people who wouldn't use AI if it were mandated by the project in question. Wetware filters are usually sufficient.
Just say NO
to Social Media then you won't get your knickers in a twist about some trivial matter.
Social Media == SM == Sado Masochism
Give it up people and get a life outside the SM world. Look after your own mental health and not the bank balance of so called influencers.
A barrel of rotting fish...
used to conceal the stench of a decaying corpse.
I would be interested in the stats of the prevalence of this crap v. particular projects.
If it were it the case that projects that could provide enormous leverage through supply chain attacks or other vulnerabilities, or projects that were intrinsically security sensitive were being unduly targeted I would be deeply suspicious that an organised compaign was being waged by the usual state sponsored suspects and plain unsponsored criminals as a distraction.
The longer a real vulnerability remains undetected, unreported and unremedied the greater the opportunity for these malefactors to take advantage of the situation.
Re: the usual state sponsored suspects
Go on, tell us who's on this list. Who is the state sponsoring to fuck up OSS?
Re: the usual state sponsored suspects
You're talking to a bot, sabroni.
Re: the usual state sponsored suspects
The amusing thing here is that most of the usual state-actor suspects when it comes to little acts of cyber-vandalism are probably the least likely to be involved in this kind of thing. Russia? China? Iran? North Korea? All huge fans of open-source. Its not in their interests to sabotage it, not when it provides their cheapest and easiest means of achieving parity with their rivals. As for the idea that western governments could be behind it, surely their efforts would be better spent infiltrating the project, than on stupid stuff like this? It makes no sense.
No, this smacks more of either people chasing bug bounties for minimal effort, or trying to generate 'activity' to pad out a CV for a tech job, but again, with minimum effort. Or they're just trolling, which is motive enough, for some people.
Re: the usual state sponsored suspects
I haven't run across the trolling option as yet, but I have been expecting it.
Let's just say that when it happens, the blasters are set to kill. That one will get nipped in the bud before it has a chance to take off.
Yep. It's getting bad.
The biggest problem is management getting sold a bill of goods about so-called "AI" and thinking that they can take experienced coders off the job, because new graduates can do the same job with the help of AI.
On the bright side, so far the messes are rather small and easy to clean up ... but they are rapidly getting larger as upper management buys into pie in the sky that the various AI pushers are selling.
It's a major clusterfuck in progress.
I give folks one chance ... if they ship me an AI generated report (and they are almost always obvious), I'll take them aside and tell them not to do that, and why. If they send me a second one, I drop 'em on the floor as noise. Easier that way.
Re: Yep. It's getting bad.
Oops! Attached this to the wrong post. Sorry
"I am hesitant to say that 'more tech' is what will solve the problem,"
Simply blocking the submitters might.
While I sympathise with your viewpoint, the risk is that these idiots will then go round telling everyone that you 'unreasonably' blocked them for making a bug report, so obviously don't care that your work is full of errors. This sort of thing spreads through (anti)social media like wildfire.
I don't block them from anything.
I silently drop their reports in the bitbucket.
If it's important, there will be another in my inbox by and by, this time from a clued individual ... and usually before the submission from the clueless one.
"Recently I've noticed an uptick in extremely low-quality, spammy, and LLM-hallucinated security reports to open source projects," he wrote, pointing to similar findings from the Curl project in January. "These reports appear at first glance to be potentially legitimate and thus require time to refute."
Larson argued that low-quality reports should be treated as if they're malicious.
The question is: who's funding them? I wouldn't be surprised if there was a "grassroots organisation" funding this whose fundamental mission was to close as many open source projects as possible, especially those that compete with the ones that their funders run...
Cynical, moi? Yes.
"The question is: who's funding them?"
Nobody. It's newbies either trying to be "helpful", or trying to make their mark. They don't understand that folks with several decades of experience in OSS can smell bullshit before opening the email. They also don't understand that they have to learn to crawl before they can walk.
"Recently I've noticed an uptick in extremely low-quality, spammy, and LLM-hallucinated security reports to open source projects," he wrote, pointing to similar findings from the Curl project in January. "These reports appear at first glance to be potentially legitimate and thus require time to refute."
Just like social media. We're drowning under an avalanche of bullshit, each item takes time and energy to refute, but in the time taken to refute it 10 other pieces of nonsense have already gone viral. And the people sharing this nonsense don't really care... if that one turned out to be a lie then this next one must be true because it looks true.