Google Search results polluted by buggy AI-written code frustrate coders
- Reference: 1714548728
- News link: https://www.theregister.co.uk/2024/05/01/pulumi_ai_pollution_of_search/
- Source link:
This mess started with Pulumi's decision to publish the result of its users' prompts on a curated AI Answers page. Google's crawlers indexed the resulting robo-responses – but when users find them, the AI answers are often inaccurate.
"It has happened," wrote developer Arian van Putten in a social media [1]post over the weekend. "The number one Google result was an official Pulumi documentation page that was clearly written by an LLM (it had a disclaimer that it was) and hallucinated an AWS feature that didn't exist. This is the beginning of the end."
[2]
As The Register [3]opined in 2022 and [4]reported in January this year, search quality has declined because search engines index low-quality AI-generated content and present it in search results. This remains an [5]ongoing area of concern.
[6]
[7]
[8]Pulumi AI and its online archive of responses, [9]AI Answers , is a case in point. Google's search crawler indexes the output of Pulumi's AI and presents it to search users alongside links to human-authored content. Software developers have found some of the resulting AI-authored documentation and code inaccurate or even non-functional.
The problem was noted on March 21, 2024 by developer Pete Nykänen in a GitHub Issues [10]post to the Pulumi AI code repository. "Today I was googling various infrastructure related searches and noticed a worrying trend of Pulumi AI answers getting indexed and ranking high on Google results, regardless of the quality of the AI answer itself or if the question involved Pulumi in the first place. This happened with multiple searches and will probably get even worse as the time goes on."
[11]
Others have also [12]raised the issue .
A rising tide of muck
Nykänen told The Register in an email that he began noticing Pulumi AI search result issues around the time he posted to GitHub last month.
"As an engineer, I spend a lot of time searching for answers online and it was not difficult to notice the AI answers rising to the top of the search results overnight, even for keywords unrelated to Pulumi itself," he noted. "I filed the issue and hoped that Pulumi would rectify the situation (which they promised to do) but sadly the issue still persists."
"Documentation, especially infrastructure related, is already often incorrect, hard to find, outdated or otherwise missing. While tools like Pulumi AI can provide value to some, filling the internet with unconfirmed, possibly hallucinated, answers is actually pretty malicious. And the longer it goes on, the worse it gets."
Nykänen argued that with AI content already appearing at the top of search results and more companies creating content generation tools, he hopes that those involved in AI consider how their work impacts the integrity of the web.
[13]
"I don't think it's too late for Pulumi either and hopefully they will decide to hide their AI generated content from search engine scrapers," he suggested.
Aaron Friel, an AI engineer at Pulumi, acknowledged Nykänen's concerns, responding the following day that the developer has "taken steps to remove more than half (almost two thirds) of AI Answers, and we plan to continue to ensure that these AI answers are complementary to our existing documentation."
Friel noted that Pulumi also plans to make sure its site mentions real APIs and upstream documentation. Testing generated code is also on the to-do list.
Hello? Google?
That was a month ago, and Google hasn't yet gotten the memo. When The Register tried the keywords cited by Nykänen on Monday "aws lightsail xray" – [14]Pulumi AI's answer was the second search result. And when we tried again on Tuesday, it ranked at the top of the page – above the official AWS documentation.
We asked Google what it thought of the situation and a company spokesperson told us it "always aims to surface high quality information, but on some niche topics or unusual queries, there may not be a lot of high quality content available to rank highly in Search."
The search giant also reminded us that it policies mean "Low value content that’s created at scale to manipulate Search rankings is spam, however it is produced", and that recent updates to its tech "reduced low quality, unoriginal content on Search by 45 percent, and aim to tackle unhelpful content that’s designed to rank well in Search."
Microsoft's Bing search engine could be ahead of the game in terms of filtering AI-generated material as it did not have this problem for the same query, though results it produced included a Chat button that launched an AI-generated response if you took the bait and clicked rather than just hitting return to submit the query. Brave Search also omitted the Pulumi AI response. DuckDuckGo, meanwhile, returned the Pulumi AI result as the fourth item on its search results page for the [15]query .
[16]Open Source world's Bruce Perens emits draft Post-Open Zero Cost License
[17]European Commission starts formal probe of Meta over election misinformation
[18]Politicians call for ban on 'killer robots' and the curbing of AI weapons
[19]OpenAI slapped with GDPR complaint: How do you correct your work?
Another GitHub Issue [20]post on Monday, referring to van Putten's complaint, has asked for the removal of Pulumi AI's [21]answer about AWS EBS direct APIs – which Pulumi evidently does not support.
Several AI hallucinations flagged in March have [22]already [23]been [24]dealt with .
In an email to The Register , Pulumi co-founder and CEO Joe Duffy defended his firm's AI effort – but allowed that more drastic intervention might be called for if the issue can't be adequately addressed.
"Pulumi AI has transformed how most of our customers work, enabling them to navigate a sea of hundreds of clouds with the myriad ways you can use all of their services," Duffy explained. "We processed a 50 percent increase in prompts quarter on quarter, which is a testament to how useful our customers are finding it to their daily work."
A startup that promises to do better ...
Duffy claimed that Pulumi has tested and improved its code quality over time and has seen a double-digit improvement in the success rates for code examples quarter over quarter.
"That said, we know these aren't perfect," he conceded. "Because our AI answers are indexable by Google, they show up in search results. I'll be the first to admit, I was surprised at how highly Google is ranking these pages, since in general they have no inbound links – a far cry from how PageRank used to work – and I would have expected it to prefer our older, more mature content."
Asked when Pulumi first realized its AI had issues, Duffy acknowledged Pulumi has been aware its AI isn't perfect since it launched last year, and has invested to improve its quality.
"We have a new typechecker loop that feeds back into the AI and improves our results," he explained. "We've tweaked it to be better at Python, and we've taught it about our cloud SDKs. All of these have had material increases in quality – and it will just keep getting better from here. Although there's been some negative sentiment on social media, far and away the feedback we get directly is that the AI is helpful, especially when just getting started in the cloud – it truly is daunting to even get started navigating hundreds of clouds each with tens of thousands of services."
Duffy revealed that Pulumi has already removed 100,000 AI answers and will take down more in future.
Despite the challenges, Duffy expects AI will improve over time. "We move fast and try innovative new ideas regularly – and sometimes they just don't work out the way we intended," he admitted. "If we can’t get to a good place quickly, we will absolutely consider delisting all of them and building back up more slowly."
Duffy added that Pulumi's AI Answers clearly state that they're the product of AI. "Despite the hallucinations, we regularly hear 'Even if imperfect, we prefer to have something 80 percent correct, [rather] than nothing at all'." ®
Get our [25]Tech Resources
[1] https://x.com/ProgrammerDude/status/1784833971731223033
[2] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2ZjISw4RiKVa8OBPKasmbRQAAAEI&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[3] https://www.theregister.com/2022/12/06/internet_ai_gpt_ios/
[4] https://www.theregister.com/2024/01/17/google_search_results_spam/
[5] https://www.reddit.com/r/artificial/comments/1c7fvam/ai_has_made_google_search_so_bad_people_are/
[6] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZjISw4RiKVa8OBPKasmbRQAAAEI&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[7] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZjISw4RiKVa8OBPKasmbRQAAAEI&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[8] https://pulumi.com/ai
[9] https://www.pulumi.com/ai/answers
[10] https://github.com/pulumi/pulumi-ai/issues/79
[11] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZjISw4RiKVa8OBPKasmbRQAAAEI&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[12] https://www.reddit.com/r/pulumi/comments/1avos5m/your_seo_spam_with_your_ai_makes_your_tool/
[13] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZjISw4RiKVa8OBPKasmbRQAAAEI&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[14] https://www.pulumi.com/ai/answers/bLHAi4DutXJvbJyNngGRvS/optimizing-aws-lightsail-and-x-ray-deployment
[15] https://duckduckgo.com/?va=g&t=hk&q=aws+lightsail+xray&ia=web
[16] https://www.theregister.com/2024/04/30/bruce_perens_post_open_license/
[17] https://www.theregister.com/2024/04/30/european_commission_launches_proceedings_meta_misinformation/
[18] https://www.theregister.com/2024/04/30/kill_killer_robots_now/
[19] https://www.theregister.com/2024/04/29/openai_hit_by_gdpr_complaint/
[20] https://github.com/pulumi/pulumi-ai/issues/83
[21] https://www.pulumi.com/ai/answers/e2Zg4j16E4Wnd6cUhN55ML/unlocking-aws-ebs-snapshot-capabilities-with-typescript
[22] https://github.com/pulumi/pulumi-ai/issues/68
[23] https://github.com/pulumi/pulumi-ai/issues/76
[24] https://github.com/pulumi/pulumi-ai/issues/75
[25] https://whitepapers.theregister.com/
Well there goes the neighborhood
we regularly hear 'Even if imperfect, we prefer to have something 80 percent correct, [rather] than nothing at all'."
Great. Looks like Borkzilla has always been ahead of the curve after all.
So, this is the future of programming. This explains Crapita, Fujitsu et al. to a tee.
I am not reassured.
Hopefully
This will mean "professionals" get more work* at better rates. Though I would want even more if the task was to sort out an AI-generated mess...
* Assuming they can be found.
Re: Well there goes the neighborhood
Something that "works 80%" has a one in five chance of borking something important. It reminds me of the old days of PCs claiming 95% IBM-PC compatibility. These machines invariably ran into the remaining 5% causing important programs to crash,
Is anyone in any way even slightly surprised?
This was inevitable from the start.
A visit to an insanitary river - y'know, the sort where they just dump in the sewage - will show you what always floats to the top... and the more of it there is, the thicker the floating layer is.
'the AI answers rising to the top of the search results overnight'
People spend years trying to get their own websites higher up the search results, in a legitimate manner.
What's the %$@&%£ point!
"We move fast and try innovative new ideas regularly"
Translation:
+++ "We throw any old crap at it and see what sticks to the wall"
> Despite the challenges
+++ Despite our total lack of ability to even test before we publish
> Duffy expects AI will improve over time
+++ damn it, we must get lucky someday! More monkeys, more typewriters!
+++ Despite our total lack of ability to even test before we publish
Testing code before publishing: now there's an idea!
In our programming courses, we have automated test suites for answers to programming exercises, so by the time an answer reaches the TAs or lecturers we know it meets some basic functionality standards. Setting up such a test suite does require actually understanding the problem , which current LLMs clearly don't, seeing how they happily hallucinate all sorts of non-existent stuff.
"The number one Google result was an official Pulumi documentation page"
Er.... Nope. The first half page or more of search "results" from Google would be a stream of nearly but not quite completely irrelevant and utterly disinteresting adverts and sponsored results.
Oh, what a massive surprise...
Just ran the test for "AWS Lightsail xray" in private mode and with AdBlock :
- Google -> first
- Bing -> first
- DDG -> seven
- Kagi (no private mode obviously) -> fourteen
And I just blocked the domain on Kagi so I won't be bothered with it in the future.
The real issue, like many people spoke about already, is about the fact that current/future models are trained on content produced by other models (and probably their own older selves), which is basically inbreeding and will make it harder and harder to access to actual good content.
The root cause maybe?
" The real issue, like many people spoke about already, is about the fact that current/future models are trained on content produced by other models "
It quite possibly goes deeper than that -- down to product and service vendors not offering real support any more. This leaves users to the mercies of third party advice offerings and quality-blind search tools, so verifiable reliability inevitably falls through the floor. If the primary source were accessible to deal with these queries, the problem would not occur. The use of AI merely escalates a problem that has been exemplified for ages by crowd sourced advice sites such as Stack Exchange.
If you still actually use google
If you want to clean up your google searches to clear out a lot of the AI generated bollocks and disinformation.
Use the following at the beginning of your search
"before:2023"
That should filter out a lot of the SEO & AI bullshit.