The AIpocalypse is here for web sites as search referrals plunge
- Reference: 1750587247
- News link: https://www.theregister.co.uk/2025/06/22/ai_search_starves_publishers/
- Source link:
Google's AI-generated summaries of web pages, [1]officially released in May 2024 , show up atop its search results pages so search users don't have to click through to the source website.
A year later, enterprise AI analytics biz BrightEdge [2]reported that Google AI Overviews had generated more search impressions (up 49 percent), but click-throughs to the actual web sites dropped 30 percent
[3]
That means AI Overviews is leading more people to use Google Search to find answers to their queries. But those people are less likely to follow search results links that lead to the source website. Good for Google. Terrible for the ecosystem of web sites that had learned to depend on search referrals for buyers, readers, and viewers.
[4]
[5]
Kevin Indig, who writes about search engine optimization (SEO), marked the one-year anniversary of AI Overviews with a usability study. Based on data from the 70 individuals surveyed, he [6]observed that when AI Overviews are absent, "outbound click rates rise to an average of 28 percent on desktop and 38 percent on mobile."
Ahrefs, an SEO site, in April [7]said AI Overviews reduced clicks by about 35 percent.
[8]
Citing data provided by SimilarWeb (which SimilarWeb shared with El Reg , Barron's last week [9]reported that search referrals to top US travel and tourism have fallen 20 percent year on year, while news and media sites saw search-driven traffic drop by 17 percent during that period.
Other categories of website also showed declining search referral traffic: e-commerce (-9 percent); finance (-7 percent); food/drink (- 7 percent); and lifestyle/fashion (-5 percent).
Meanwhile, AI search engine referrals have replaced only about 10 percent of traditional search referral traffic, according to SimilarWeb.
[10]
While the stats vary depending on who's measuring, the story is consistent: web publishers, who provided the content that trained these AI models, face dramatically diminishing visitors, which means lower advertising and subscription revenues, even amid overall growth in search impressions.
This may help explain the [11]lawsuits web publishers have [12]filed against AI firms .
[13]New GitHub Copilot limits push AI users to pricier tiers
[14]US patent office wants an AI to scan for prior art, but doesn't want to pay for it
[15]American coders are most likely to use AI
[16]Nvidia bets on Gates-backed nuclear startup to keep its AI ambitions from melting down
The latest indication of this [17]widely reported trend comes from Matthew Prince, CEO of Cloudflare, which has been developing tools to thwart bots that scrape web data for AI training.
Speaking on Thursday at an Axios event in Cannes, France, Prince [18]reportedly said that ten years ago, Google's average ratio of pages crawled to visitors referred was 2:1. Six months ago, that ratio had increased to 6:1. Today, according to the Prince, it's 18:1.
While Prince did not clarify the breakdown of crawler duties (e.g. indexing, copying data for training, or something else), the suggestion is that AI companies are taking more content and giving less in return.
And that's allegedly the case for two other AI model makers that have implemented web search, [19]OpenAI and [20]Anthropic .
According to Prince, OpenAI's ratio of pages crawled to visitors referred has gone from 250:1 to 1,500:1 while Anthropic's ratio has gone from 6,000:1 to 60,000:1.
As we noted earlier this week, [21]AI crawlers have become a burden for many websites, which end up bearing the cost to serve content to AI firms for commercial AI services.
Despite much ado about the potential for an AI-based challenger to [22]disrupt Google , the Chocolate Factory still controls about 90 percent of the search market, according to BrightEdge. But it's eating all the cacao beans that made its business possible in the first place.
Google, OpenAI, Anthropic and Cloudflare did not immediately respond to requests for comment. ®
Get our [23]Tech Resources
[1] https://blog.google/products/search/generative-ai-google-search-may-2024/
[2] https://www.brightedge.com/news/press-releases/one-year-google-ai-overviews-brightedge-data-reveals-google-search-usage
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2aFgolXQ0AUJ0cQtTbLfdqQAAANE&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44aFgolXQ0AUJ0cQtTbLfdqQAAANE&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33aFgolXQ0AUJ0cQtTbLfdqQAAANE&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[6] https://www.growth-memo.com/p/the-first-ever-ux-study-of-googles
[7] https://ahrefs.com/blog/ai-overviews-reduce-clicks/
[8] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44aFgolXQ0AUJ0cQtTbLfdqQAAANE&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[9] https://www.barrons.com/articles/ai-google-search-internet-economy-932092ef
[10] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33aFgolXQ0AUJ0cQtTbLfdqQAAANE&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[11] https://www.theregister.com/2025/06/05/reddit_sues_anthropic_over_ai/
[12] https://www.theregister.com/2024/10/22/publishers_sue_perplexity_ai/
[13] https://www.theregister.com/2025/06/20/github_begins_enforcing_premium_request/
[14] https://www.theregister.com/2025/06/20/us_patent_office_ai/
[15] https://www.theregister.com/2025/06/19/us_coders_lead_world_in_ai/
[16] https://www.theregister.com/2025/06/18/nvidia_ai_smr_investment/
[17] https://www.wsj.com/tech/ai/google-ai-news-publishers-7e687141
[18] https://www.axios.com/2025/06/19/ai-search-traffic-publishers
[19] https://openai.com/index/introducing-chatgpt-search/
[20] https://www.anthropic.com/news/web-search-api
[21] https://www.theregister.com/2025/06/17/bot_overwhelming_websites_report/
[22] https://www.theregister.com/2024/01/30/ai_is_changing_search/
[23] https://whitepapers.theregister.com/
You live in dreamland. Not the Alice style, more one of the various Rick & Morty styles.
While I like the idea, so far it seems that those techbro companies are allowed whatever, with clear disregard for all laws (copyright and all). So .. no.
But should anybody illegally stream a single song...
"If the crawling is so problematic, and easily attributable, set up a robots.txt and take a DMCA case if they continue to crawl."
This. But I'd also be looking to add something akin to NOINDEX - NOAI maybe - to the headers, and a clear, human readable "not for use by AI" in every page footer.
That should make your position clear, and DMCA cases much easier. Some kind of cease and desire injunction might also be in order.
Personally, I don't care if the bots crawl my sites for training purposes. But this particular application of the resulting AI is a bit much. Note it's the application rather than the AI per se that I object to.
The obvious solution is for everyone to immediately add NOINDEX to every page of their site; Google will have to remove every page on the web from their index, and become irrelevant over night. But nobody is going to do that, because they still need traffic from Google.
Looks like the days of making web sites for fun and profit are nearly over.
Compliance with robots.txt is voluntary, not legally enforced. There's no good basis to sue just because someone ignored it.
Skip "AI" results in firefox
look up "firefox udm" and use "Search" in the tool-bar.
https://addons.mozilla.org/en-US/firefox/addon/udm14/
HTH
Re: Skip "AI" results in firefox
Or block class "bzXtMb M8OgIe dRpWwb" in your ad blocker
Re: Skip "AI" results in firefox
Or just don't use Google
This AI problem is already (at least) stage 2 of enshittification...
... where part of stage 1 was websites trying to suppress even the most trivial bits of content being spelled out in search results so that they can harvest more clicks. Like they also stopped linking to other websites just because they might lose visitors before they otherwise would be done clicking around. Both already was the beginning of the end of the web as we (well, some of us) once knew it...
It's funny that search engine AI is now attacking the state of the web after the ad-commercialisation which was the first great wave set out to kill web functionality and usability, i.e. the core of the web.
Of course, because this world is built on commerce and commerce alone, this attack in itself is just a next stage commercialisation attempt, another fun part of which is that its principle is to damage what it lives off. A classic parasitisation process.
Re: This AI problem is already (at least) stage 2 of enshittification...
The whole AI mess is clearly a major breakthrough in enshittification. It's probably the singularity of enshittification.
Eat the hand that feeds it
Google is eating the goose that laid the golden eggs.
We are entering the final stages of the total value extraction of the ecosystem: clear cutting the forest.
Be careful what you wish for
In 2023, Canada enacted the [1]Online News Act . If it sounds Orwellian, good. The government decreed that major search engines should pay the news sites that they link to.
To no one's surprise, except the Canadian government and the news sites themselves,, Google wasn't interested in paying sites that they were giving publicity to, but if they had to, they'd just not link to them. So, they simply [2]removed the links to those news sites .
Even less surprising, the smaller news outlets suddenly found their traffic drop by as much as 90%. It turns out demanding that big search engines pay for the right to link to you isn't really a compelling argument when there are literally millions of other news sites that they can just as easily link to for free.
[1] https://www.canada.ca/en/canadian-heritage/services/online-news.html
[2] https://www.cbc.ca/news/politics/google-canada-online-news-1.6892879
If the crawling is so problematic, and easily attributable, set up a robots.txt and take a DMCA case if they continue to crawl.
Given what the courts have ruled counts as an”effective” security measure, this should be a fairly straightforward case.