News: 0180029100

  ARM Give a man a fire and he's warm for a day, but set fire to him and he's warm for the rest of his life (Terry Pratchett, Jingo)

Wikipedia Urges AI Companies To Use Its Paid API, and Stop Scraping (techcrunch.com)

(Monday November 10, 2025 @05:20PM (msmash) from the tussle-continues dept.)


Wikipedia on Monday laid out a simple plan to ensure its website continues to be supported in the AI era, [1]despite its declining traffic . From a report:

> In a blog post, the Wikimedia Foundation, the organization that runs the popular online encyclopedia, called on AI developers to use its content "responsibly" by ensuring its contributions are properly attributed and that content is [2]accessed through its paid product , the Wikimedia Enterprise platform.

>

> The opt-in, paid product allows companies to use Wikipedia's content at scale without "severely taxing Wikipedia's servers," the Wikimedia Foundation blog post explains. In addition, the product's paid nature allows AI companies to support the organization's nonprofit mission. While the post doesn't go so far as to threaten penalties or any sort of legal action for use of its material through scraping, Wikipedia recently noted that AI bots had been scraping its website while trying to appear human.



[1] https://news.slashdot.org/story/25/10/17/0931209/wikipedia-says-ai-is-causing-a-dangerous-decline-in-human-visitors

[2] https://techcrunch.com/2025/11/10/wikipedia-urges-ai-companies-to-use-its-paid-api-and-stop-scraping/


Do they Need More Money? (Score:1)

by dbialac ( 320955 )

Take a look at the size of Wikipedia's bank account. They constantly continue to solicit for funds as though they're desperate for funds on their site despite having billions upon billions of funds, enough to last pretty much off of the interest alone.

Re: (Score:1)

by BrendaEM ( 871664 )

Work in AI, eh?

Re:Do they Need More Money? (Score:4, Interesting)

by swillden ( 191260 )

>> Take a look at the size of Wikipedia's bank account. They constantly continue to solicit for funds as though they're desperate for funds on their site despite having billions upon billions of funds, enough to last pretty much off of the interest alone.

> Work in AI, eh?

So... you didn't actually look at the size of WikiMedia Foundation's bank account.

WikiMedia absolutely has enough money to run Wikipedia indefinitely if they treated their current pile of money as an endowment and just used the income from it to support the site. They don't have "billions upon billions", but they [1]do have [wikimediafoundation.org] almost $300M, and they spend about $3M per year on hosting, and probably about that much again on technical staff to run the site, so about $6M per year. That's 2% per year. Assuming they can get a 6% average return on their assets, they can fully fund Wikipedia forever, and then some.

So, what do they do with all of the donations instead, if the money isn't needed to run Wikipedia? It funds the foundation's grant programs. Of course, you might actually like their grant programs. I think some of their grants are great, myself, and if they were honest about what they're using it for I might be inclined to give. But they're not, and the fact that they continue lying to Wikipedia's user base really pisses me off, so I don't give and I strongly discourage everyone I can from giving, at every opportunity.

[1] https://wikimediafoundation.org/who-we-are/financial-reports/

Re: Do they Need More Money? (Score:2)

by Midnight_Falcon ( 2432802 )

I don't think you read the next page of the audit report. They spend $100mm a year in salaries, so the scenario you suggest means they'd fire everyone.

Re: (Score:2)

by jdawgnoonan ( 718294 )

That does not matter, AI companies are ripping off the content to make money. AI is basically parasitic technology in every way.

Re: Do they Need More Money? (Score:3)

by Midnight_Falcon ( 2432802 )

I took a look, and your statement is wholly incorrect. There's not billions, just 67MM in long term investments, and they spend most of their revenue on operating costs of over 120mm/year. Where did you come up with this information? I got mine from [1]https://wikimediafoundation.or... [wikimediafoundation.org], see the KPMG audit report.

[1] https://wikimediafoundation.org/who-we-are/financial-reports/#a1-2023-2024

Wikipedia, The Most Important World Site (Score:4, Interesting)

by BrendaEM ( 871664 )

If you were trying to rebuild your society, after the way things are heading happens, you would want Wikipedia first.

Re: (Score:2)

by taustin ( 171655 )

Yeah, I'm sure the internet will be working just fine after what catastrophe causes you to need to rebuild society.

There are, however, [1]actual options [amazon.com] for such information.

[1] https://www.amazon.com/Book-Ultimate-Guide-Rebuilding-Civilization/dp/B0CJCKGRW1/ref=sr_1_1?crid=2WOM2NN83VU7I&dib=eyJ2IjoiMSJ9.HL3pPeDIiDqkXgWZkwPrGJ-JACEHLblSrC2sO3PHviyXWvn9kSlqY1tkchVtzXgMQQnAcTXiC8GA8zvcAfWndRmNkjH_g4mLpdx_hkjLwEcT-Jhj8_qYrN36DJjfa8CjeXOmH-SC4OBMKZHf4S0L6EidNmzsaN0cYhyeQzDZL39oYz43JKZVOwjh80qgo63SL6RarhnXB9ssp9a_PBgLO0VmSBHkdXIwS_ToRsD9NlE.7U7Hpx7J6CcZqcTV8Cap7dsQ7LD1gqMEBMGIG-j-JXA&dib_tag=se&keywords=the+book&qid=1762814343&sprefix=the+book%2Caps%2C176&sr=8-1

LLMs suck for information sometimes (Score:2)

by euxneks ( 516538 )

LLMs can be confidently incorrect - I would hope that teachers are teaching this in classrooms as much as they poo-poo'd wikipedia a decade or so ago.

You can literally just download the whole site (Score:5, Informative)

by SQL Error ( 16383 )

[1]https://dumps.wikimedia.org/ [wikimedia.org]

Available as a database, or a collection of individual pages. Mirrored and archived. There are torrents as well.

[1] https://dumps.wikimedia.org/

Re: (Score:2)

by eriks ( 31863 )

My thoughts exactly! I have a few (very old) copies of Wikipedia hanging around somewhere. I should go torrent a fresh copy. Way back when, I used to keep a text-only copy on my phone (Kiwix, which appears to still be a thing) for when I didn't have data. I bet I still have that SD card somewhere. I think it was about 10GB uncompressed back then.

I guess it goes to show how stupid and greedy these AI companies are. I'm sure that a lot of the primary training data for most models *is* Wikipedia. So lett

tell me about it (Score:5, Insightful)

by ErikKnepfler ( 4242189 )

I run a very small boutique hosting service and traffic has more than doubled since AI, all attributable to them. OpenAI in particular just seems to come along and hit like 30-60K links per day, no robots.txt rate limiting, just a "gimme all your data" scraping posture. Amazon is by far the worst, and it's also seems intentionally designed to conceal whether it's Amazon's AI teams or Amazon's cloud infrastructure clients doing the scraping. I've caught many of them using BS user-agent strings having generic "firefox" etc of course, many do so with apparent impunity.

Re: (Score:2)

by h33t l4x0r ( 4107715 )

It's impolite to ignore robots.txt, but it's not illegal. It's up to you to block the bots if they're bothering you.

Re: (Score:2)

by taustin ( 171655 )

If only one could get a reliable list of all IP addresses they use, it would be trivial.

Re: (Score:2)

by taustin ( 171655 )

Become? Were they ever not?

I remember the days.... (Score:2)

by sdinfoserv ( 1793266 )

Single mothers with children, and grandmothers were getting hit with multi thousand dollar demands from the music police over intellectual property violations from their 10 years old kids and grand kids downloading a song or two. Now, these oligarchs piss all over any intellectual property in the name of "training ai", and it's too big to fail investments.

Sue Them (Score:2)

by jdawgnoonan ( 718294 )

Do not beat around the bush, sue them. They are ripping off the content to make money.

Smoking is the leading cause of statistics.