Pew: Quarter of web pages vanished in past decade
- Reference: 1716235253
- News link: https://www.theregister.co.uk/2024/05/20/webpages_vanish_decade/
- Source link:
A [1]report published by the Pew Research Center finds that digital decay is erasing online news, Twitter/X posts, and web links.
The Pew team - Athena Chapekis, Samuel Bestvater, Emma Remy, and Gonzalo Rivero - found that 25 percent of web pages that existed at some point between 2013 and October 2023 no longer function. As for older web pages that existed before 2013, 38 percent can no longer be accessed. As a point of comparison, about 8 percent of web pages from 2023 have disappeared.
[2]
The study, based on an analysis of web crawl data from [3]Common Crawl , some 50,000 Wikipedia articles, and Twitter/X posts from last year, confirms longstanding concerns about the ephemeral nature of digital content.
[4]
[5]
"The internet is an unimaginably vast repository of modern life, with hundreds of billions of indexed webpages," Pew says. "But even as users across the world rely on the web to access books, images, news articles and other resources, this content sometimes disappears from view."
This has particular implications for news and government websites. A sample of half a million news pages from Common Crawl in March/April 2023 found that 23 percent had at least one broken link. For government websites – the same size sample during the same period – 21 percent had at least one broken link.
[6]
In a sample of 50,000 English-language Wikipedia pages, 82 percent included at least one reference link – a cited source – and 11 percent of these are no longer accessible. This makes fact checking much more difficult.
For the social media platform X, which was still referred to as Twitter during the March 8 to April 27, 2023 sample period, some 18 percent of 5 million tweets were no longer accessible by June 15 of that year. Some had been suspended, some had been removed, and some had been made private.
Among the portion of tweets that got removed, half vanished within the first six days of posting, and 90 percent were no longer available within 46 days.
[7]
Twitter posts, however, also get resurrected – about 6 percent resurface at some point, a phenomenon the researchers attribute to accounts that went private and then returned to public status, or suspended accounts that got reinstated.
[8]The internet – not as great as we all thought it was going to be, eh?
[9]How the Internet Archive faces potential destruction at the hands of Big Four publishers
[10]Internet Archive's 2046 Wayforward Machine says Google will cease to exist
Mark Graham, director of the [11]Wayback Machine at the non-profit Internet Archive, told The Register : "This is why we do what we do."
The Wayback Machine crawls the web and archives copies of websites for posterity.
"We have identified and fixed more than 21 million broken URLs in Wikipedia articles, replacing them with archived versions of those web resources from The Wayback Machine," he explained.
Graham said the consequence of digital decay is the loss of our collective history. "When a society or people lose the ability to cite and compare and contrast, they lose the ability to do any kind of critical analysis or contextualize current events."
"We operate in the forever-now," he said. "We lose access to our memory. The ability to contextualize the events of our times depends upon having a record of our times." ®
Get our [12]Tech Resources
[1] https://www.pewresearch.org/data-labs/2024/05/17/when-online-content-disappears/
[2] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/personaltech&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2ZkvIAvRDlZcGfHvZCos31gAAAAQ&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[3] https://commoncrawl.org/
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/personaltech&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZkvIAvRDlZcGfHvZCos31gAAAAQ&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/personaltech&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZkvIAvRDlZcGfHvZCos31gAAAAQ&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[6] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/personaltech&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZkvIAvRDlZcGfHvZCos31gAAAAQ&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[7] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_onprem/personaltech&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZkvIAvRDlZcGfHvZCos31gAAAAQ&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[8] https://www.theregister.com/2018/09/14/pew_internet_satisfaction_study/
[9] https://www.theregister.com/2023/03/20/internet_archive_lawsuit_latest/
[10] https://www.theregister.com/2021/10/01/internet_archive_wayforward_machine/
[11] https://web.archive.org/
[12] https://whitepapers.theregister.com/
Re: this content sometimes disappears from view
Yes, but...
All too many commercial web sites periodically and arbitrarily change the URIs to access precisely the same content. I suppose they assume that there's no problem since you can still get at it by entering at the home page, and they don't have the imagination to consider any other way of getting there, but wilfully breaking links is totally contrary to the foundational concept of the WorldWide Web and rather anti-social. If the content was worth putting up, you should maintain the URIs for ever. It's not like it's even hard with modern web server platforms.
On the other hand, a vast amount of web content is ephemeral spam, of no positive value whatsoever. These should indeed disappear completely as soon as the bastards are rumbled.
-A.
Deleting old product data
Disappearing content can often be done maliciously (IMHO). Anyone like me regularly looking for help with problems on the Microsoft website will have experienced them deleting whole sub-sites on a regular basis. It’s never done with proper redirects or signposts to alternative sources for the information. They just decide one day to take out 10,000 pages and they are gone. The reason is almost always that offering support to keep customers ticking over on an old version is not good business. So much better to force them to pay for the latest by removing the documentation and user forums. Removing the documentation is pretty bad but removing the user forums, where desperate customers post their issues and other customers solve their problem for free with no input from Microsoft is just rude. I’d like to see a law requiring companies to maintain copies of manuals for products they sell for as long as their website exists. The additional costs for disk space would be trivial.
Not only that ..
Some web pages with negative reviews of the $product are retrospectively replaced with positive adverts for said $product. Down the electronic memory hole we go /s
this content sometimes disappears from view
I've noticed that a fair number of the "official" reviews of films given on IMdb now point to parked domains.
The problem is that while big companies often keep their content around, smaller outfits have hosting to pay, domains to register, and when something runs to it's end then everything lapses and all of that content vanishes. Moreso if it was, say, a hobby site and the site maintainer went on to do something else, got married, whatever, and no longer has the time or inclination to continue. I also think a number of smaller sites got abandoned as simply having a page on Facebook was so much simpler (and cheaper) than dealing with HTML and hosting and uploading and...
In the case of big companies, it could be mergers or cutbacks or simply abandoning something seen as generating loss. A large number (at the time) of sites vanished when Geocities shut down (Yahoo ! bought it, screwed up, then eventually Googled it into a shallow grave).
Ultimately, keeping stuff on the web costs somebody, somewhere, money. When the money goes away, so does all that data.