Judge demands social media sites prove they didn't help radicalize mass shooter
- Reference: 1710890110
- News link: https://www.theregister.co.uk/2024/03/19/social_media_sites_headed_to/
- Source link:
Depending on the outcome, the case could reshape liability rules for social media sites.
In a court [1]order [PDF] published on Tuesday, New York 8th district Supreme Court justice Paula Feroleto denied a dismissal request from Meta, Reddit, Twitch's company Amazon, YouTube owner Alphabet, plus Discord and 4Chan. They all need to go to court to argue their case.
[2]
"Many of the social media/internet defendants have attempted to establish that their platforms are mere message boards," Feroleto wrote.
[3]
[4]
"This may ultimately prove true," the judge noted. However, "the Court has determined the complaint sufficiently pleads viable causes of action to go forward at this stage of the litigation," she added.
Section 230 goes to court
At issue in the case is how the platforms characterize their services, versus plaintiffs' contention that they were liable for Payton Gendron's shooting at a Tops Friendly Markets store in Buffalo, New York, in a predominantly Black neighborhood of the city.
According to the complaint, Gendron – who has pleaded guilty, was sentenced to life in prison and is awaiting a [5]federal death penalty trial – was radicalized by the content he discovered on the defendants' platforms.
"By his own admission, Gendron, a vulnerable teenager, was not racist until he became addicted to social media apps and was lured, unsuspectingly, into a psychological vortex by defective social media applications designed, marketed, and pushed out by Social Media Defendants," the plaintiffs argue in their [6]complaint [PDF].
[7]
The platforms argue their sites are simply message boards, and that they are protected by the First Amendment right to freedom of speech and the Communications Decency Act (CDA) – specifically [8]Section 230 , which protects platforms from liability for the content posted by their users.
The plaintiffs are families of people killed by Gendron and agree that the CDA and First Amendment protects the platforms. However, as the judge notes, that's not the argument they're making.
[9]Can YouTube be held liable for pushing terror vids? Asking for a Supreme Court...
[10]So, where does Silicon Valley stand on S.230? Zuck wants 'industry best practices', Pichai demurs, Dorsey urges more Bluesky thinking
[11]Click here to see the New Zealand livestream mass-murder vid! This is the internet Facebook, YouTube, Twitter built!
[12]Make Facebook, Twitter, Google et al liable for daft garbage netizens post online – US Senator
"[Plaintiffs] instead contend the defendants' platforms are negligently, defectively and harmfully designed 'products' that drove Gendron to specific materials and that they are therefore liable," Feroleto wrote. If viewed as products and not platforms, Section 230 is irrelevant, the plaintiffs argue.
New York law has long established that product manufacturers are liable for harm incurred by any person injured as a result of their products – even if they weren't using the product themselves, the judge said.
"Contrary to the defense assertions, at this stage of the proceedings, the plaintiffs' allegations concerning product liability establish a basis for 'duty' to these plaintiffs," Feroleto found. As such, the case can proceed.
[13]
It's not clear when – or even if – the case will go to trial. But it's headed that way unless the platforms successfully appeal or settle the matter. The result could set a precedent that makes it harder to use Section 230 as a get-out-of-jail-free card for social media platforms – something that US officials have [14]long tried to do .
We asked all of the affected platforms for comment on this matter. Reddit and YouTube, the only two platforms to respond to our questions, both indicated they intend to appeal.
YouTube told us it has the deepest sympathies for victims and their families, and has invested in tech and policies to identify and remove hate content for years.
"While we disagree with today's decision and will be appealing, we will continue to work with law enforcement, other platforms, and civil society to share intelligence and best practices," a YouTube spokesperson told The Register .
Reddit declared that hate and violence have no place on its platform and pointed us to its [15]content policy that prohibits hate content based on identity or vulnerability as well as messages that glorify, encourage or incite violence.
"We are constantly evaluating ways to improve our detection and removal of this content, including through enhanced image-hashing systems, and we will continue to review the communities on our platform to ensure they are upholding our rules," Reddit told us. ®
Get our [16]Tech Resources
[1] https://regmedia.co.uk/2024/03/19/tops-shooting-social-media-case-order.pdf
[2] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_offbeat/legal&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2ZfptaGT@GgReI3ybYSY@9gAAAUg&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_offbeat/legal&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZfptaGT@GgReI3ybYSY@9gAAAUg&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_offbeat/legal&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZfptaGT@GgReI3ybYSY@9gAAAUg&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[5] https://apnews.com/article/buffalo-supermarket-shooting-death-penalty-b6ab8eda541c2c0a0b512a89b0471b91
[6] https://regmedia.co.uk/2024/03/19/tops-shooting-social-media-case-complaint.pdf
[7] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_offbeat/legal&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44ZfptaGT@GgReI3ybYSY@9gAAAUg&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[8] https://www.theregister.com/Tag/Section%20230/
[9] https://www.theregister.com/2023/02/22/us_supreme_court_Section_230/
[10] https://www.theregister.com/2021/03/24/congress_Section_230/
[11] https://www.theregister.com/2019/03/15/new_zealand_murder/
[12] https://www.theregister.com/2018/07/30/create_desert_call_it_peace/
[13] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_offbeat/legal&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33ZfptaGT@GgReI3ybYSY@9gAAAUg&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[14] https://www.theregister.com/2022/09/09/biden_tech_reform_Section230/
[15] https://www.redditinc.com/policies/content-policy
[16] https://whitepapers.theregister.com/
No clean hands
It will be interesting to watch how the claims develop in court, but they don't have to be one size fits all. Meta's properties are based on ultra-invasive profiling, targeting, and monetization. Neither Insta for Facebook have clean hands, quite the contrary they are eyeballs deep in a literal genocide. YouTube is in the same boat, steering people to dangerous but viral content and making piles of cash of the traffic. They don't passively serve content, they shovel it down peoples throats, and aggressively steer into fraudulent and dangerous content to sell ad impressions. Reddit promotes posts, and had turned a blind eye to toxic and criminal content for ages. Some communities are eventually shut down, but like Meta and Google, it's internal content moderation team is incapable of meeting the flood violating content, so only a fraction of the worst material is removed by the company. Reddit also turns a blind eye when banned communities partially decamp to another platform, but maintain a toe hold on Reddit that is tacitly pointing users to offsite troves of violating materials.
4chan and 8* are structured as traditional message boards, but glorify both the content and the community behind it, so they are in effect promoting ALL the posts, not just the ones chosen by an algorithm. That makes them MORE complicit, not less, and just being a message board doesn't protect you, as the old stormfront, atomwaffen, etc etc found out. That said it is not automatic, and in the narrower case of this shooting it will be on the plaintiff to provide additional evidence.
Discord is tricky as it wasn't founded as a haven for poorly socialized (literal or mental age) adolescents. That said it was quite slow to act to clean up toxic or criminal servers and did promote both servers and content by various means over the years. That is one I am watching for in the upcoming trial, but seems like the most likely to be dropped at this point.
Your call out on TikTok points to quite a glaring omission from the discussion at least. Their "algorithm" included a feature to massively boot specific content essentially at the push of a button, as well as targeting data second only to Facebook and Google. Viral content on TikTok regularly promotes self harm, propaganda, bogus medical scams, hate speech, and conspiracy theories promoting extremism and outright terrorism. But based on this suit, the companies called out are ones that the plaintiffs can link to this case, so while you are right TikTok is a huge problem, it may not play a part in this particular suit.
Re: IANAL
I agree, but I don't think that's what these lawyers are going for and that really confuses me.
The issue of being a "simple message board" vs something 'more' is The Algorithm™ - the way that most of these sites actively push content to which you are not subscribed at you, and this instant echo-chamber creation has been the subject of numerous studies showing that it causes all kinds of harm. Radicalisation is an obvious outcome; you show interest in something, so you're shown more of it, some of that will doubtless be more extreme, and so we proceed to circle down the drain.
This is further worsened by no serious attempt to actually defend against fake news, including now both audio, still photo and video deepfake content. At the scale these platforms operate it's very hard to do, but they only got that scale in the first place by ignoring all of these issues, putting just about no resources into them and pocketing all the spare cash. Cry me a river should the legally mandated cost of doing business go up, in order to clean up the cesspit that they themselves created. They can afford it, and then some.
Without a recommendations algorithm, users would have to actively search for content of interest. Specific message boards / pages / groups / Subreddits / whatever-you-want-to-call-them that specialised in "radical" content would be easy enough for automated searches to find and flag for human moderator review, if they're easy enough for regular users to find. With an appropriately scaled and resourced review team, both "objectionable" (FSVO "objectionable") or outright per-country/state illegal content would be more rapidly found. Disinformation is harder to tackle, but the framework for tackling it can only be established once the aforementioned review infrastructure is present.
None of this seems to be the focus of the lawsuit; they seem to be trying to argue over a legal distinction between these things being "a product" or not. That seems pretty difficult. Perhaps the idea of being "a product" is legally "proved" by the presence of a recommendations engine which implies immediate, albeit computer-generated corporate content bias pushed at individual users? Somehow, though, I doubt it...
Re: IANAL
"where is TikTok?"
My best guess is that they're focusing on services this specific attacker used and he didn't use TikTok? It's a long list as it is, but maybe he did list all of those as places he found material that made him want to commit mass murder. That restriction is the only reason why TikTok couldn't fit into the list. Whether this suit will prove viable is a separate question.
It’s the algorithms on trial
Burn them to the ground. Let people find their own content, not have it pushed to them left right and centre.
Creative interpretations to try and get around section 230, not the first time it's been tried. I suspect if platforms lose they'll go to a federal court saying state law is preempted in this case by section 230.
Or you could fix ...
....your dumb ass gun culture!.
Nahhh. Freedumbs and all that
Just carry on with the weekly mass murders
IANAL
Meta, Reddit, Twitch's company Amazon, YouTube owner Alphabet, plus Discord and 4Chan (where is TikTok?)
But it seems to me the only ones in this group with a defence are Discord and 4Chan i.e. the ones acting as "plain-old messageboards" without promoting content to er, "like-minded users"?
Reg needs a popcorn icon. Or maybe dry-roasted peanuts.