Scanning phones to detect child abuse evidence is harmful, 'magical' thinking
- Reference: 1665646211
- News link: https://www.theregister.co.uk/2022/10/13/clientside_scanning_csam_anderson/
- Source link:
If adopted, these rules would – according to a top British computer security expert – authorize the reading and analysis of people's previously private communication for the sake of potentially preventing the spread of child sex abuse material and terrorism communications.
Ross Anderson, professor of security engineering in the Department of Computer Science and Technology at the UK's University of Cambridge, argues that these proposed regulations – which, frankly, rely on technical solutions such as device-side message scanning and crime-hunting machine-learning algorithms in place of police, social workers, and teachers – lead to magical thinking and unsound policies.
[1]
In [2]a paper titled Chat Control or Child Protection? , to be distributed via ArXiv, Anderson offers a rebuttal to [3]arguments advanced in July by UK government cyber and intelligence experts Ian Levy, technical director of the UK National Cyber Security Centre, and Crispin Robinson, technical director of cryptanalysis at Government Communications Headquarters (GCHQ), the UK's equivalent to the NSA.
[4]
[5]
That pro-snoop paper, penned by Levy and Robinson and titled [6]Thoughts on Child Safety on Commodity Platforms , was referenced on Monday by EU Commissioner for Home Affairs, Ylva Johansson, before the European Parliament’s Civil Liberties (LIBE) Committee [7]in support of the EU Child Sexual Abuse Regulation ( [8]2022/0155 ), according to Anderson.
The occasion for the debate is the approaching August 3, 2024 expiration of an EU law that authorizes online service providers to voluntarily detect and report the presence of child sexual abuse material in users' communications and files. Without replacement rules, supporters of the proposed child safety regime argue that harmful content will be ignored.
[9]
But online rights groups contend the contemplated legislation would cause its own harm.
"The proposed EU Child Sexual Abuse Regulation is a draft law which is supposed to help tackle the spread of child sexual abuse material," [10]said the European Digital Rights Initiative (EDRi), in response to Johansson's proposal.
"Instead, it will force the providers of all our digital chats, messages and emails to know what we are typing and sharing at all times. It will remove the possibility of anonymity from many legitimate online spaces. And it may also require dangerous software to be downloaded onto every digital device."
[11]
Meanwhile the UK is considering its own [12]Online Safety Bill , which also imagines [13]bypassing encryption via device-side scanning. Similar proposals, such as [14]the EARN IT bill , keep surfacing in the US.
The paper by Levy and Robinson – itself a response to [15]a paper opposing device-side scanning that Anderson co-authored with 13 other security experts in 2021 – outlines the various types of harms children may encounter online: consensual peer-to-peer indecent image sharing; viral image sharing; offender to offender indecent image/video sharing; offender to victim grooming; offender to offender communication; offender to offender group communication; and streaming of on-demand contact abuse.
Anderson argues that this taxonomy of harms reflects the interests of criminal investigators rather than welfare of children. "From the viewpoint of child protection and children’s rights, we need to look at actual harms, and then at the practical priorities for policing and social work interventions that can minimize them," he says.
Anderson calls into question the data used to fuel media outrage and political concern about harms to children. Citing the 102,842 reports from National Center for Missing and Exploited Children (NCMEC), the US-based non-profit coordinating child abuse reports from tech firms, to the UK's National Crime Agency (NCA), he estimates that this led to 750 prosecutions for indecent images, "well under 3 percent of the 2019 total of 27,233 prosecutions for indecent image offences, of which 26,124 involved images of children." And the number of such prosecutions peaked in 2016 and has since fallen, he says.
"In short, the data do not support claims of large-scale growing harm that is initiated online and that is preventable by image scanning," says Anderson.
The danger of relying on dodgy evidence
However, real harm is done by false positives, he observes, pointing to [16]Operation Ore , an internet child abuse crackdown that began two decades ago and led to false accusations.
Levy and Robinson propose "have language models running entirely locally on the client to detect language associated with grooming." They liken this approach to the on-device CSAM-scanning proposed by Apple (and [17]subsequently shelved , at least for the time being) in the US. While they acknowledge the problems raised at the time – false positives, mission creep, vulnerability to tampering – they assert, "Through our research, we’ve found no reason why client side scanning techniques cannot be implemented safely in many of the situations one will encounter."
Anderson says law enforcement agencies long ago gave up scanning emails for keywords like "bomb" because it doesn't work and because traffic analysis, for which content access is not required, is more effective. And he doesn't expect natural language processing (NLP) models will perform any better.
"The use of modern NLP models to detect illegal speech – whether sexual grooming, terrorist recruitment or hate speech – is highly error-prone," he says. "Our research group has long experience of [18]looking for violent online political extremism , as well as fraud and spam. Going by text content alone, it can be difficult to get error rates significantly below 5–10 percent, depending on the nature of the material being searched for."
[19]British intelligence recycles old argument for thwarting strong encryption: Think of the children!
[20]Tories spar over UK's delayed Online Safety Bill
[21]Tech world may face huge fines if it doesn't scrub CSAM from encrypted chats
[22]Europe proposes tackling child abuse by killing privacy, strong encryption
With 5 percent false positives, Anderson suggests each of Europe's 1.6 million police officers would have 625 alarms about potential harms to deal with daily – not exactly a practical scenario. That's not to say there aren't options, just that the technical fixes breaking encryption aren't fit for purpose.
In an email to The Register , Anderson indicated the private sector has shown an interest in helping governments get into content scanning.
"There's a company called Thorn that is lobbying for the scanning contract and would love to get a government mandate for its software to be installed into your chat clients," he said.
"The chat service operators would hate that, which may be a reason why Apple produced its own client-side scanning software that caused a storm last year before part of it was withdrawn. There are also some UK startups that GCHQ and the Home Office funded to produce prototypes. Perhaps this would just be used as a means of bullying Big Tech into doing the job themselves.
"However the big soft spot is how Big Tech handles user reporting, which ranges from bad (Facebook) to almost not at all (Twitter). There is a real case for governments to mandate better performance here, as the paper sets out and as I also discuss in [23]my paper on the UK Online Safety Bill , which came out last week."
Anderson in his Chat Control paper suggests that child safety and privacy campaigners could make common cause to advance rules that compel online service providers to take down illegal content when reported.
"At present, tech firms pay attention to takedown requests from the police and from copyright lawyers, as ignoring them can be expensive – but ignore ordinary users including women and children," he says. "That needs to be fixed, whether by criminal sanctions or by significant financial penalties."
Anderson's recommendations for dealing with child abuse focuses on traditional, complicated approaches: quality, community-based policing rather than push-button fixes; social engagement; empowering young people; and respect for human rights.
"The idea that complex social problems are amenable to cheap technical solutions is the siren song of the software salesman and has lured many a gullible government department on to the rocks," Anderson says. "Where ministers buy the idea of a magical software 'solution,' as the industry likes to call its products, the outcomes are often disappointing and sometimes disastrous."
Beyond that, Anderson says that pervasive surveillance, without cause, violates human rights law. "The rule of law must take precedence over 'national security’," he concludes. "We must maintain a moral advantage over competing authoritarian states, not just a military and technological advantage. End-to-end encryption must therefore remain available for moral reasons."
And, he says, encryption must remain for valid cybersecurity reasons, as Levy and Robinson [24]acknowledged previously , and he and his fellow technologists argued in their previous paper. ®
Get our [25]Tech Resources
[1] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_security/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2Y0fhuKw-CtkjJrrap7bZgwAAANM&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[2] https://www.cl.cam.ac.uk/~rja14/Papers/chatcontrol.pdf
[3] https://www.theregister.com/2022/07/22/british_encryption_scanning/
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_security/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Y0fhuKw-CtkjJrrap7bZgwAAANM&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_security/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33Y0fhuKw-CtkjJrrap7bZgwAAANM&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[6] https://arxiv.org/abs/2207.09506
[7] https://twitter.com/YlvaJohansson/status/1580160162827935744
[8] https://oeil.secure.europarl.europa.eu/oeil/popups/ficheprocedure.do?reference=2022/0155(COD)&l=en
[9] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_security/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Y0fhuKw-CtkjJrrap7bZgwAAANM&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[10] https://edri.org/our-work/johanssons-address-to-meps-shows-why-the-csa-law-will-fail-the-children-meant-to-benefit-from-it/
[11] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_security/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33Y0fhuKw-CtkjJrrap7bZgwAAANM&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[12] https://bills.parliament.uk/bills/3137
[13] https://www.eff.org/deeplinks/2022/08/uks-online-safety-bill-attacks-free-speech-and-encryption
[14] https://www.eff.org/deeplinks/2022/02/if-earn-it-passes-what-happens-your-iphone-wont-stay-your-iphone
[15] https://www.theregister.com/2021/10/15/clientside_side_scanning/
[16] https://www.theguardian.com/technology/2007/apr/19/hitechcrime.money
[17] https://www.theregister.com/2021/12/16/apple_deletes_csam_scanning_plan/
[18] https://arxiv.org/abs/2111.04479
[19] https://www.theregister.com/2022/07/22/british_encryption_scanning/
[20] https://www.theregister.com/2022/07/15/online_safety_bill_delayed/
[21] https://www.theregister.com/2022/07/07/uk_online_safety_bill_chat_scanning/
[22] https://www.theregister.com/2022/05/12/eu_encryption_csam/
[23] https://www.lightbluetouchpaper.org/2022/10/04/the-online-safety-bill-reboot-it-or-shoot-it/
[24] https://www.lawfareblog.com/principles-more-informed-exceptional-access-debate
[25] https://whitepapers.theregister.com/
Plus it would mean 90% of your CPU power would be churning away running computationally hard data scanning.
If people in the 1960s knew what computers and the Internet did to our freedom
The frog has been boiling very slowly over the last couple of decades, if people in the 1960s knew about the surveillance and thought-policing going on in the 2020s, people would be shocked, in outrage. There would be mass protests and even riots on the streets.
Trying to police what media people are reading or looking at is an absolute Orwellian nightmare. It is not very far from thought control. And there are absolutely no excuses for prosecuting people who read or look at information. It is a manifestation of moral zealots who are on the OCD spectrum, trying to impose their own thought suppression and control, on the general public. Using the age old-excuse of"protecting women/children".
I think we have a fundamental natural, human right, to be able to receive any information we want to, period. And that the government has no right to take that away from us. It has already expanded from forbidden images into forbidden text, such as information that could be "useful to terrorists", another moral panic. Only distribution of such material should be ever be prosecuted.
They could use exactly the same excuses, i.e. that policing people's thoughts would protect children from harm. And that thinking the wrong things could possibly lead to abuse. Therefore we must all be under surveillance for undesirable thoughts.
Instead, go after those people who are actually creating and distributing these images.
The problem comes down to the two-way, fully traceable nature of the Internet itself. None of this would be possible with radio or satellite TV, because only the Internet allows the state to determine what you are reading or watching... They can't round up people and imprison them for watching/listening to "forbidden" radio or TV stations, because even if it was illegal, it would be practically unenforceable due to the laws of physics.
Perhaps it's time for a new medium, in a few decades from now, something using quantum communications, that does not have this "easy surveillance" property...
There is already research being done into "quantum anonymous networks"....
[1]https://researchoutreach.org/articles/securing-communication-quantum-anonymous-networks
And real abuse images could still be detected from photos of the victims, using facial recognition or some similar techniques.
[1] https://researchoutreach.org/articles/securing-communication-quantum-anonymous-networks
Re: If people in the 1960s knew what computers and the Internet did to our freedom
They did - or at least the potential was obvious to many. There are numerous instances of this understanding (TV shows e.g. Star Trek: "the ultimate computer", "the changeling", "the return of the Archons"; the outer limits: "O.B.I.T."; short stories e.g. E.M. Forster's "the machine stops", C. Smith's "Alpha Ralpha boulevard").
It's been obvious to thinking folks for ages that automata can be great servants but terrible masters, and that in the hands of ideologues any technology can be applied to inappropriate ends. The saddest state is when the ideologues gain positions of power that allow them to apply the technologies without regard for the wider implications.
Re: If people in the 1960s knew what computers and the Internet did to our freedom
Instead, go after those people who are actually creating and distributing these images.
Exactly. In fact, the are volunteer organizations doing the work of infiltrating internet CP groups. They already find far more images and leads than can be processed to conviction. Processed to conviction mean to trace the IP addresses, the follow through with warrants and arrests and court cases. THAT is the bottleneck. The evidence for that is well documented in "The Internet Is Overrun With Images of Child Sexual Abuse. What Went Wrong?" [NYT, 2019].
What is interesting is that that NYT article, despite offering explicit evidence that the bottleneck is hiring the experts and detectives to follow up the overflowing plenty of leads, arrives at the dumbfounding conclusion that universal device/communication surveillance and encryption backdoors are warranted, not budgeting to solve that bottleneck.
Re: If people in the 1960s knew what computers and the Internet did to our freedom
Computers are literal magic to a lot of these people - even to people who have spent their entire lives embedded in the technology sphere, who think that computers can solve all the problems of the world because they've made them solve some of the problems they personally encountered.
Hiring people is hard. Waving a magical wrongthink detector box at the problem is far easier.
we’ve found no reason why client side scanning techniques cannot be implemented
Apart from the slight issue of persuading people to allow the software to run?
"If you have nothing to hide then you have nothing to fear!"
"If I have nothing to hide then why are you spying on me?"
Re: we’ve found no reason why client side scanning techniques cannot be implemented
> "If you have nothing to hide then you have nothing to fear!"
Which makes about as much sense as saying that you shouldn't be mad if someone points a gun at you, as long as he doesn't shoot.
A core principle of democracy ought to be that positions of power need to be well justified, not just exertion of power. By grabbing someone's data, you enter a position of power over him. This is already problematic, regardless of whether you do something with that data or not.
Basically, being denied hiding is in itself something we should fear.
To extend the metaphor... in order for me to get shot, a gun needs to be carried in my presence, drawn, have its safety removed, pointed, and triggered. Every additional step that gets taken in that sequence is an additional, escalating threat. It is not true that no such step is a threat until the last one.
Similarly, every single step that facilitates an abuse of power is in itself a threat. Sometimes they are needed, but every such step needs individually strong justification, before it can be allowed to happen. And simply "because I might later need to do the subsequent steps" is a terrible, terrible justification.
Sponsorship...
What's the betting that Prof. Ross Anderson's work is being bankrolled by Google, Facebook and their ilk?
Re: Sponsorship...
Low to zero. He's not exactly been favourable to them in the past, and his research focus over the years make it clear about his views: https://www.cl.cam.ac.uk/~rja14/
Besides Google, Facebook and their ilk probably wouldn't be that bothered by doing mandatory client side scanning as that gives them the slippery slope to include additional data into targeted advertising
Every bit of research in the field and every statistic on prosecutions shows that there is far less of this abuse happening now than ever before, and none of this fishing ever winds up showing a meaningful need for it. There's a reason why child abuse is the go-to for both the groups that want to breach security measures just for the sake of doing so and the groups that have delusional conspiracy theories about satanic cults draining children of their fluids to stay young.
Much more difficult for abusers to get away with it nowadays
The fact that children are better connected than ever before in history probably makes it much harder for abusers to get away with it... Children are always in touch with their friends and have easier ways to report it and seek assistance about it. Support groups on the internet make it easier for victims to realise they are not alone, etc. And not just sexual abuse too, but physical and emotional abuse too.
Re: Much more difficult for abusers to get away with it nowadays
Bingo: if you really want to impact child abuse, make it easier for kids to figure out something is wrong and reach out. It's hard to do that while there are groups that don't want any topic that has anything remotely to do with sex getting discussed within a mile of any kid anywhere, though.
Re: Much more difficult for abusers to get away with it nowadays
On reaching out,.... once kids know their device is bugged, how likely are they to have open honest conversations about personal matters using it? Their anonymity will be gone, they'll be wary of sharing personal concerns for fear it's all recorded, and it will rob them of a chance to get help. Snooping on kids will be massively damaging, so instead of thinking of the kids, can we actually think of the kids?
It would appear that there is way too many people thinking about the kids.
Re: Much more difficult for abusers to get away with it nowadays
Well so much of it comes down to power, a human instinct. Many of these justifications for "protecting children" are just an excuse for authority figures, including parents, to exert their dominance over children. Just as animals have a dominance hierarchy, we people do, which varies on an individual basis. And there's probably a primal drive to do so, which we should be aware of.
In many cases taking away freedom from older children and teenagers just serves to delay their development of responsibility. The opportunity to make mistakes and learn from them. So peoples' development is being stunted. And this has been getting worse since the 1980s moral panics over "stranger danger". No wonder why we have a generation of adults who put "safety" over freedom, who expect to be protected from every risk in society.
Empowering children to feel in charge of their own life, to a certain degree, and not feel as if they are slaves to the system, is probably one step towards improving their rights....
We have already made great strides with Feminism (the non-radical type) in the early 20th Century in asserting womens' rights.. And we abolished the very serious problem with racism in the 1950s and 1960s. How about we look at the rights of children now and how terribly they are treated in general by society, almost imprisoned like slaves? With parents wielding near-total power and control over them. Our society's blindness to such a fundamental human rights violation is staggering.
And I am not one of those social justice types who wants "microaggressions" or other similar things abolished... This is for real, true freedom for everyone.
I recall the 'Satanic ritual abuse' panic, it started in the US, then got a foothold over here in the UK, and it bothered me that people gave it any credibility. Especially when children were in more danger from Priests. That said, this is all theatre, how much abuse happens without being recorded or shared online, and isn't prevention more important than detection? Also, once bad guys know they are being eavesdropped, they'll switch tactics. We were taking our shoes off at airpoirts _afrer_ the failed attempt, not before, and checking shoes has not defeated terrorism.
Complex systems need testing
Perhaps a group of 650 individuals, with the public being free to audit the collected data in real-time?
Re: Complex systems need testing
I don't think I could sit through that much tractor p0rn.
"Beyond that, Anderson says that pervasive surveillance, without cause, violates human rights law"
There's a simple solution to that (as indicated by the proposed effective neutering of the UK data protection law) - and that's just to abandon human rights law. I guess I shouldn't have suggested it, but there are murmurs in the UK that indicate it's already been thought of.
Illegal speech???
...models to detect illegal speech...
This is a real WTF? Speech is not and can never be "illegal". You cannot detect "illegal" speech. If (some) speech is "illegal", then we have given up our freedom of expression.
The illegality is the action that follows the speech or is accompanied by it. The speech itself is never ever illegal.
Re: Illegal speech???
Well, in the USA, perhaps, by statute.
But your post illustrates the basic fallacy that there *are no rights* other than those which a legislative body permits. 'Human rights' are a myth; in spite of their adoption and support by the United Nations, look how many places simply do not permit those rights to be exercised. As long as a government - legitimate or not - can usurp those rights for even one of its citizens, those rights may as well not exist.
Human rights are a fact
It is not because there are countries that do not recognize them that Human Rights do not exist.
Just like it is not because the school bully goes bullying that he is justified in doing so.
It would, of course, be nice if every country recognized the charter, but we live in an imperfect world and we have to make do with it.
Re: Illegal speech???
Crying "fire" in a crowded theatre simply to try and cause a dangerous stampede would be illegal speech, and [1]has been even in the US as long ago as 1884.
So speech can be illegal.
[1] https://timesmachine.nytimes.com/timesmachine/1884/09/25/109777936.pdf
Re: Illegal speech???
The action is the intended disruption/panic; that is the illegal part. The speech of "fire" is protected.
I know, it is a very fine line, but it is an important distinction.
"there *are no rights* other than those which a legislative body permits"
Ultimately, that is true. But there's a major distinction in principle between Common Law and Civil Law in that the former allows everything that is not expressly prohibited but the latter does the opposite. In an effective democracy (particularly a Common Law one) it takes legislation to extend the scope of the prohibited, which acts a check on its extension. Unfortunately, at least in the UK, there has been a progressive move towards secondary legislation, which allows regulations to be introduced without full scrutiny by Parliament, effectively undermining democratic protection.
Nevertheless, human rights do still exist insofar as violation of them is internationally considered an offence, witness the European rejection of the UK "snoopers' charter" legislation. Whether such rejection has any practical effect is of course another matter.
Scan This!
Call me paranoid, but!
This proposed legislation and the crazy technical solutions being proposed has nothing to do with preventing child abuse. ('Protecting the children' is a typical dog whistle that is often applied to proposals to shame opponents into keeping shtum. I mean, only a pervert would argue against any measure to protect kids, right?)
It hasn't got much to do with the 'Prevention of terrorism' either. Using mass surveillance suggests we are all terrorist suspects. Trawling the millions of private messages of tens of millions of UK citizens every day, for say, a thousand terrorist plots is insane. You'd need to know who they were first, and getting an anonymous mobile phone isn't that hard. Or perhaps stopping using a mobile phone for their plotting.....
But illegal speech? Now that's more like it! Well not illegal speech, more like speech that's of interest to someone. Political opponents, trade unionists, whistle blowers, or anyone whose speech might be said to be illegal by some future politician.
Again, I might be paranoid,but that doesn't mean the buggers aren't out to get us!
Where's a black helicopter when I need it?
Mine's the one with the tracker in the pocket!
Never mind the fact that, even if their detection technology had a 100% true positive rate and a 0% false, all you'll catch is low-hanging fruit: The idiots who browse CP on an iPhone in Starbucks and the people who stumbled across things by accident (or were sent them by malicious parties). This won't touch anyone who has made even the smallest effort to harden their devices.
The great minds behind this seem to believe that, if they just write enough rules and implement enough technology to enforce those rules, everyone will simply behave. They don't understand the technocratic nightmare they're creating. Magical thinking indeed.