Apple responds to critics of CSAM scan plan with FAQs - says it'd block governments subverting its system
(2021/08/09)
- Reference: 1628545029
- News link: https://www.theregister.co.uk/2021/08/09/apple_csam_faq/
- Source link:
Apple's announcement last week that it will soon be scanning photos on iPhones and iPads that sync to iCloud for child sexual abuse material (CSAM) prompted pushback from thousands of security and privacy professionals and a response from the company that attempts to mollify its critics.
The iDevice biz revealed two [1]child safety initiatives that are initially being rolled out in the US and later in other countries depending on regulatory approval.
One is a system to alert children and their parents when the Messages app sends or receives pictures deemed explicit (but not necessarily illegal) by an on-device machine learning algorithm.
[2]
The other is system to scan photos on iOS and iPadOS devices that sync to iCloud Photos to see if the hashes (identifiers) of on-device images match any hashes of known CSAM material (illegal), which Apple will use to flag iCloud accounts for cancellation and reporting to the National Center for Missing and Exploited Children (NCMEC).
[3]
[4]
Apple published a technical summary
Went about as well as you'd expect
The announcement
"Apple's current path threatens to undermine decades of work by technologists, academics and policy advocates towards strong privacy-preserving measures being the norm across a majority of consumer electronic devices and use cases," said the letter, which currently lists more than 6,000 signatures. "We ask that Apple reconsider its technology rollout, lest it undo that important work."
Technical experts, privacy advocates, academics, and others have spent the past weekend debating the issue via social and online media. Pretty much everyone agrees that CSAM is a problem.
[9]
The question is whether a company that has said, "Privacy is a human right" – even if it doesn't offer that in China – should be attempting to tackle child safety by running its own scanning code – with consent obtained via the decision to use iCloud Photos rather than explicit permission – on its customers' devices.
[10]Apple is about to start scanning iPhone users' devices for banned content, warns professor
[11]You may be distracted by the pandemic but FYI: US Senate panel OK's backdoors-by-the-backdoor EARN IT Act
[12]Don't be fooled, experts warn, America's anti-child-abuse EARN IT Act could burn encryption to the ground
[13]Upcoming Android privacy changes include ability to blank advertising ID, and 'safety section' in Play store
Alex Stamos, director of the Stanford Internet Observatory and former CSO of Facebook, attempted to stake out the middle ground between those horrified by Apple's approach and those who would give the fight against CSAM priority over any other considerations.
"In my opinion, there are no easy answers here," wrote Stamos in a [14]Twitter thread , insisting it's okay to have nuanced opinions on these issues. "I find myself constantly torn between wanting everybody to have access to cryptographic privacy and the reality of the scale and depth of harm that has been enabled by modern comms technologies."
He said he's happy to see Apple finally take some responsibility for the impact of its massive platform but is also frustrated with its approach. "They both moved the ball forward technically while hurting the overall effort to find policy balance."
Stamos has [15]speculated this system could allow Apple to introduce end-to-end encryption for iCloud backups by preempting the inevitable concern about CSAM that would come up if it did so. Apple however had not publicly stated any intention to deploy full iCloud encryption.
All down to Apple
On Monday, Eric Rescorla, CTO of Mozilla, published [16]a technical analysis of Apple's system that suggests the security of the company's CSAM scanning effort depends on Apple behaving in a trustworthy manner. "It's important to realize that there's nothing in the system that prevents Apple from scanning photos that never leave the device; they've just chosen not to do so," he wrote.
Apple's "child safety" initiative represents a major shift for the company that just a few years ago cited the importance of "personal safety" by rejecting the FBI's request that it modify its software [17]to decrypt the iPhone of a terror suspect .
[18]
In [19]an open letter to its customers in 2016, Apple explained its defense of privacy by stating, "we believe the contents of your iPhone are none of our business."
"Specifically, the FBI wants us to make a new version of the iPhone operating system, circumventing several important security features, and install it on an iPhone recovered during the investigation," the company said. "In the wrong hands, this software – which does not exist today – would have the potential to unlock any iPhone in someone’s physical possession."
Starting with forthcoming operating system updates iOS 15, iPadOS 15, watchOS 8, and macOS Monterey, the contents of your iPhone will be Apple's business if you sync images to iCloud.
Security experts are concerned Apple's system will allow government authorities to demand that the company add non-CSAM image hashes to its detection list to ferret out photos deemed unacceptable for political, religious, or other reasons unrelated to child safety.
Five years ago, Apple said the government could be expected to demand that sort of technical intervention in a legal filing
"Here, if Apple is forced to create software in this case, other law enforcement agencies will seek similar orders to help them hack thousands of other phones, as FBI Director Comey confirmed when he said he would 'of course' use the All Writs Act to 'return to the courts in future cases to demand that Apple and other private companies assist . . . in unlocking secure devices,'" explained the company's legal representatives.
Would Apple cave?
Yet in the FAQs published on Monday, Apple attempts to ally concern that authorities could demand access to its CSAM system for other surveillance purposes by stating that the company would simply resist. "Could governments force Apple to add non-CSAM images to the hash list?" Apple asks, and then answers, "Apple will refuse any such demands."
Legal experts have not been impressed. "So basically: all that stands between users and governments demanding [the addition] of non-CSAM images to the hash list is Apple's firm refusal?" said Elizabeth Joh, law professor at UC Davis, [21]via Twitter .
To which Daphne Keller, Platform Regulation Director, Stanford Cyber Policy Center, and former Associate General Counsel at Google, [22]replied , "Speaking as someone who has litigated and lost on this exact issue in three countries (UK, Germany, France), I feel confident in saying the firm refusal to filter for new things beyond CSAM doesn’t mean much in the face of state power."
Indeed, when China directed Apple to enforce its ban on VPN software, the company [23]complied .
But there's a more basic concern about Apple's utilization of its customers' devices in a way that might be used against them: ownership and control. Ben Thompson, a business analyst who writes the [24]Stratechery blog , described Apple's approach as a mistake.
"One’s device ought to be one’s property, with all of the expectations of ownership and privacy that entails; cloud services, meanwhile, are the property of their owners as well, with all of the expectations of societal responsibility and law-abiding which that entails," he wrote.
"It’s truly disappointing that Apple got so hung up on its particular vision of privacy that it ended up betraying the fulcrum of user control: being able to trust that your device is truly yours." ®
Get our [25]Tech Resources
[1] https://www.apple.com/child-safety/
[2] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_security/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2YRGlnNaWjC2TH3joErd9EgAAARM&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_security/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YRGlnNaWjC2TH3joErd9EgAAARM&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_security/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YRGlnNaWjC2TH3joErd9EgAAARM&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[5] https://www.apple.com/child-safety/pdf/CSAM_Detection_Technical_Summary.pdf
[6] https://twitter.com/jonathanmayer/status/1423690035405201408?s=20
[7] https://www.apple.com/child-safety/pdf/Expanded_Protections_for_Children_Frequently_Asked_Questions.pdf
[8] https://appleprivacyletter.com/
[9] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_security/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YRGlnNaWjC2TH3joErd9EgAAARM&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[10] https://www.theregister.com/2021/08/05/apple_csam_scanning/
[11] https://www.theregister.com/2020/07/06/revised_earn_it_act/
[12] https://www.theregister.com/2020/03/06/earn_it_bill_encryption/
[13] https://www.theregister.com/2021/07/29/android_privacy_changes/
[14] https://twitter.com/alexstamos/status/1424054544556646407?s=20
[15] https://twitter.com/alexstamos/status/1424054578438307840?s=20
[16] https://educatedguesswork.org/posts/apple-csam-intro/
[17] https://www.theregister.com/2017/11/20/warrant_texas_shooter_iphone/
[18] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_security/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YRGlnNaWjC2TH3joErd9EgAAARM&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[19] https://www.apple.com/customer-letter/
[20] https://www.eff.org/files/2016/03/15/apple-reply-to-govt-opposition-to-apple-motion-to-vacate.pdf
[21] https://twitter.com/elizabeth_joh/status/1424737315864289283?s=20
[22] https://twitter.com/daphnehk/status/1424765665320214529?s=20
[23] https://www.reuters.com/article/us-china-apple-vpn/apple-says-it-is-removing-vpn-services-from-china-app-store-idUSKBN1AE0BQ
[24] https://stratechery.com/2021/apples-mistake/
[25] https://whitepapers.theregister.com/
The iDevice biz revealed two [1]child safety initiatives that are initially being rolled out in the US and later in other countries depending on regulatory approval.
One is a system to alert children and their parents when the Messages app sends or receives pictures deemed explicit (but not necessarily illegal) by an on-device machine learning algorithm.
[2]
The other is system to scan photos on iOS and iPadOS devices that sync to iCloud Photos to see if the hashes (identifiers) of on-device images match any hashes of known CSAM material (illegal), which Apple will use to flag iCloud accounts for cancellation and reporting to the National Center for Missing and Exploited Children (NCMEC).
[3]
[4]
Apple published a technical summary
[5]PDF
of its systems and said that for its CSAM scheme there's only "a one in one trillion chance per year of incorrectly flagging a given account." It, however, provided [6]no way to verify that figure, according to Princeton professor Jonathan Mayer.Went about as well as you'd expect
The announcement
[7]PDF
elicited a swift reaction in the form of [8]an open letter opposing the move for its potential harm to privacy and security."Apple's current path threatens to undermine decades of work by technologists, academics and policy advocates towards strong privacy-preserving measures being the norm across a majority of consumer electronic devices and use cases," said the letter, which currently lists more than 6,000 signatures. "We ask that Apple reconsider its technology rollout, lest it undo that important work."
Technical experts, privacy advocates, academics, and others have spent the past weekend debating the issue via social and online media. Pretty much everyone agrees that CSAM is a problem.
[9]
The question is whether a company that has said, "Privacy is a human right" – even if it doesn't offer that in China – should be attempting to tackle child safety by running its own scanning code – with consent obtained via the decision to use iCloud Photos rather than explicit permission – on its customers' devices.
[10]Apple is about to start scanning iPhone users' devices for banned content, warns professor
[11]You may be distracted by the pandemic but FYI: US Senate panel OK's backdoors-by-the-backdoor EARN IT Act
[12]Don't be fooled, experts warn, America's anti-child-abuse EARN IT Act could burn encryption to the ground
[13]Upcoming Android privacy changes include ability to blank advertising ID, and 'safety section' in Play store
Alex Stamos, director of the Stanford Internet Observatory and former CSO of Facebook, attempted to stake out the middle ground between those horrified by Apple's approach and those who would give the fight against CSAM priority over any other considerations.
"In my opinion, there are no easy answers here," wrote Stamos in a [14]Twitter thread , insisting it's okay to have nuanced opinions on these issues. "I find myself constantly torn between wanting everybody to have access to cryptographic privacy and the reality of the scale and depth of harm that has been enabled by modern comms technologies."
He said he's happy to see Apple finally take some responsibility for the impact of its massive platform but is also frustrated with its approach. "They both moved the ball forward technically while hurting the overall effort to find policy balance."
Stamos has [15]speculated this system could allow Apple to introduce end-to-end encryption for iCloud backups by preempting the inevitable concern about CSAM that would come up if it did so. Apple however had not publicly stated any intention to deploy full iCloud encryption.
All down to Apple
On Monday, Eric Rescorla, CTO of Mozilla, published [16]a technical analysis of Apple's system that suggests the security of the company's CSAM scanning effort depends on Apple behaving in a trustworthy manner. "It's important to realize that there's nothing in the system that prevents Apple from scanning photos that never leave the device; they've just chosen not to do so," he wrote.
Apple's "child safety" initiative represents a major shift for the company that just a few years ago cited the importance of "personal safety" by rejecting the FBI's request that it modify its software [17]to decrypt the iPhone of a terror suspect .
[18]
In [19]an open letter to its customers in 2016, Apple explained its defense of privacy by stating, "we believe the contents of your iPhone are none of our business."
"Specifically, the FBI wants us to make a new version of the iPhone operating system, circumventing several important security features, and install it on an iPhone recovered during the investigation," the company said. "In the wrong hands, this software – which does not exist today – would have the potential to unlock any iPhone in someone’s physical possession."
Starting with forthcoming operating system updates iOS 15, iPadOS 15, watchOS 8, and macOS Monterey, the contents of your iPhone will be Apple's business if you sync images to iCloud.
Security experts are concerned Apple's system will allow government authorities to demand that the company add non-CSAM image hashes to its detection list to ferret out photos deemed unacceptable for political, religious, or other reasons unrelated to child safety.
Five years ago, Apple said the government could be expected to demand that sort of technical intervention in a legal filing
[20]PDF
opposing the FBI's request to modify its software."Here, if Apple is forced to create software in this case, other law enforcement agencies will seek similar orders to help them hack thousands of other phones, as FBI Director Comey confirmed when he said he would 'of course' use the All Writs Act to 'return to the courts in future cases to demand that Apple and other private companies assist . . . in unlocking secure devices,'" explained the company's legal representatives.
Would Apple cave?
Yet in the FAQs published on Monday, Apple attempts to ally concern that authorities could demand access to its CSAM system for other surveillance purposes by stating that the company would simply resist. "Could governments force Apple to add non-CSAM images to the hash list?" Apple asks, and then answers, "Apple will refuse any such demands."
Legal experts have not been impressed. "So basically: all that stands between users and governments demanding [the addition] of non-CSAM images to the hash list is Apple's firm refusal?" said Elizabeth Joh, law professor at UC Davis, [21]via Twitter .
To which Daphne Keller, Platform Regulation Director, Stanford Cyber Policy Center, and former Associate General Counsel at Google, [22]replied , "Speaking as someone who has litigated and lost on this exact issue in three countries (UK, Germany, France), I feel confident in saying the firm refusal to filter for new things beyond CSAM doesn’t mean much in the face of state power."
Indeed, when China directed Apple to enforce its ban on VPN software, the company [23]complied .
But there's a more basic concern about Apple's utilization of its customers' devices in a way that might be used against them: ownership and control. Ben Thompson, a business analyst who writes the [24]Stratechery blog , described Apple's approach as a mistake.
"One’s device ought to be one’s property, with all of the expectations of ownership and privacy that entails; cloud services, meanwhile, are the property of their owners as well, with all of the expectations of societal responsibility and law-abiding which that entails," he wrote.
"It’s truly disappointing that Apple got so hung up on its particular vision of privacy that it ended up betraying the fulcrum of user control: being able to trust that your device is truly yours." ®
Get our [25]Tech Resources
[1] https://www.apple.com/child-safety/
[2] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_security/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2YRGlnNaWjC2TH3joErd9EgAAARM&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_security/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YRGlnNaWjC2TH3joErd9EgAAARM&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_security/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YRGlnNaWjC2TH3joErd9EgAAARM&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[5] https://www.apple.com/child-safety/pdf/CSAM_Detection_Technical_Summary.pdf
[6] https://twitter.com/jonathanmayer/status/1423690035405201408?s=20
[7] https://www.apple.com/child-safety/pdf/Expanded_Protections_for_Children_Frequently_Asked_Questions.pdf
[8] https://appleprivacyletter.com/
[9] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_security/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YRGlnNaWjC2TH3joErd9EgAAARM&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[10] https://www.theregister.com/2021/08/05/apple_csam_scanning/
[11] https://www.theregister.com/2020/07/06/revised_earn_it_act/
[12] https://www.theregister.com/2020/03/06/earn_it_bill_encryption/
[13] https://www.theregister.com/2021/07/29/android_privacy_changes/
[14] https://twitter.com/alexstamos/status/1424054544556646407?s=20
[15] https://twitter.com/alexstamos/status/1424054578438307840?s=20
[16] https://educatedguesswork.org/posts/apple-csam-intro/
[17] https://www.theregister.com/2017/11/20/warrant_texas_shooter_iphone/
[18] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_security/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YRGlnNaWjC2TH3joErd9EgAAARM&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[19] https://www.apple.com/customer-letter/
[20] https://www.eff.org/files/2016/03/15/apple-reply-to-govt-opposition-to-apple-motion-to-vacate.pdf
[21] https://twitter.com/elizabeth_joh/status/1424737315864289283?s=20
[22] https://twitter.com/daphnehk/status/1424765665320214529?s=20
[23] https://www.reuters.com/article/us-china-apple-vpn/apple-says-it-is-removing-vpn-services-from-china-app-store-idUSKBN1AE0BQ
[24] https://stratechery.com/2021/apples-mistake/
[25] https://whitepapers.theregister.com/
AnoNymousGerbil
Staff probably in india working getting minimum wage and don't care what happens, just click "YES!" like those Google employed people manually verifying disputed copyright claims.
Anonymous Coward
Govs tell apple they restrict apple markets unless they give in, or that there be leaked pictures of someone at apple with eppstein or something and they give in faster than anything...
There's no way of verifying trusted way where or what that HASHDATA has as source images.
Besides that all one has to next wait is some neat crypto UNlocker (or just hoax of that) for appleOS that tells "gib uz bitmonies or else we UNcrypt some nastyimages to your photo library" and see how that goes among the users...
There's so many ways one can think to exploit this system it aint even funny and they can't be that blind they don't see those.
I do not trust Apple in this kinda things, just remember few years back FaceTime issue where one could spy others through that without people being aware. Things like that can prevented easier, but if' there's basically whole system-wide "backdoor" to something like this, how long they think it takes it to be exploited...
I was just about to buy M1 macbook air, had been looking it last week but hadn't been pressing the BUY yet (deal ends 15th so I was waiting if anything better comes alone). There is no way I will now do that. I'll stick to my macmini until it's done and start to slowly jump the ship to other platforms. For phone it will hurt most because I have been using iOS since the beginning...
You asked for it, you got it
Snake
You wanted Apple to be your nanny state for as long as you believed it was suiting your purposes (walled garden, Safari proxy, blocked apps, etc etc etc).
You gave them approval, so now they are simply stepping up the level of nannism.
Think of the children. You were OK with all that as long as you believed you had "nothing to hide".
Enjoy Big Brother. You've earned it.
It has nothing to do with kiddie porn.....
They just want to be able to run their classifier over every picture & video in a users private piece of kit.
it it designed to :
1. set a legal precedent
2. use existing material to train their A.I on other none related material.
3. allow their staff to access private content to validate results.....
think they over looked one small matter....
for their staff to validate the results.... it requires them to load the "kiddie porn" onto a viewing device controlled by apple, to be viewed by staff employed by apple...
or are they going to use a 3rd party?