Apple stalls CSAM auto-scan on devices after 'feedback' from everyone on Earth
- Reference: 1630702086
- News link: https://www.theregister.co.uk/2021/09/03/apple_scanning_pause/
- Source link:
"Previously we announced plans for features intended to help protect children from predators who use communication tools to recruit and exploit them and to help limit the spread of Child Sexual Abuse Material," the company said in a statement posted to its [1]child safety webpage .
"Based on feedback from customers, advocacy groups, researchers and others, we have decided to take additional time over the coming months to collect input and make improvements before releasing these critically important child safety features."
We have decided to take additional time over the coming months to collect input and make improvements
Last month, Apple announced its child safety initiative, which involves adding a nudity detection algorithm to its Messages chat client, to provide a way to control the sharing of explicit images, and running code on customer's iDevices to detect known child sexual abuse material among on-device photos destined for iCloud storage.
These features were due to debut in the public releases of iOS 15, iPadOS 15, watchOS 8, and macOS Monterey operating system software, expected later this month or next. But faced with objections from [2]more than 90 advocacy organizations , Apple has opted to pause the rollout.
[3]
ACLU attorney Jennifer Granick via Twitter heralded the delay as a victory for civil liberties advocacy. "It's great that Apple plans to engage with independent privacy and security experts before announcing their genius plans," she [4]said . "They should start with end to end encryption for iCloud backups."
[5]
[6]
Matthew Green, associate professor of computer science at the Johns Hopkins Information Security Institute, via Twitter urged Apple to [7]engage with the technical and policy communities , and to talk to the public, before rolling out new technology.
"This isn’t a fancy new Touchbar," he [8]said "It’s a privacy compromise that affects 1bn users."
[9]
Apple's declared goal, keeping children safe and preventing the distribution of illegal child sexual abuse material (CSAM), has broad support. But its approach does not. Its explicit photo intervention in its Messages app has been described as a [10]more of a danger to children than a benefit.
And its decision to conduct CSAM scans using owner's computing resources and customer-owned hardware for the scheme has been widely characterized as an erosion of property rights and backdoor that will be used for government surveillance and control.
As NSA whistleblower Edward Snowden [11]put it , "Apple plans to erase the boundary dividing which devices work for you, and which devices work for them."
[12]Apple wants to scan iCloud to protect kids, can't even keep them safe in its own App Store – report
[13]Fake Apple rep amasses 620,000+ stolen iCloud pics, vids in hunt for images of nude women to trade
[14]Apple's bright idea for CSAM scanning could start 'persecution on a global basis' – 90+ civil rights groups
[15]Apple says its CSAM scan code can be verified by researchers. Corellium starts throwing out dollar bills
Apple's plan also contradicts its own marketing about privacy. The Electronic Frontier Foundation, one of dozens of organizations that expressed concerns about Apple's plans, highlighted the company's reversal by citing the text of its 2019 CES billboard: "What happens on your iPhone, stays on your iPhone."
"Now that Apple has built [a backdoor], they will come," wrote EFF deputy executive director Kurt Opsahl in [16]a post last month. "With good intentions, Apple has paved the road to mandated security weakness around the world, enabling and reinforcing the arguments that, should the intentions be good enough, scanning through your personal life and private communications is acceptable."
[17]
In a statement emailed to The Register , Evan Greer, director of Fight for the Future, condemned Apple's "spyPhone" proposal.
"Apple’s plan to conduct on-device scanning of photos and messages is one of the most dangerous proposals from any tech company in modern history," she said. "Technologically, this is the equivalent of installing malware on millions of people’s devices – malware that can be easily abused to do enormous harm."
Apple’s plan to conduct on-device scanning of photos and messages is one of the most dangerous proposals from any tech company in modern history
Apple – rather than actually engaging with the security community and the public – published a list of Frequently Asked Questions and responses to address the concern that censorious governments will demand access to the CSAM scanning system to look for politically objectionable images.
"Could governments force Apple to add non-CSAM images to the hash list?" the company asked in its interview of itself, and then responded, "No. Apple would refuse such demands and our system has been designed to prevent that from happening."
Apple however has not refused government demands in China with regard to VPNs or [18]censorship . Nor has it refused government demands in Russia, with regard to its 2019 law requiring pre-installed Russian apps.
Tech companies uniformly say they comply with all local laws. So if China, Russia, or the US were to pass a law requiring on-device scanning to be adapted to address "national security concerns" or some other plausible cause, Apple's choice would be to comply or face the consequences – it would no longer be able to say, "We can't do on-device scanning." ®
Get our [19]Tech Resources
[1] https://www.apple.com/child-safety/
[2] https://www.theregister.com/2021/08/19/apple_csam_condemned/
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_security/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2YTKbInwwlKkJBIhczPbybwAAAFU&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[4] https://twitter.com/granick/status/1433804876081889282?s=20
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_security/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YTKbInwwlKkJBIhczPbybwAAAFU&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[6] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_security/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YTKbInwwlKkJBIhczPbybwAAAFU&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[7] https://www.theregister.com/2021/08/18/apples_csam_hashing/
[8] https://twitter.com/matthew_d_green/status/1433783951085260835?s=20
[9] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_security/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YTKbInwwlKkJBIhczPbybwAAAFU&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[10] https://www.eff.org/deeplinks/2021/08/apples-plan-scan-photos-messages-turns-young-people-privacy-pawns
[11] https://edwardsnowden.substack.com/p/all-seeing-i
[12] https://www.theregister.com/2021/08/26/apple_app_age/
[13] https://www.theregister.com/2021/08/24/los_angeles_county_man_pretended/
[14] https://www.theregister.com/2021/08/19/apple_csam_condemned/
[15] https://www.theregister.com/2021/08/17/corellium_apple_bounty/
[16] https://www.eff.org/deeplinks/2021/08/if-you-build-it-they-will-come-apple-has-opened-backdoor-increased-surveillance
[17] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_security/front&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YTKbInwwlKkJBIhczPbybwAAAFU&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[18] https://www.theregister.com/2017/11/22/apple_vpns_china_skype/
[19] https://whitepapers.theregister.com/
"Could governments force Apple to add non-CSAM images to the hash list?" the company asked in its interview of itself, and then responded, "No. Apple would refuse such demands and our system has been designed to prevent that from happening."
But Apple aren't creating these hash lists, its done by a third party and will be constantly updates when new images are identified by law enforcement. So it would be trivial for government agencies to create a new hash, claim its of some abuse image and get it added to the list Apple uses.
Two things that would be totally acceptable: 1. Apple has the right to keep illegal materials off its servers. 2. Most people don’t want to look at some illegal materials at all, and would be happy if illegal materials can’t get on their phone.
That’s what they should have done. 1. When the phone uploads images to Apples servers, the phone just refuses. (And you have the choice to send the photo to a manual review).
2. When my browser downloads illegal images, it refuses and returns a status 403 instead, if I opt-in. Nobody learns about it, Apples lawyers make sure that I haven’t legally downloaded such an image, and opting in shows that I’m legally not attempting to download anything. So I have strong legal protection, and again nobody learns what’s been filtered out.
#1 would almost certainly breach their reporting obligations under federal law for CSAM. Telling the phone to just not upload it means they have knowledge of its existence. (Not directly, but by building a system that knows what to look for and then flagging it to just not upload, they're intentionally preventing receiving material that would have to be reported, which is probably not kosher.)
#2 would, too, because if there's a match to the point where something is preventing the download, there is awareness of a URL ostensibly containing it.
"Apple has the right to keep illegal materials off its servers", "When my browser downloads illegal images" - but how is it known to be illegal? That's the sticky point that's easier said than done. YOU know what's illegal, but to a computer, it's just random bits of, well, bits, hence why anyone can upload anything.
This is why the CSAM database exists, but to fit your preference, it would have to exist on your phone scanning your browser, which brings us back round to square one.
Hey Apple!
You took your reputation as "more on the side of your customer's privacy than any other vendor out there", you doused it in petrol, and set light to it.
"pause"? - Don't make me laugh cynically in your face. You think that "pausing" the rollout will help mend your reputation? Nope, you're toast. If you ever run another "privacy focused" ad campaign like you did recently in the UK you'll just remind customers that you're a bunch of hypocrites. The only thing that might save you is an about face, adding security mechanisms to your products that make it impossible for you to ever try adding anything like this again, and *proving* it to the public.
You've already, to my personal knowledge, cost yourself sales from this - i.e. people have said to me "I'm not buying any more Apple kit because of this." and one person I know is in the middle of eradicating Apple products from his house (quite a few) as a direct consequence of this.
Re: Hey Apple!
So just to be clear…Apple thus far has not enabled CSAM scanning on the server side, which how they've gotten away with that is unclear because it's a requirement under US federal law and probably elsewhere.
So you'd rather they go along with what every other cloud company already does and scan it on the server side without explicitly making users aware that's what's happening?
I get the implications of having an on-device scanner. It is absolutely not the privacy nightmare people are claiming, because there's no breach of privacy in scanning material you were uploading anyway. I do understand the possibility of a government requiring it to be adjusted to scan other things. But…
I also understand that if people are uploading all their photos to iCloud anyway (which is what's going on here), there's nothing stopping a government from requiring server-side scanning from any company, Apple included, for all these various nefarious purposes that people keep mentioning. Apple's not even allowed to run their own servers in China; neither is any other foreign company, to my knowledge. Beyond trying to promote local competition, why do you think that is?
Further, I understand that there was never anything stopping any government from handing Apple on-device scanning code and forcing them to adopt it for sales to continue in that area—or even just passing a law requiring that device makers do it themselves. The idea that literally the only thing preventing this has been Apple not developing on-device scanning is absolutely absurd. It's not like they invented the idea of it, nor are they even going about it in a particularly novel way as far as the matching goes.
There are problems with lots of things Apple does. The fact that people have seized upon this one (on-device scanning, not discussing iMessage AI recognition as that's a separate thing) as the end of them is mind-boggling to me.
It may not be as specific as you think
Rob Braxman has done a good job in presenting some ways what Apple is proposing actually works. Look him up on Odysee (or YouTube if you must). It looks a lot like the age old excuse of something being "for the children" when it's yet another trojan to get the army through the gates.
Rob shows that it isn't just hash lookups. For a hash to be identical, the images would have to be identical. There are already programs that can analyze photos and suggest the content with probabilities. He shows examples where the analysis reports the image is of a female in swimwear/underwear, posing suggestively, age, race etc. The image was a of a woman in a bathing suit, on a beach in what I'll call a yoga pose. There are a couple of more examples and they are so close it's scary. With the processing power going up and up on mobiles, doing this sort of analysis on a phone is easier and easier. Apples outline shows that they phone would do the analysis, create a hash of the description, build a locked thumbnail and upload to iCloud. If the program thinks you have too many suspects, it will rat you out and somebody could then review the thumbnails. The problem is they'll never have the human resources to do that so some bot will be making the decision about whether you are to be 'swatted' or not.
It doesn't have to be CSAM. The program can be instructed to look for A/V files from political rallies/protests. It might look for landmarks to track your location even when you are out of range or in airplane mode. Perhaps they'll trigger everybody's phone to be on the lookout for certain people when the phone is detected to be in a certain place at a certain time. The technologies are a spy master's dream come true.
My new phone is going to be deGoogled as soon as it arrives. I don't think I have anything to hide, but I am not interested in letting The Man forage through my private life and wrestle me to the ground if there is anything about me they object to. They less they know, the better off I am.
"Apple would refuse such demands"
Don't piss on my cornflakes and tell me it's raining