Research finds data poisoning can't beat facial recognition
(2022/03/15)
- Reference: 1647352805
- News link: https://www.theregister.co.uk/2022/03/15/research_finds_data_poisoning_cant/
- Source link:
If there was ever a reason to think data poisoning could fool facial recognition software, a recently published paper showed that reasoning is bunk.
Data poisoning software alters images by manipulating individual pixels. Those changes are invisible to the naked eye, but if effective they make them useless to facial recognition software.
Researchers from Stanford University, Oregon State University and Google teamed up for [1]the paper in which they single out two particular reasons why data poisoning won't keep people safe. First, the code written to "poison" photographs is freely available online. Second, there's no reason to assume a poisoned photo will be effective against future recognition models.
[2]
Far from providing security, the paper's authors said data poisoning to prevent facial recognition provides a false sense of security, and could actually harm users who wouldn't have posted photographs online otherwise.
[3]
[4]
The researchers faced off against two data poisoning programs: Fawkes and LowKey, both which subtly alter images at the pixel level that, while invisible to humans, is enough to confuse facial recognition software. Both are freely available online, and that's problem number one, the authors said.
"An adaptive model trainer with black-box access to the [poisoning method] employed by users can immediately train a robust model that resists poisoning." Unfortunately for those poisoning models, their code is freely available online, and the researchers said it will stay that way for as long as the products exist.
[5]IRS doesn't completely scrap facial recognition, just makes it optional
[6]Clearview AI plans tech to ID faces as they age, seek big government deals
[7]Techniques to fool AI with hidden triggers are outpacing defenses – study
[8]Facebook ditches its creepy, controversial robot – yes, its facial-recognition AI
With that availability in mind, the paper said it stands to reason facial recognition software companies are aware of poisoning software like Fawkes and LowKey. As the researchers show in the paper, all they needed was black box access to image poisoner code. There's no reason to assume the major players haven't already accounted for them, too.
There's another problem with data poisoning, though, and that is time.
[9]
"We find there exists an even simpler defensive strategy: model trainers can just wait for better facial recognition systems, which are no longer vulnerable to these particular poisoning attacks," the paper said.
In the cases the researchers examined, they didn't even have to wait that long: Both Fawkes and LowKey were ineffective against versions of facial recognition software released within a year of their appearance online (the same month for LowKey).
There's no arms race between poisoning and facial recognition to be found here, the researchers said. Poisoning attacks are only effective once, can likely be countered via a black box, and if that fails all the system has to do is wait for an update.
[10]
There's been plenty of experiments done on [11]fooling facial recognition with varying levels of success, and it looks like data poisoning is yet another unsuccessful attempt at promoting online privacy.
"In light of this, we argue that users' only hope is a push for legislation that restricts the use of privacy-invasive facial recognition systems," the paper said. ®
Get our [12]Tech Resources
[1] https://arxiv.org/abs/2106.14851
[2] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2YjDGO1ItC6kgWsDp2tfLPgAAAAU&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YjDGO1ItC6kgWsDp2tfLPgAAAAU&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YjDGO1ItC6kgWsDp2tfLPgAAAAU&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[5] https://www.theregister.com/2022/02/23/irs_facial_deletion/
[6] https://www.theregister.com/2022/02/28/in_brief_ai/
[7] https://www.theregister.com/2022/02/25/dnn-trojan-attacks/
[8] https://www.theregister.com/2021/11/02/facebook_facial_recognition/
[9] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YjDGO1ItC6kgWsDp2tfLPgAAAAU&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[10] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YjDGO1ItC6kgWsDp2tfLPgAAAAU&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[11] https://www.theregister.com/2021/09/16/makeup_facial_recognition/
[12] https://whitepapers.theregister.com/
Data poisoning software alters images by manipulating individual pixels. Those changes are invisible to the naked eye, but if effective they make them useless to facial recognition software.
Researchers from Stanford University, Oregon State University and Google teamed up for [1]the paper in which they single out two particular reasons why data poisoning won't keep people safe. First, the code written to "poison" photographs is freely available online. Second, there's no reason to assume a poisoned photo will be effective against future recognition models.
[2]
Far from providing security, the paper's authors said data poisoning to prevent facial recognition provides a false sense of security, and could actually harm users who wouldn't have posted photographs online otherwise.
[3]
[4]
The researchers faced off against two data poisoning programs: Fawkes and LowKey, both which subtly alter images at the pixel level that, while invisible to humans, is enough to confuse facial recognition software. Both are freely available online, and that's problem number one, the authors said.
"An adaptive model trainer with black-box access to the [poisoning method] employed by users can immediately train a robust model that resists poisoning." Unfortunately for those poisoning models, their code is freely available online, and the researchers said it will stay that way for as long as the products exist.
[5]IRS doesn't completely scrap facial recognition, just makes it optional
[6]Clearview AI plans tech to ID faces as they age, seek big government deals
[7]Techniques to fool AI with hidden triggers are outpacing defenses – study
[8]Facebook ditches its creepy, controversial robot – yes, its facial-recognition AI
With that availability in mind, the paper said it stands to reason facial recognition software companies are aware of poisoning software like Fawkes and LowKey. As the researchers show in the paper, all they needed was black box access to image poisoner code. There's no reason to assume the major players haven't already accounted for them, too.
There's another problem with data poisoning, though, and that is time.
[9]
"We find there exists an even simpler defensive strategy: model trainers can just wait for better facial recognition systems, which are no longer vulnerable to these particular poisoning attacks," the paper said.
In the cases the researchers examined, they didn't even have to wait that long: Both Fawkes and LowKey were ineffective against versions of facial recognition software released within a year of their appearance online (the same month for LowKey).
There's no arms race between poisoning and facial recognition to be found here, the researchers said. Poisoning attacks are only effective once, can likely be countered via a black box, and if that fails all the system has to do is wait for an update.
[10]
There's been plenty of experiments done on [11]fooling facial recognition with varying levels of success, and it looks like data poisoning is yet another unsuccessful attempt at promoting online privacy.
"In light of this, we argue that users' only hope is a push for legislation that restricts the use of privacy-invasive facial recognition systems," the paper said. ®
Get our [12]Tech Resources
[1] https://arxiv.org/abs/2106.14851
[2] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2YjDGO1ItC6kgWsDp2tfLPgAAAAU&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YjDGO1ItC6kgWsDp2tfLPgAAAAU&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YjDGO1ItC6kgWsDp2tfLPgAAAAU&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[5] https://www.theregister.com/2022/02/23/irs_facial_deletion/
[6] https://www.theregister.com/2022/02/28/in_brief_ai/
[7] https://www.theregister.com/2022/02/25/dnn-trojan-attacks/
[8] https://www.theregister.com/2021/11/02/facebook_facial_recognition/
[9] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YjDGO1ItC6kgWsDp2tfLPgAAAAU&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[10] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YjDGO1ItC6kgWsDp2tfLPgAAAAU&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[11] https://www.theregister.com/2021/09/16/makeup_facial_recognition/
[12] https://whitepapers.theregister.com/
Re: Data overload
msobkow
So all you have to do is convince the general public to use generated images instead of their own photos en-masse, and then, as the article points out, within a few months the effort is stymied anyhow.
Maybe people should just wake up to what was obvious to me in the mid '80s when the internet still ran on dial-up and ISDN:
THERE IS NO "PRIVACY" ON THE INTERNET. IT ISN'T DESIGNED FOR IT.
Re: Data overload
Doctor Syntax
Also the same image, real or fake, with multiple times with different identities attached and multiple faces with the same identity.
I wouldn't even call what they were doing poisoning. It doesn't force the signal to noise ratio into oblivion.
Arguably, swapping pixels won't be enough, but changing facial features digitally would
vekkq
Brain won't notice anyway.
Re: Arguably, swapping pixels won't be enough, but changing facial features digitally would
Swarthy
In all pictures uploaded, swap your left eye for your right, and have your mouth as a mirror image. That should do it.
Make pervasive facial recognition illegal?
Mike 16
Because as we all know, every law enforcement officer and politician is simply incapable of breaking the law, no matter what riches and powers await.
"You can make it illegal, but you can't make it unpopular" (allegedly said of prostitution by New Orleans mayor Martin Behrman)
quelle surprise
Adrian 4
Did anyone expect any different ?
Data overload
When the model is fed with faces maybe we should be feeding it generated fake faces instead of poisoned real faces.
There exist systems to generate fake faces. You can generate and publish many more fake faces than real faces exist. Each recognition program has a limited capacity to discriminate. That is the threshold you must reach. So, if 1 in 50 faces published online is a real face, then you reduce the chances of you being in the set. Generate a data overload by improving the generators.