Twitter's AI image-crop algo is biased towards people who look younger, skinnier, and whiter, bounty challenge at DEF CON reveals
(2021/08/11)
- Reference: 1628643106
- News link: https://www.theregister.co.uk/2021/08/11/defcon_twitter_ai/
- Source link:
Twitter’s image-cropping AI algorithm favors people who appear younger, thinner, and have fairer skin as well as those that are able-bodied.
The saliency algorithm is used to automatically crop pictures posted on the social media platform. It focuses on the most interesting parts of the image to catch people’s attention as they scroll through their Twitter feeds. Last year, netizens [1]discovered the tool preferred photographs of women over men, and those with lighter skin over darker skin.
Engineers at Twitter’s ML Ethics, Transparency and Accountability (META) team, later [2]confirmed these biases. In an attempt to discover other potential flaws in its image cropping algorithm, the group sponsored an [3]algorithmic bias bounty competition hosted at this year’s [4]DEF CON hacking conference in Las Vegas and organised by AI Village, a community of hackers and data scientists working at the intersection of machine learning and security.
[5]
The top three results announced this week revealed Twitter’s saliency algorithm preferred people who appeared more conventionally attractive, English over Arabic, and was more likely to crop out people in wheelchairs.
[6]
[7]
The winner, Bogdan Kulynych, a graduate student at the École polytechnique fédérale de Lausanne in Switzerland, was awarded $3,500. He generated a series of fake faces and tweaked their appearances to test which ones were ranked highest in saliency scores by Twitter’s algorithm.
“The target model is biased towards...depictions of people that appear slim, young, of light or warm skin color and smooth skin texture, and with stereotypically feminine facial traits,” he [8]said .
[9]Twitter uses HackerOne bounties to find biases in its image-cropping AI model
[10]Twitter U-turns after conferring society's highest honor – a blue check mark – on very obvious bot accounts
[11]YouTube's recommendation engine is pretty naff, Mozilla study finds
[12]Google tweaks Android Messages app to auto-classify or auto-delete messages
Second place went to Halt AI, a Canadian startup, found the problematic tool had [13]spatial biases and it was more likely to crop out people lower down in photographs sitting in wheelchairs compared to those standing upright. The elderly with white or gray hair was also more likely to be left out in image previews created by the algorithm. Halt AI won $1,000 in funds.
Roya Pakzad, who founded TaraazResearch, a non-profit org focused on technology and human rights, came in third place. She [14]translated texts in memes from English to Arabic and fed them to the saliency algorithm, the software favored the English ones. Twitter favors English-speaking users more likely to be in Western countries over other languages. She was awarded $500.
[15]
Twitter said it had stopped using the auto image-cropping tool since March on its mobile app. Rich Harang, a machine-learning researcher volunteering for AI Village, applauded Twitter for sponsoring the challenge. “Twitter has the same problem now that anyone who runs a bug bounty contest runs: how to fix what the participants found,” he told The Register .
“The results have given Twitter several new examples of representational harm that their saliency model caused, but maybe more importantly, the new tools and approaches that the contest participants developed can help Twitter in continuing to look for additional bias issues beyond representational harms. Twitter's next step is to figure out how to take these new tools and mitigate the new harms that have been discovered.”
The Register has contacted Twitter for comment. ®
Get our [16]Tech Resources
[1] https://www.theregister.com/2020/09/21/twitter_image_cropping_ai/
[2] https://www.theregister.com/2021/05/19/twitter_machine_learning_image_cropping/
[3] https://www.theregister.com/2021/08/02/twitter_now_crowdsourcing_solutions_to/
[4] https://defcon.org/
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2YRNLdq9Qth@KCNlvtA@ccwAAANM&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[6] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YRNLdq9Qth@KCNlvtA@ccwAAANM&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[7] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YRNLdq9Qth@KCNlvtA@ccwAAANM&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[8] https://github.com/bogdan-kulynych/saliency_bias
[9] https://www.theregister.com/2021/08/02/twitter_now_crowdsourcing_solutions_to/
[10] https://www.theregister.com/2021/07/13/twitter_verified_fake/
[11] https://www.theregister.com/2021/07/08/youtubes_mozilla_algorithm/
[12] https://www.theregister.com/2021/06/30/android_messages_india_only_filters/
[13] https://github.com/erickmu1/Twitter-Algorithmic-Bias
[14] https://github.com/royapakzad/image-crop-analysis
[15] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YRNLdq9Qth@KCNlvtA@ccwAAANM&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[16] https://whitepapers.theregister.com/
The saliency algorithm is used to automatically crop pictures posted on the social media platform. It focuses on the most interesting parts of the image to catch people’s attention as they scroll through their Twitter feeds. Last year, netizens [1]discovered the tool preferred photographs of women over men, and those with lighter skin over darker skin.
Engineers at Twitter’s ML Ethics, Transparency and Accountability (META) team, later [2]confirmed these biases. In an attempt to discover other potential flaws in its image cropping algorithm, the group sponsored an [3]algorithmic bias bounty competition hosted at this year’s [4]DEF CON hacking conference in Las Vegas and organised by AI Village, a community of hackers and data scientists working at the intersection of machine learning and security.
[5]
The top three results announced this week revealed Twitter’s saliency algorithm preferred people who appeared more conventionally attractive, English over Arabic, and was more likely to crop out people in wheelchairs.
[6]
[7]
The winner, Bogdan Kulynych, a graduate student at the École polytechnique fédérale de Lausanne in Switzerland, was awarded $3,500. He generated a series of fake faces and tweaked their appearances to test which ones were ranked highest in saliency scores by Twitter’s algorithm.
“The target model is biased towards...depictions of people that appear slim, young, of light or warm skin color and smooth skin texture, and with stereotypically feminine facial traits,” he [8]said .
[9]Twitter uses HackerOne bounties to find biases in its image-cropping AI model
[10]Twitter U-turns after conferring society's highest honor – a blue check mark – on very obvious bot accounts
[11]YouTube's recommendation engine is pretty naff, Mozilla study finds
[12]Google tweaks Android Messages app to auto-classify or auto-delete messages
Second place went to Halt AI, a Canadian startup, found the problematic tool had [13]spatial biases and it was more likely to crop out people lower down in photographs sitting in wheelchairs compared to those standing upright. The elderly with white or gray hair was also more likely to be left out in image previews created by the algorithm. Halt AI won $1,000 in funds.
Roya Pakzad, who founded TaraazResearch, a non-profit org focused on technology and human rights, came in third place. She [14]translated texts in memes from English to Arabic and fed them to the saliency algorithm, the software favored the English ones. Twitter favors English-speaking users more likely to be in Western countries over other languages. She was awarded $500.
[15]
Twitter said it had stopped using the auto image-cropping tool since March on its mobile app. Rich Harang, a machine-learning researcher volunteering for AI Village, applauded Twitter for sponsoring the challenge. “Twitter has the same problem now that anyone who runs a bug bounty contest runs: how to fix what the participants found,” he told The Register .
“The results have given Twitter several new examples of representational harm that their saliency model caused, but maybe more importantly, the new tools and approaches that the contest participants developed can help Twitter in continuing to look for additional bias issues beyond representational harms. Twitter's next step is to figure out how to take these new tools and mitigate the new harms that have been discovered.”
The Register has contacted Twitter for comment. ®
Get our [16]Tech Resources
[1] https://www.theregister.com/2020/09/21/twitter_image_cropping_ai/
[2] https://www.theregister.com/2021/05/19/twitter_machine_learning_image_cropping/
[3] https://www.theregister.com/2021/08/02/twitter_now_crowdsourcing_solutions_to/
[4] https://defcon.org/
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2YRNLdq9Qth@KCNlvtA@ccwAAANM&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[6] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YRNLdq9Qth@KCNlvtA@ccwAAANM&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[7] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YRNLdq9Qth@KCNlvtA@ccwAAANM&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[8] https://github.com/bogdan-kulynych/saliency_bias
[9] https://www.theregister.com/2021/08/02/twitter_now_crowdsourcing_solutions_to/
[10] https://www.theregister.com/2021/07/13/twitter_verified_fake/
[11] https://www.theregister.com/2021/07/08/youtubes_mozilla_algorithm/
[12] https://www.theregister.com/2021/06/30/android_messages_india_only_filters/
[13] https://github.com/erickmu1/Twitter-Algorithmic-Bias
[14] https://github.com/royapakzad/image-crop-analysis
[15] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YRNLdq9Qth@KCNlvtA@ccwAAANM&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[16] https://whitepapers.theregister.com/
JDPower666
Here's a novel idea to get round these issues - just show the whole frigging image!
so ?
Denarius
it does what it says on tin, selecting more interesting parts of images. Marketting droids aim for maximum interest with lipservice to anything else. What would one expect ? Deep Woke where anyone of low melanin is excluded ?
Leave out the older men?
Yeah, I'm all for the bots cropping me out of the pictures. That way I can hide better when the enevitable happens and the killer bots come rolling through the streets.