Twitter uses HackerOne bounties to find biases in its image-cropping AI model
(2021/08/02)
- Reference: 1627890845
- News link: https://www.theregister.co.uk/2021/08/02/twitter_now_crowdsourcing_solutions_to/
- Source link:
Twitter's saliency algorithm – otherwise known as its automated image cropping tool – has a problem with gender and race bias. The micro-blogging service is hoping to fix it by offering what it reckons is the industry's first algorithmic bias bounty competition.
The saliency algorithm employed by Twitter uses machine learning to crop images around the first spot eyes most frequently fall. In fall of 2020, some users complained the [1]image cropping favoured light skin over dark, and women's legs and breasts over their faces.
Twitter promised to investigate, [2]decided the machine learning code it employed was not really ready to have the keys to the castle, and gave the image-cropping controls back to humans. Employees [3]published an academic paper describing the problem, and the company made its code public through [4]GitHub .
[5]
Now the company has re-shared the saliency model, and its code, and asked for help in its improvement.
[6]
[7]
Twitter META Director Rumman Chowdhurty and Product Manager Jutta Williams [8]wrote on the company blog:
We want to take this work a step further by inviting and incentivizing the community to help identify potential harms of this algorithm beyond what we identified ourselves.
The competition is available through [9]HackerOne and is a part of [10]DEF CON AI Village with five cash prizes ranging from US$500 to US$3500 up for grabs. The bounty program is open until August 6 and the winners will be announced on August 8 at a Twitter DEF CON workshop.
"Your mission is to demonstrate what potential harms such an algorithm may introduce," states Twitter's HackerOne post.
[11]Google revamps bug bounty program
[12]Compsci student walks off with $50,000 after bug bounty report blows gaping hole in Shopify software repos
[13]AWS launches BugBust contest: Help fix a $100m problem for a $12 tshirt
To score a bounty, participants must find harms that come from the process of cropping or displaying images and videos. Extra points will be awarded to participants who detail harms falling on marginalized communities. The harms can be either malicious or unintentional – although unintentional harms seem to gain more points, as do situations that affect more users or are more severely damaging to a person's well-being.
Users employing denial of service, model inversion and Black-box model extraction or copying attacks will be disqualified, as will those that lead to exploitable behaviours like remote code extraction.
It's an interesting problem for Twitter to be outsourcing, and if successful will have implications reaching into [14]facial recognition technology and related AI problems. As one professional working in digital security puts it, it requires reverse-engineering human biases.
When you're auditing something like a saliency algorithm on a platform like Twitter, you're not just looking for the bias in the algorithm but basically trying to reverse-engineer the biases it reflects in a part of human vision we don't precisely understand! It's a hefty lift. — Queen of Summerween 🎃🏖✨ (@KateRoseBee) [15]July 30, 2021
Twitter has, of course, outsourced assessments of its biases before – albeit in a very ad hoc fashion as millions of users daily damn or praise the tweets it serves. The company has also frequently been accused of bias against politically conservative voices. Opening itself to a formal assessment of algorithmic bias will probably do little to silence its critics. ®
Get our [16]Tech Resources
[1] https://www.theregister.com/2020/09/21/twitter_image_cropping_ai/
[2] https://www.theregister.com/2021/05/19/twitter_machine_learning_image_cropping/
[3] https://arxiv.org/abs/2105.08667
[4] https://github.com/twitter-research/image-crop-analysis
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2YQfCXaNPwYYeeG8pI@Pg4QAAAIQ&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[6] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YQfCXaNPwYYeeG8pI@Pg4QAAAIQ&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[7] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YQfCXaNPwYYeeG8pI@Pg4QAAAIQ&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[8] https://blog.twitter.com/engineering/en_us/topics/insights/2021/algorithmic-bias-bounty-challenge
[9] https://hackerone.com/twitter-algorithmic-bias?type=team
[10] https://aivillage.org/
[11] https://www.theregister.com/2021/07/28/google_revamps_vulnerability_reward_program/
[12] https://www.theregister.com/2021/07/27/shopify_bug_bounty_payout/
[13] https://www.theregister.com/2021/06/25/aws_bugbust_contest/
[14] https://www.theregister.com/2021/07/01/us_facial_recognition_backlash/
[15] https://twitter.com/KateRoseBee/status/1421226332759883776?ref_src=twsrc%5Etfw
[16] https://whitepapers.theregister.com/
The saliency algorithm employed by Twitter uses machine learning to crop images around the first spot eyes most frequently fall. In fall of 2020, some users complained the [1]image cropping favoured light skin over dark, and women's legs and breasts over their faces.
Twitter promised to investigate, [2]decided the machine learning code it employed was not really ready to have the keys to the castle, and gave the image-cropping controls back to humans. Employees [3]published an academic paper describing the problem, and the company made its code public through [4]GitHub .
[5]
Now the company has re-shared the saliency model, and its code, and asked for help in its improvement.
[6]
[7]
Twitter META Director Rumman Chowdhurty and Product Manager Jutta Williams [8]wrote on the company blog:
We want to take this work a step further by inviting and incentivizing the community to help identify potential harms of this algorithm beyond what we identified ourselves.
The competition is available through [9]HackerOne and is a part of [10]DEF CON AI Village with five cash prizes ranging from US$500 to US$3500 up for grabs. The bounty program is open until August 6 and the winners will be announced on August 8 at a Twitter DEF CON workshop.
"Your mission is to demonstrate what potential harms such an algorithm may introduce," states Twitter's HackerOne post.
[11]Google revamps bug bounty program
[12]Compsci student walks off with $50,000 after bug bounty report blows gaping hole in Shopify software repos
[13]AWS launches BugBust contest: Help fix a $100m problem for a $12 tshirt
To score a bounty, participants must find harms that come from the process of cropping or displaying images and videos. Extra points will be awarded to participants who detail harms falling on marginalized communities. The harms can be either malicious or unintentional – although unintentional harms seem to gain more points, as do situations that affect more users or are more severely damaging to a person's well-being.
Users employing denial of service, model inversion and Black-box model extraction or copying attacks will be disqualified, as will those that lead to exploitable behaviours like remote code extraction.
It's an interesting problem for Twitter to be outsourcing, and if successful will have implications reaching into [14]facial recognition technology and related AI problems. As one professional working in digital security puts it, it requires reverse-engineering human biases.
When you're auditing something like a saliency algorithm on a platform like Twitter, you're not just looking for the bias in the algorithm but basically trying to reverse-engineer the biases it reflects in a part of human vision we don't precisely understand! It's a hefty lift. — Queen of Summerween 🎃🏖✨ (@KateRoseBee) [15]July 30, 2021
Twitter has, of course, outsourced assessments of its biases before – albeit in a very ad hoc fashion as millions of users daily damn or praise the tweets it serves. The company has also frequently been accused of bias against politically conservative voices. Opening itself to a formal assessment of algorithmic bias will probably do little to silence its critics. ®
Get our [16]Tech Resources
[1] https://www.theregister.com/2020/09/21/twitter_image_cropping_ai/
[2] https://www.theregister.com/2021/05/19/twitter_machine_learning_image_cropping/
[3] https://arxiv.org/abs/2105.08667
[4] https://github.com/twitter-research/image-crop-analysis
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2YQfCXaNPwYYeeG8pI@Pg4QAAAIQ&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[6] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YQfCXaNPwYYeeG8pI@Pg4QAAAIQ&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[7] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YQfCXaNPwYYeeG8pI@Pg4QAAAIQ&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[8] https://blog.twitter.com/engineering/en_us/topics/insights/2021/algorithmic-bias-bounty-challenge
[9] https://hackerone.com/twitter-algorithmic-bias?type=team
[10] https://aivillage.org/
[11] https://www.theregister.com/2021/07/28/google_revamps_vulnerability_reward_program/
[12] https://www.theregister.com/2021/07/27/shopify_bug_bounty_payout/
[13] https://www.theregister.com/2021/06/25/aws_bugbust_contest/
[14] https://www.theregister.com/2021/07/01/us_facial_recognition_backlash/
[15] https://twitter.com/KateRoseBee/status/1421226332759883776?ref_src=twsrc%5Etfw
[16] https://whitepapers.theregister.com/
Should have asked the TV advertisers
Jimmy2Cows
The saliency algorithm employed by Twitter uses machine learning to crop images around the first spot eyes most frequently fall. In fall of 2020, some users complained the image cropping favoured light skin over dark, and women's legs and breasts over their faces.
Everything new is old. TV advertisers have known for decades where eyes most frequently fall in an image, using that knowledge to place their product accordingly. Clue... when the image contains people, often where viewers first look is at features of sexual and/or physical attractiveness.
Twitter has simply re-discovered basic human nature. It's not an algorithmic bias, they just chose the wrong algorithm to detect where to crop around. Should've tried to locate faces, then go from there. Which, as we've seen over and over, has other problems if using AI for detection.
Korev
Interestingly "Bounty" is sometimes used as a racist insult (ie black on the outside, white on the inside); perhaps not the best choice of terminology...
Avoidance
How to avoid paying wages?
Create a competition! Instead of spending money on hiring and getting right workers to do the job you need done, just offer $500 and poor souls from around the world will do all the work you need, even if you only pay a handful of people.
In many countries companies must have to pay at least minimum wage.