Twitter: Our AI image-cropping algorithm is biased toward White people, women
(2021/05/20)
- Reference: 1621466780
- News link: https://www.theregister.co.uk/2021/05/19/twitter_machine_learning_image_cropping/
- Source link:
Twitter said its AI-powered image-cropping algorithm is slightly biased in favor of White people and women after all, and has taken steps to ditch its reliance on the machine-learning code.
The algorithm was used to automatically decide which portion of a photo shared on the social network was best for the screen viewing it. You can upload pictures in all sorts of sizes and aspect ratios; at some point the images have to be displayed, and Twitter used a trained system to decide which parts of the media to show. The option to view the whole image was available – the algorithm was used to create a preview of the media.
Specifically, the software relied on a [1]saliency algorithm designed to keep the most interesting and relevant parts of the image in the preview crop. For example, a snap taken of people at the beach should focus on their faces rather than the sky or sand.
[2]
[3]
[4]
Twitter users, however, spotted that the tool, when given a choice of things to pick, seemed to zero in on people with lighter skin and women’s chests. After the social network investigated the issue in October, it [5]told The Register it didn’t find any evidence of racial or gender biases, though admitted its engineers probably needed to conduct more tests.
Now, three of its techies in its Machine Learning Ethics, Transparency, and Accountability team and its Content Understanding Research team have discovered that the saliency algorithm is, in fact, skewed towards White people and women.
We're told the software is about eight per cent more skewed towards displaying women compared to men, and four per cent more for people with lighter skin compared to those with darker skin. When these groups were narrowed down by gender, there was a seven per cent difference in favor of White women compared to Black women, and two per cent for White men compared to black men.
[6]
The difference seems slight but with millions upon millions of pictures tweeted a day, a good number of people are going to encounter the bias.
The team conducted their tests by comparing the way the software handled images of Black females, Black males, White females, and White males, running the experiment 10,000 times. In the previous study, staff looked at how the code cropped White people versus Black people, White versus Indian, White versus Asian, and male versus female, running the tests just 200 times. The small sample size made it more difficult to detect any significant levels of bias.
[7]Researchers say objects can hide from computer vision by seeking out unusual company that trips correlation bias
[8]Facebook job ads algorithm still discriminates on gender, LinkedIn not so much
[9]AI brain drain to Google and pals threatens public sector's ability to moderate machine-learning bias
[10]There's nothing AI and automation can't solve – except bias and inequality in the workplace, says report
There seemed to be no clear evidence that the algorithm focused on parts of people’s bodies other than their faces, however.
“We found that no more than 3 out of 100 images per gender have the crop not on the head,” they explained in a [11]paper shared on arXiv this week. “The crops not on heads were due to high predicted salient scores on parts of the image such as a number on the jersey of sports players or a badge. These patterns were consistent across genders.”
It’s possible that the problematic crops of women’s chests were caused by logos or images on their clothes, which grabbed the attention of the code.
[12]
Since March, Twitter has gradually rolled out an [13]update for smartphone users so that its app doesn’t rely on its saliency algorithm, and instead gave people more control of how images are displayed. In short, Twitter now displays "standard aspect ratio photos in full on iOS and Android — meaning without the saliency algorithm crop," and "a true preview of the image in the tweet composer field, so tweet authors know how their tweets will look before they publish."
“We considered the trade-offs between the speed and consistency of automated cropping with the potential risks we saw in this research,” Rumman Chowdhury, director of software engineering at Twitter, [14]said on Wednesday.
“One of our conclusions is that not everything on Twitter is a good candidate for an algorithm, and in this case, how to crop an image is a decision best made by people.”
Twitter said it’s still working to improve its image-cropping techniques for photos displayed on its website and for multiple photos. ®
Get our [15]Tech Resources
[1] https://www.theregister.com/2020/10/05/twitter_ai_images/
[2] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2YKXe1tfdOseGuoj2V6hJegAAAAo&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YKXe1tfdOseGuoj2V6hJegAAAAo&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YKXe1tfdOseGuoj2V6hJegAAAAo&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[5] https://www.theregister.com/2020/09/21/twitter_image_cropping_ai/
[6] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YKXe1tfdOseGuoj2V6hJegAAAAo&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[7] https://www.theregister.com/2021/05/07/computer_vision_correlation_bias/
[8] https://www.theregister.com/2021/04/09/facebook_algorithm_discriminating/
[9] https://www.theregister.com/2021/02/04/ai_brain_drain/
[10] https://www.theregister.com/2020/12/15/ai_report_bias_workplace/
[11] https://arxiv.org/abs/2105.08667
[12] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YKXe1tfdOseGuoj2V6hJegAAAAo&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[13] https://twitter.com/dantley/status/1369682375668424707
[14] https://blog.twitter.com/engineering/en_us/topics/insights/2021/sharing-learnings-about-our-image-cropping-algorithm.html
[15] https://whitepapers.theregister.com/
The algorithm was used to automatically decide which portion of a photo shared on the social network was best for the screen viewing it. You can upload pictures in all sorts of sizes and aspect ratios; at some point the images have to be displayed, and Twitter used a trained system to decide which parts of the media to show. The option to view the whole image was available – the algorithm was used to create a preview of the media.
Specifically, the software relied on a [1]saliency algorithm designed to keep the most interesting and relevant parts of the image in the preview crop. For example, a snap taken of people at the beach should focus on their faces rather than the sky or sand.
[2]
[3]
[4]
Twitter users, however, spotted that the tool, when given a choice of things to pick, seemed to zero in on people with lighter skin and women’s chests. After the social network investigated the issue in October, it [5]told The Register it didn’t find any evidence of racial or gender biases, though admitted its engineers probably needed to conduct more tests.
Now, three of its techies in its Machine Learning Ethics, Transparency, and Accountability team and its Content Understanding Research team have discovered that the saliency algorithm is, in fact, skewed towards White people and women.
We're told the software is about eight per cent more skewed towards displaying women compared to men, and four per cent more for people with lighter skin compared to those with darker skin. When these groups were narrowed down by gender, there was a seven per cent difference in favor of White women compared to Black women, and two per cent for White men compared to black men.
[6]
The difference seems slight but with millions upon millions of pictures tweeted a day, a good number of people are going to encounter the bias.
The team conducted their tests by comparing the way the software handled images of Black females, Black males, White females, and White males, running the experiment 10,000 times. In the previous study, staff looked at how the code cropped White people versus Black people, White versus Indian, White versus Asian, and male versus female, running the tests just 200 times. The small sample size made it more difficult to detect any significant levels of bias.
[7]Researchers say objects can hide from computer vision by seeking out unusual company that trips correlation bias
[8]Facebook job ads algorithm still discriminates on gender, LinkedIn not so much
[9]AI brain drain to Google and pals threatens public sector's ability to moderate machine-learning bias
[10]There's nothing AI and automation can't solve – except bias and inequality in the workplace, says report
There seemed to be no clear evidence that the algorithm focused on parts of people’s bodies other than their faces, however.
“We found that no more than 3 out of 100 images per gender have the crop not on the head,” they explained in a [11]paper shared on arXiv this week. “The crops not on heads were due to high predicted salient scores on parts of the image such as a number on the jersey of sports players or a badge. These patterns were consistent across genders.”
It’s possible that the problematic crops of women’s chests were caused by logos or images on their clothes, which grabbed the attention of the code.
[12]
Since March, Twitter has gradually rolled out an [13]update for smartphone users so that its app doesn’t rely on its saliency algorithm, and instead gave people more control of how images are displayed. In short, Twitter now displays "standard aspect ratio photos in full on iOS and Android — meaning without the saliency algorithm crop," and "a true preview of the image in the tweet composer field, so tweet authors know how their tweets will look before they publish."
“We considered the trade-offs between the speed and consistency of automated cropping with the potential risks we saw in this research,” Rumman Chowdhury, director of software engineering at Twitter, [14]said on Wednesday.
“One of our conclusions is that not everything on Twitter is a good candidate for an algorithm, and in this case, how to crop an image is a decision best made by people.”
Twitter said it’s still working to improve its image-cropping techniques for photos displayed on its website and for multiple photos. ®
Get our [15]Tech Resources
[1] https://www.theregister.com/2020/10/05/twitter_ai_images/
[2] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2YKXe1tfdOseGuoj2V6hJegAAAAo&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YKXe1tfdOseGuoj2V6hJegAAAAo&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YKXe1tfdOseGuoj2V6hJegAAAAo&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[5] https://www.theregister.com/2020/09/21/twitter_image_cropping_ai/
[6] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YKXe1tfdOseGuoj2V6hJegAAAAo&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[7] https://www.theregister.com/2021/05/07/computer_vision_correlation_bias/
[8] https://www.theregister.com/2021/04/09/facebook_algorithm_discriminating/
[9] https://www.theregister.com/2021/02/04/ai_brain_drain/
[10] https://www.theregister.com/2020/12/15/ai_report_bias_workplace/
[11] https://arxiv.org/abs/2105.08667
[12] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YKXe1tfdOseGuoj2V6hJegAAAAo&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[13] https://twitter.com/dantley/status/1369682375668424707
[14] https://blog.twitter.com/engineering/en_us/topics/insights/2021/sharing-learnings-about-our-image-cropping-algorithm.html
[15] https://whitepapers.theregister.com/
Algorithms within algorithms
HildyJ
As Google pointed out at this year's I/O: For people of color, photography has not always seen us as we want to be seen, even in some of our own Google products. Google says it's making changes to auto white balance adjustments to bring out natural brown tones.
Because almost all images are auto white balanced, which is itself an algorithm, dark portions of images tend to have less definition. Then Twitter's algorithm kicks in and will see more detail in and tend to focus on light skinned people versus dark skinned people.
Subject vs Artificial Idiocy
a_yank_lurker
As an serious amateur photographer I sometimes take a photo of a scene with people in it. They may be facing the camera but they are not focus of the scene. Often landscape photographers will have a person in the photo to provide a sense of depth and scale. In some photos, people may provide a setting for the scene but are otherwise not critical.
So how does artificial idiocy know what the subject of the photo is? Without asking the photographer it is just guessing.
“One of our conclusions is that not everything on Twitter is a good candidate for an algorithm, and in this case, how to crop an image is a decision best made by people.”
Just imagine all the hours wasted as user trieds to figure out where the UI for controlling the cropping was, and all the frustration that caused as it wasn't there. Needlessly.