What's 2 + 2? Personal info, sniffs Twitter: Anti-doxxing AI goes off the rails, bans tweets with numbers in them
(2020/09/01)
- Reference: 1598927947
- News link: https://www.theregister.co.uk/2020/09/01/twitter_user_suspension/
- Source link:
Netizens are being locked out of their Twitter accounts for tweeting innocuous posts and images, such as math equations, that trigger the social network's system that prevents the sharing of private personal information.
Revealing personally identifiable non-public information, such as someone’s home address or cellphone number, also known as doxxing, is against the site’s rules. Doxxing is often used to harass people, as it invites strangers to stalk them, for instance.
“You may not publish or post other people's private information without their express authorization and permission,” [1]states Twitter’s policy. “We also prohibit threatening to expose private information or incentivizing others to do so.”
Some users, however, have been wrongly punished for breaking the rules after they shared what appear to be ordinary tweets and pictures that contained no personal info. A Reg reader alerted us to the problem after he was barred from using two of his accounts when he [2]tweeted an image of the front page of The Evening Standard, a British newspaper, and, again, when he posted an screenshot of a Wikipedia article.
It didn’t give any indication of what the private information was determined to be
Louis Maddox, a programmer based in London, showed The Register both of the offending images and neither of them contain any private personal information about anyone. Nevertheless, Twitter asked him to delete the tweets or appeal its request.
Maddox opted to appeal both times, and was subsequently temporarily barred from using Twitter. Although his account is visible online, he cannot tweet or view his timeline at the moment whilst Twitter processes both of his appeals. He reckons there might be a bug in the automated methods Twitter uses to analyze images, which we presume involves some kind of artificial intelligence due to the speed at which it works.
“It didn’t give any indication of what the private information was determined to be,” he told El Reg .
"I have no idea how this works on the backend other than that Twitter feeds all images it receives through neural networks to do things like automatically cropping an image. Since the tweet gets flagged immediately, there is no chance it was due to human intervention from a moderator or a bad actor trolling through the report feature, then it must be down to automated image recognition."
Another Twitter user, who goes by the handle AltentX, also discovered that accounts were being automatically, and incorrectly, flagged for doxxing when they themselves and others posted images on the social media platform.
Dear [3]@TwitterSupport , we think we just found an error about twitter Private Info Image Detection, where any account who tweet this kind of image will suddenly locked entire account on the device. May I share about this privately on DM? Just aware this could lead to lock trolling — AltentX 🚀 (@altentx) [4]August 28, 2020
“The images I uploaded were in Indonesian, and the first image just reads, 'oops sorry, the website is under construction, please try again later.' Another image is an algebra problem,” AltentX told us. “I don't think Twitter’s AI models detect the meaning of words.”
What’s common with all the offending images is that they involve a white box against a dark background. “The key is a centered white box, and a dark color around the box. Then your account will be locked,” AltentX added.
As collated by Maddox, [5]numerous tweeters have been thrown in the platform's jail for supposedly sharing private information on Twitter, requiring them to delete their posts to continue using the site, even if their posts contained no personal info.
A spokesperson for Twitter was not available to comment. ®
Get our [6]Tech Resources
[1] https://help.twitter.com/en/rules-and-policies/personal-information
[2] https://regmedia.co.uk/2020/09/01/appeal.jpg
[3] https://twitter.com/TwitterSupport?ref_src=twsrc%5Etfw
[4] https://twitter.com/altentx/status/1299353703112081410?ref_src=twsrc%5Etfw
[5] https://twitter.com/i/events/1300468150367121409
[6] https://whitepapers.theregister.com/
Revealing personally identifiable non-public information, such as someone’s home address or cellphone number, also known as doxxing, is against the site’s rules. Doxxing is often used to harass people, as it invites strangers to stalk them, for instance.
“You may not publish or post other people's private information without their express authorization and permission,” [1]states Twitter’s policy. “We also prohibit threatening to expose private information or incentivizing others to do so.”
Some users, however, have been wrongly punished for breaking the rules after they shared what appear to be ordinary tweets and pictures that contained no personal info. A Reg reader alerted us to the problem after he was barred from using two of his accounts when he [2]tweeted an image of the front page of The Evening Standard, a British newspaper, and, again, when he posted an screenshot of a Wikipedia article.
It didn’t give any indication of what the private information was determined to be
Louis Maddox, a programmer based in London, showed The Register both of the offending images and neither of them contain any private personal information about anyone. Nevertheless, Twitter asked him to delete the tweets or appeal its request.
Maddox opted to appeal both times, and was subsequently temporarily barred from using Twitter. Although his account is visible online, he cannot tweet or view his timeline at the moment whilst Twitter processes both of his appeals. He reckons there might be a bug in the automated methods Twitter uses to analyze images, which we presume involves some kind of artificial intelligence due to the speed at which it works.
“It didn’t give any indication of what the private information was determined to be,” he told El Reg .
"I have no idea how this works on the backend other than that Twitter feeds all images it receives through neural networks to do things like automatically cropping an image. Since the tweet gets flagged immediately, there is no chance it was due to human intervention from a moderator or a bad actor trolling through the report feature, then it must be down to automated image recognition."
Another Twitter user, who goes by the handle AltentX, also discovered that accounts were being automatically, and incorrectly, flagged for doxxing when they themselves and others posted images on the social media platform.
Dear [3]@TwitterSupport , we think we just found an error about twitter Private Info Image Detection, where any account who tweet this kind of image will suddenly locked entire account on the device. May I share about this privately on DM? Just aware this could lead to lock trolling — AltentX 🚀 (@altentx) [4]August 28, 2020
“The images I uploaded were in Indonesian, and the first image just reads, 'oops sorry, the website is under construction, please try again later.' Another image is an algebra problem,” AltentX told us. “I don't think Twitter’s AI models detect the meaning of words.”
What’s common with all the offending images is that they involve a white box against a dark background. “The key is a centered white box, and a dark color around the box. Then your account will be locked,” AltentX added.
As collated by Maddox, [5]numerous tweeters have been thrown in the platform's jail for supposedly sharing private information on Twitter, requiring them to delete their posts to continue using the site, even if their posts contained no personal info.
A spokesperson for Twitter was not available to comment. ®
Get our [6]Tech Resources
[1] https://help.twitter.com/en/rules-and-policies/personal-information
[2] https://regmedia.co.uk/2020/09/01/appeal.jpg
[3] https://twitter.com/TwitterSupport?ref_src=twsrc%5Etfw
[4] https://twitter.com/altentx/status/1299353703112081410?ref_src=twsrc%5Etfw
[5] https://twitter.com/i/events/1300468150367121409
[6] https://whitepapers.theregister.com/
Re: Simplify all the things
stiine
No, but if you don't add an avatar to online accounts, most of them default to a silouette, a blank white head on a dark background. Twitter must have some wonderful DevOps people....
They will never automate this no matter how hard they try
DS999
It can do pattern matching for "this has already been found to be something we've banned" but the idea that an AI can be trained to determine what is or is not doxxing is ludicrous. If I tweet the address of a voting site, or a historic building that was destroyed in a storm, how can the AI tell I'm not doxxing whoever lives there? How is going to handle bad actors who resort to code words like "the drug store at 111 E Main St sells Tums" and have that mean that's where someone skinheads should beat up lives?
Re: They will never automate this no matter how hard they try
Anonymous Coward
It isn't. That doesn't matter though, that just shows we need evan moar of it¹, and possibly make posting pictures that do not get whiteboxed illegal. Don't you think of the chiiiiildren?
¹ it's the American Tourist approach to policy: if they don't understand, SPEAK LOUDER.
Simplify all the things
What’s common with all the offending images is that they involve a white box against a dark background. “The key is a centered white box, and a dark color around the box. Then your account will be locked,” AltentX added. Isn't a white box against a dark background the "spherical horse in a vacuum" of machine learning?