Let's play a game: Deepfake news anchor or a real person?
- Reference: 1676126594
- News link: https://www.theregister.co.uk/2023/02/11/deepfake_news_anchors/
- Source link:
These videos represent "the first time Graphika has observed state-aligned IO [influence operation] actors using video footage of AI-generated fictitious people in their operations," the social media analytics firm's researchers reported, detailing the Spamouflage deepfakes in a report
[1]PDF
this week.The videos show a male and female broadcaster, both purporting to be reporting for a media company called Wolf News. In one, the news anchor condemns America for its failures to stop gun violence. The other stresses the need for China-US cooperation on global economic recovery efforts.
[2]
"At first glance, the Wolf News anchors present as real people," Graphika said. "Our initial hypothesis was that they were paid actors that had been recruited to appear in the videos."
[3]
[4]
This would align with Spamouflage's usual modus operandi, according to Graphika and [5]other researchers including Google who track the miscreants as Dragonbridge and have documented their use of real people on camera in their earlier disinformation campaigns.
Some of the crew's other influence operations have included attempts to [6]meddle in the 2022 American midterm elections and [7]trolling rare-earth mining companies by using thousands of phony social media accounts, prompting a stern finger-wagging by the Pentagon.
[8]
"But further investigation revealed the Wolf News presenters were almost certainly created using technology provided by a British AI video company called Synthesia," Graphika researchers claimed.
Reverse image searches unearthed a ton of unrelated videos showing the same man and woman "news anchors" who appear in the Spamouflage Wolf News broadcast. The AI-generated people speak in multiple languages in these vids, including Arabic, Romanian, Spanish and English, like [9]this one promoting freight broker services.
In it, the male anchor says, "Hello, my name is Mr Cruise. And I'm an avatar."
[10]
Cruise is one of [11]Synthesia's "100+" avatars , and it turns out his name is Jason. The female anchor from the Wolf News videos is Anna.
Synthesia's [12]ethics page says it "will not offer [its] software for public use. All content will go through an explicit internal screening process before being released to our trusted clients."
Additionally, its [13]avatar FAQ page says there's limits as to what the avatars can and cannot say: "Political, sexual, personal, criminal and discriminatory content is not tolerated or approved."
Spamouflage's fake news stories would violate both policies. Synthesia could not be reached for comment. Victor Riparbelli, Synthesia's co-founder and chief executive, [14]told the New York Times earlier this week Spamouflage broke Synthesia's terms of service by using the tech to make its videos.
That is to say, Spamouflage somehow got hold of and used Synthesia's technology to produce its content, contrary to Synthesia's rules. There is no suggestion of wrongdoing by Synthesia.
[15]Will 2023 be the year of dynamite disinfo deepfakes, cooked up by rogue states?
[16]China bans deepfakes created without permission or for evil
[17]Intel says it can sort the living human beings from the deepfakes in real time
[18]Here's how crooks will use deepfakes to scam your biz
While [19]government agencies and [20]security researchers alike have sounded the alarm about deepfakes being used in political influence operations in the not-too-distant future, until now AI-generated media has been largely limited to fake faces — not entirely fake people, according to Graphika.
However, despite the use of avatars, the rest of the Spamouflage videos look like "low quality political spam," the researchers wrote, echoing [21]Google's assessment of the crew's YouTube videos, 83 percent of which had fewer than 100 views.
Additionally, the deepfake element of the newscasts "was almost certainly created using a commercial service" as opposed to in-house capability, the report concluded. This suggests Spamouflage and others don't have the technical expertise to pull off highly sophisticated deepfakes and will continue to use available commercial tools.
However, as Graphika notes, "this also raises questions about how to effectively moderate the use of these products and services." ®
Get our [22]Tech Resources
[1] https://public-assets.graphika.com/reports/graphika-report-deepfake-it-till-you-make-it.pdf
[2] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2Y@fJsYPwCDf-hZKYLODLXAAAAIw&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Y@fJsYPwCDf-hZKYLODLXAAAAIw&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33Y@fJsYPwCDf-hZKYLODLXAAAAIw&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[5] https://www.theregister.com/2023/01/27/google_tag_dragonbridge_takedown/
[6] https://www.theregister.com/2022/10/27/chinese_misinformation_dragonbridge_elections/
[7] https://www.theregister.com/2022/06/28/dod_china_dragonbridge/
[8] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Y@fJsYPwCDf-hZKYLODLXAAAAIw&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[9] https://www.youtube.com/watch?v=0cm7Vl2kvfA
[10] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33Y@fJsYPwCDf-hZKYLODLXAAAAIw&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[11] https://www.synthesia.io/features/avatars
[12] https://www.synthesia.io/ethics
[13] https://www.synthesia.io/features/avatars#faq
[14] https://www.nytimes.com/2023/02/07/technology/artificial-intelligence-training-deepfake.html
[15] https://www.theregister.com/2023/01/18/enemy_deep_fakes_ai/
[16] https://www.theregister.com/2022/12/12/china_deep_synthesis_deepfake_regulation/
[17] https://www.theregister.com/2022/11/15/intel_fakecatcher/
[18] https://www.theregister.com/2022/09/28/trend_deepfake_video/
[19] https://www.aha.org/fbi-tlp-alert/2021-03-10-fbi-tlp-white-pin-malicious-actors-almost-certainly-will-leverage
[20] https://www.theregister.com/2022/09/28/trend_deepfake_video/
[21] https://blog.google/threat-analysis-group/over-50000-instances-of-dragonbridge-activity-disrupted-in-2022/
[22] https://whitepapers.theregister.com/
Content or Presentation?
The generic TV news program has one or two people sitting at a desk in front of a green screen reading from a teleprompter. The image displayed to the viewer is almost entirely fake so I could see the temptation to try to replace the humans (and their salaries) by software, making the entire studio redundant.
The content suggests that its a Chinese company trying to develop this -- they'll take a couple of generic news items (Chinese generic news items) and use them to generate convincing looking news programs. Obviously from a modern Western perspective its Nefarious Propaganda coming from the State Machine but this is missing the point -- propaganda is just perspective and there's essentially no difference between 'their' propaganda and 'our' propaganda, its just selling impressions and ideas. (So it pays to be skeptical.)
I actually prefer to read my material because TV news is essentially 'content free' -- a story is told in a few seconds using easily manipulated imagery and selective quotes to tell a predetermined story. The only problem with it is that far too much material comes from anonymous spokespersons - PR plants, in other words - and a surprising amount is downright inaccurate. (Science education has been lagging for some years so we appear to be feeling the consequences!)
(As for 'meddling', this is one of the biggest canards sold to the US public (at least). US politics is dominated by Big Money -- a local race (Los Angeles Mayor) had the losing candidate spending something north of $100 million on their campaign, for example. Much of the money in politics is funneled from 'dark' sources -- attempts to pass laws forcing disclosure have been repeatedly stymied by legislators and courts -- and the sums involved are astounding. It would be difficult for a nation state to be heard without investing vast sums of money. Meanwhile our own State Department directly and indirectly spends a lot of taxpayer funds on 'promoting democracy' -- i.e. meddling -- in elections all over the world to ensure the correct result -- and Heaven Help anyone who chooses the wrong candidate!)
Racism
Isn't this racist that Chinese use AI equivalent of "whiteface"?
如果你想撒谎，那就露出你的真面目和声音。
Time to binge-watch...
... The Capture (BBC) again.
I’ll get the pop-corn and the covfefe packet. It’s in my coat pocket.