Study: How Amazon uses Echo smart speaker conversations to target ads
- Reference: 1651042333
- News link: https://www.theregister.co.uk/2022/04/27/amazon_audio_data/
- Source link:
Academics at the University of Washington, University of California-Davis, University of California-Irvine, and Northeastern University claim "Amazon processes voice data to infer user interests and uses it to serve targeted ads on-platform (Echo devices) as well as off-platform (web)."
The researchers – Umar Iqbal, Pouneh Nikkhah Bahrami, Rahmadi Trimananda, Hao Cui, Alexander Gamero-Garrido, Daniel Dubois, David Choffnes, Athina Markopoulou, Franziska Roesner, Zubair Shafiq – describe their findings in [1]a paper titled, "Your Echos are Heard: Tracking, profiling, and ad-targeting in the Amazon smart speaker ecosystem."
[2]
The ten academics say that smart speaker interaction – that's you talking to the device – generates ad auction bids from advertisers that are as much as 30x higher than the bids would be without Amazon Echo speaker data. What's more, they say, the way Amazon and Skills developers – makers of software integrated with Amazon's Alexa voice assistant service – operate is often inconsistent with privacy policies.
[3]
[4]
That may not be a surprise to you; either way, the paper provides a technical dive, and a thorough analysis, of how Amazon, Echo devices, Alexa, and adverts are all tied together.
To understand how Amazon and Skills developers handle audio data, the boffins created an auditing framework to evaluate how voice data gets collected, used, and shared. They did so because Amazon Echo smart speakers do not provide an interface to assess how data gets used and there's no ready-made mechanism for understanding what happens to smart speaker data sent over the internet.
[5]
So the researchers created multiple fake personas with different smart-speaker usage profiles, and then simulated interactions to test statistical differences in amounts bid for audio and web advertisements. In this way, they claim they were able to infer the effect of smart speaker interactions involving the constructed personas.
[6]Amazon opens MASSIVE AI speech dataset so Alexa can speak your language
[7]Amazon Alexa can be hijacked via commands from own speaker
[8]Shine on: Boffins bedazzle Alexa and her voice-controlled assistant kin with silent laser-injected commands
[9]Amazon and Google: Trust us, our smart-speaker apps are carefully policed. Boffins: Yes, well, about that...
Technically, the auditing framework involved setting up a custom Raspberry Pi router to record the network endpoints contacted by Amazon Echo and emulating an Amazon Echo by setting up Alexa Voice Service SDK, in order to capture unencrypted network traffic.
This hardware and software setup was used to issue voice commands to an Amazon Echo in order to watch how the data was used for audio ads (via Amazon Music, Spotify, and Pandora), on the web for display ads (personas using a browser logged into Amazon account and Alexa web companion app), and on non-Echo devices.
The fake personas were configured to install and interact with Skills associated with their respective interests. These included: Connected Car, Dating, Fashion & Style, Pets & Animals, Religion & Spirituality, Smart Home, Wine & Beverages, Health & Fitness, and Navigation & Trip Planners.
The researchers say that Echo interaction data is collected both by Amazon and third-parties and that Amazon shares user data with as many as 41 ad partners. They say that ad targeting enabled by the data leads to ad bids as much as 30x higher, and that Amazon's inference of ad interests from voice data is a clear violation of the company's privacy policy and public statements.
[10]
In addition, over 70 percent of Skills fail to mention Alexa or Amazon, and a mere 2.2 percent make their data collection practices clear in their privacy policies, the researchers contend.
Transparency
"Amazon’s inference of advertising interests from users’ voice is a clear violation of their policies and public statements," the researchers state in their paper. "Amazon does not provide transparency in usage of data and thus cannot be reliably trusted to protect user privacy."
The paper also observes that Amazon was granted [11]a patent in 2018 titled "Voice-based determination of physical and emotional characteristics of users" that describes how the "current physical and/or emotional condition of the user may facilitate the ability to provide highly targeted audio content, such as audio advertisements promotions, to the user."
Asked to comment, an Amazon spokesperson challenged some of the papers conclusions without citing specific inaccuracies. "Many of the conclusions in this research are based on inaccurate inferences or speculation by the authors, and do not accurately reflect how Alexa works," the spokesperson told The Register in an emailed statement.
Umar Iqbal, postdoctoral researcher at the University of Washington and lead author of the paper, responded, "This is a broad statement without any concrete details."
Asked to provide specifics, Amazon said with regard to the 30x bid price change, "The price of ad auctions are influenced by several factors that are not linked to voice requests or Alexa interactions." Amazon also defended its handling of privacy policy compliance. "We require Skill developers that collect personal information to provide a privacy policy, which we display on the Skill's detail page, and to collect and use that information in compliance with their privacy policy and applicable law," the company said.
Amazon also insisted it does not sell data, which the paper does not allege. "We are not in the business of selling data and we do not share Alexa requests with advertising networks," the US giant said.
If you ask Alexa to order paper towels or to play a song on Amazon Music, the record of that purchase or song play may inform relevant ads shown on Amazon or other sites where Amazon places ads
Iqbal countered, with references to the study's paper: "We find evidence of Alexa Skills directly communicating with advertising/tracking services (section 4.2). We also note that Amazon's advertising partners sync their cookies with Amazon and bid higher than non-partner advertisers (section 5.5). A logical explanation for this behavior is that Amazon/Skills share/sell user interest data with their advertising partners. We do not claim that Amazon directly shares voice input/transcripts with advertising networks."
That is to say, records of Alexa conversations are not handed over verbatim, though the nature of what is discussed is seemingly used to guide the kinds of adverts you'll be targeted with.
Amazon's statement continues, "Similar to what you'd experience if you made a purchase on Amazon.com or requested a song through Amazon Music, if you ask Alexa to order paper towels or to play a song on Amazon Music, the record of that purchase or song play may inform relevant ads shown on Amazon or other sites where Amazon places ads. Customers can opt out of interest-based ads from Amazon at anytime on our website."
Iqbal noted: "This statement actually supports our findings." ®
Get our [12]Tech Resources
[1] https://arxiv.org/abs/2204.10920
[2] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_security/research&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2YmkUTl@L9ZRfD4p3BrOv8gAAAQQ&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_security/research&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YmkUTl@L9ZRfD4p3BrOv8gAAAQQ&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_security/research&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YmkUTl@L9ZRfD4p3BrOv8gAAAQQ&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_security/research&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YmkUTl@L9ZRfD4p3BrOv8gAAAQQ&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[6] https://www.theregister.com/2022/04/20/amazon_ai_speech/
[7] https://www.theregister.com/2022/03/03/amazon_alexa_speaker_vuln/
[8] https://www.theregister.com/2020/08/14/shine_on_boffins_bedazzle_alexa/
[9] https://www.theregister.com/2020/07/29/amazon_google_voice_apps/
[10] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_security/research&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YmkUTl@L9ZRfD4p3BrOv8gAAAQQ&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[11] https://patents.google.com/patent/US10096319B1/en
[12] https://whitepapers.theregister.com/
Re: Bears in a forest
White hats apparently still worn in the Vatican, too.
How far will they be willing to go?
If they're that determined to data mine, and the law starts targeting them, will they go so far as to use their clout to lobby or campaign for a more-permissive government?
Re: How far will they be willing to go?
They'll just trget ads at legislators.
Are people seriously complaining that when you purchase something from a vendor, the vendor takes note and tries to sell you related items, or the same thing later on?
Or that if you talk to a vendor about some product, then you're surprised when they notice and try to see you the things you were discussing?
Seems a ridiculous complaint to me.
It's mining your interactions with it irrespective of whether you make a purchase or not. Any interaction can be used and that could also be accidental - or why not even expand that to be passive listening and mining since that's how the devices have to operate in order to be ready to respond?
It's not ridiculous to complain that you are unaware of essentially dubious use of your interactions
How about if you go into a shop and remark "shocking weather today isn't it". Then are confronted by a stream of sales staff touting umbrellas, rain coats, waterproofing sprays and holidays in the sun. When all you went in to the shop for was a pint of milk?
"Genossen, wir müssen alles wissen!"
© Erich Mielke, DDR, 1989.
Amazon are gonna Amazon
If you're silly enough to pay them for a bug in your home, don't be surprised when they use it against you.
Divide and rule
> watch how the data was used for audio ads (via Amazon Music, Spotify, and Pandora), on the web for display ads (personas using a browser logged into Amazon account
Just another reason for having multiple email addresses and browser profiles. Keep everything compartmentalised and isolate devices from "real life". That way none of the devices or apps need to know, nor can discover, what is revealed either intentionally or accidentally to other aspects of your life.
And it will still deliver ads for the wrong thing:
1. Pick out some key word and throw up anything that has the same word in the description (example, try searching for a DPDT toggle switch and count the hits that are SPDT,DPST, centre-off, i.e. triple throw or not toggle swtiches).
2. People who bought this also bought...something completely irrelevant; it was a small sample but we don't care.
Personally I find I'm still fascinated by the way that Amazon/Google etc. algorithms seem to segue from what you were interested in to what it assumes you must be interested in, even though that is a totally different thing that happens to have some key words in common. And of course, in the case of Google's search, may well be ( usually is) just the opposite. Searches for "How do I get rid of X" will throw up endless suggestions about how to obtain the bloody thing, but nothing about how to get shut of it.
Bears in a forest
They leave quite a mess, I'm told.