YouTube's recommendation engine is pretty naff, Mozilla study finds
- Reference: 1625743926
- News link: https://www.theregister.co.uk/2021/07/08/youtubes_mozilla_algorithm/
- Source link:
“This problem with YouTube’s recommendation algorithm is part of a bigger story about the opaque, mysterious influence that commercial algorithms can have on our lives,” the Mozilla-led [1]study , released on Wednesday, concluded.
“YouTube’s algorithm drives an estimated 700 million hours of watch time every single day, and yet the public knows very little about how it works. We have no official ways of studying it.”
[2]
Firefox-maker Mozilla built a browser extension named RegretsReporter for YouTube users to download. Once installed, the extension logged netizens’ viewing activity on YouTube, recording details on the videos watched, and made it easy for users to flag clips that they wished they hadn’t seen. The data was pooled together and analyzed in an attempt to study YouTube's recommendation engine's behavior and effectiveness.
[3]
[4]
There are a number of reasons why the volunteers regretted watching vids, ranging from the material being nothing more than bonkers QAnon conspiracy theories about political elites drinking children’s blood, to false information on COVID-19 vaccines, to inappropriate raunchy parodies of smash-hit movie Toy Story.
Mozilla researchers found that 71 per cent of all videos volunteers regretted watching on the Google-owned platform were recommended by the giant's whiz-bang AI algorithm. They also estimated that about 12.2 per cent of those reported videos contained content that violated YouTube’s own guidelines and policies – the vids shouldn't be on the site all at let alone recommended by it.
[5]
A total of 37,380 YouTube viewers across 190 countries volunteered in the crowd-sourced study; 3,362 reports were submitted for videos they regretted watching between July 2020 and May 2021. YouTube’s recommendation algorithm seems to perform worse in non-English speaking countries, where people logged higher rates of regret reports. Brazil, Germany, and France were rated worst, the US and the UK were ranked eighth and sixteenth respectively.
“The goal of our recommendation system is to connect viewers with content they love and on any given day, more than 200 million videos are recommended on the homepage alone,” a Google spokesperson told The Register in a statement.
“Over 80 billion pieces of information is used to help inform our systems, including survey responses from viewers on what they want to watch. We constantly work to improve the experience on YouTube and over the past year alone, we’ve launched over 30 different changes to reduce recommendations of harmful content. Thanks to this change, consumption of borderline content that comes from our recommendations is now significantly below 1 per cent."
[6]Florida Man sues Facebook, Twitter, YouTube for account ban
[7]Euro court rules YouTube not automatically liable for users illegally uploading copyright-protected material
[8]Campaigners warn of an 'algorithm-driven censorship' future if UK Online Safety Bill gets through Parliament
[9]Roger Waters tells Facebook CEO to Zuck off after 'huge' song rights request
YouTube has grappled with recommendation systems for years and has adjusted them to improve performance. Still, its automated software isn’t perfect: videos that violate its content policies still slip through and continue to be recommended to users. Mozilla reckons that will only continue if YouTube remains coy on how its algorithm actually works.
“We believe our research has revealed is only the tip of the iceberg, and that each of these findings deserves and requires further scrutiny,” the report stated.
[10]
“We also recognize that without intervention to enable greater scrutiny of YouTube’salgorithms, these problems will continue to go unchecked and the consequences on our communities will build. Despite the progress that YouTube claims to have made on these issues, it is still nearly impossible for researchers to verify these claims, nor study YouTube’s recommendation algorithms.”
Mozilla recommended YouTube publish data on how its systems work and be transparent about its content moderation and recommendation model. That way, Moz argued, researchers can independently audit the AI software. ®
Get our [11]Tech Resources
[1] https://foundation.mozilla.org/en/campaigns/regrets-reporter/findings/
[2] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2YOchIZdWisviI0jwP2qZ3QAAAA0&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YOchIZdWisviI0jwP2qZ3QAAAA0&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YOchIZdWisviI0jwP2qZ3QAAAA0&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[5] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44YOchIZdWisviI0jwP2qZ3QAAAA0&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[6] https://www.theregister.com/2021/07/07/florida_man_sues_social_media/
[7] https://www.theregister.com/2021/06/23/youtube_copyright_dispute/
[8] https://www.theregister.com/2021/06/23/online_safety_bill_legal_type_legal_say/
[9] https://www.theregister.com/2021/06/17/roger_waters_says_zuck_off/
[10] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33YOchIZdWisviI0jwP2qZ3QAAAA0&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[11] https://whitepapers.theregister.com/
Re: You don't say
Indeed, YouTube's recommendations are as much of running joke as Amazon's. It should be absolutely trivial to avoid recommending the exact thing I just watched/bought, yet they somehow two of the biggest companies in the world can't figure it out.
I suspect that the problem is there's simply no reason for them to bother. YouTube doesn't care if it's recommendations are relevant. As long as you're watching something , you're making them money. Regret watching something? Doesn't matter. Ignore recommendations entirely? Doesn't matter. Check lots of videos for a short time without watching the whole thing? Doesn't matter. As long as you don't leave the site to find videos somewhere else, YouTube is perfectly happy. Without any penalty from providing bad recommendations, there's simply no incentive for them to make it work.
Re: You don't say
This comment officer. Right here.
It's clear, well thought out, articulated and 100% on point.
Obviously we can't be having that, here, on the internet...
:D
In all fairness, you are 100% spot on. Youtube really does have the monopoly on non-studio produced streaming video. There are simply no alternatives that have amarket or audience share anywhere near threatening to them.
Re: You don't say
The big problem is that it has all the subtlety and sensitivity of a brick in a sock... watch one video on how to take an engine to bits, and that's all you see. Watch one on paragliding and that's all you see. Watch one on vintage computers and that's all you see... surely it is not beyond the wit of man to devise an algorithm that assumes that you might be interested in more than one thing (and incidentally, not keep showing you the same video you haven't been watching, as indicated above)?
Or - just speculating here, you understand - is it simply dumping the videos with the highest advert count? I don't see the adverts so I can't really comment. But surely youtube wouldn't be that obvious?
Re: You don't say
Dare to fall asleep while watching youtube and you will be certain to wake up to a watch history full of utter crap.
I have found, however, that immediately removing crap content from the view history seems to have a positive effect on the suggestions and every now and then the suggestions are mind blowing good, at least on technical topics.
Don't even want to think about how effective platforms such as youtube would be, should our great overlords decide to engage in the business of targeted manipulation disguised under the realms of AI algorithms, so let's just quickly ignore that possibility.
YouTube's recomendations are crap
If I watch a short film, it thinks I must want to watch it repeatedly. If I click that I'm "not interested" one of the options is that "I've already watched it", well YouTube knows that already as it shows the red progress bar all the way along the film, so why does it keep shoving the same videos at me all the time. Maybe I'd like to watch something different, for a change you know or are there only a dozen short films on YouTube?
It's the ads that irritate me most
My 3 year old daughter likes to watch the same few videos over and over. So she does.
And yes, she knows how to work the player (and press the skip ad button)
The inserted ads are almost always highly inappropriate for the content, sometimes even banned-before-watershed. In a video aimed at toddlers.
Re: It's the ads that irritate me most
Google "adblock plus" - I routinely install the chrome extension, mainly to stop YouTube showing unwanted ads
Re: It's the ads that irritate me most
Absolutely. Also for an alternative mobile app that doesn't show ads there's Skytube Extra, although because it's not on Youtube's systems, it gives only generic recommendations. Best to hide that screen entirely.
A Journey through the digital sewer
Youtube has content policies? You're kidding, right?
I'd block the site if I didn't want watch the occasional how-to video and didn't, for some obscure reason, find the lock-picking lawyer (https://www.youtube.com/c/lockpickinglawyer) entertaining.
Re: A Journey through the digital sewer
Upvoted fol LPL, if you like him check one of his collaborators: Bosnian Bill.
Stuff I've already seen
YouTube recommends videos I've already seen, and channels I'm already subscribed to. WTAF.
It's frustrating to know there's a ton of interesting stuff out there, and I can't find any of it.
It's like the internet before search engines, where you had to find stuff by hearing about it from other people.
They probably don't know. It's just another ML black box, with no reason or rhyme : just pattern matching. And nobody knows what the patterns are.
It's about time we dropped this crap in favour of actual AI
Merge with Facebook...
I think the problem here is that Youtube is recommending based on what you have seen before. There'd be a lot less complaints if it recommended based on who your friends are. So if Facebook ran the recommendations you'd only get stuff from your own silo! Problem solved!
Of course, some people might have to have more than one account, you know, for the stuff you do in the woods at night...
(AC, so no Joke icon!)
YouTube's recommendation engine is pretty naff, and in other news
The head of The Universal Church is a member of said church; and ursidae defecate in aboral zones.
Relevant
"inappropriate raunchy parodies of smash-hit movie Toy Story"
You have my attention.
Same with ads. This morning an ad for tennis shoes came up. I skipped it at the earliest opportunity. It came up again, I skipped. Repeat, repeat, repeat.
Take the hint ML engine. Take the hint.
In fairness ...
The Mozilla test group is a pretty self-selecting bunch: people with the discernment to be running Firefox and the willingness to install this add-on and participate in the test. I would guess this population is not representative of YouTube consumers as a whole. Which is not to say they don't have a valid point, but I suspect that most people will happily stare slack-jawed at the screen until Wall-E comes around with a soda.
You don't say
There's a lot of great stuff on Youtube but then there's the other 99.9% of stuff on Youtube.
I'd love Youtube to stop recommending videos I've ignored for weeks on end. Mix it up, generate a new recommended list every time the page is refreshed. There must be a few videos to choose from.
When I give up on a video after 30 seconds, that's not an indication that I want to be recommended everything else from the same channel.
I'd love Youtube to stop repeatedly recommending videos I'VE ALREADY WATCHED.
Youtube are you listening? (apparently not)