Giving Windows total recall of everything a user does is a privacy minefield
- Reference: 1716383475
- News link: https://www.theregister.co.uk/2024/05/22/windows_recall/
- Source link:
Like so many of Microsoft's AI-infused products, Windows Recall will remain in preview while Microsoft refines it based on user feedback – or simply gives up and pretends it never happened.
The principle is simple. Windows takes a snapshot of a user's active screen every few seconds and dumps it to disk. The user can then scroll through the snapshots and, when something is selected, the user is given options to interact with the content.
[1]
Microsoft said: "Recall will also enable you to open the snapshot in the original application in which it was created, and, as Recall is refined over time, it will open the actual source document, website, or email in a screenshot. This functionality will be improved during Recall's preview phase."
[2]
[3]
Improvements will certainly be needed, particularly in how the function deals with privacy.
Taking aside the fact that BitLocker will only come into play on Windows 11 Pro or Enterprise devices – everyone else must make do with "data encryption" – Windows Recall has the potential to be a privacy nightmare.
[4]
According to Microsoft, all the processing takes place on a customer's device. The company also says that for the relatively small number of users running its Edge browser – with a market share of just under 13 percent, [5]according to Statcounter – InPrivate sessions won't be stored, nor will DRM content.
Microsoft [6]said : "Recall does not perform content moderation. It will not hide information such as passwords or financial account numbers. That data may be in snapshots stored on your device, especially when sites do not follow standard internet protocols like cloaking password entry."
But that's OK – a user can opt to filter out sites, right? Only if you're using Edge. In the [7]documentation for the service, Microsoft said: "To filter out a website from a snapshot, you must be using Microsoft Edge."
[8]
"Recall won't save any content from your private browsing activity when you're using Microsoft Edge or a Chromium-based browser."
Tarquin Wilton-Jones, a developer and privacy expert at Vivaldi, a Chromium-based browser vendor, expressed hope that the automatic respecting of the InPrivate mode – or Incognito mode for Chrome – would apply outside of Edge.
"It almost certainly will not respect any browser's attempts to clear browsing data, where the browser could historically have been in any screenshots," he said.
"It also cannot respect GDPR requests to delete personal data exposed in an application when the source data is deleted by a data controller, and for this reason, it is clearly a massive privacy risk for any organization that handles private data. Who knows what other private data, or sensitive information, it might store in a freely accessible format?"
Mozilla's Chief Product Officer, Steve Teixeira, told The Register : "Mozilla is concerned about Windows Recall. From a browser perspective, some data should be saved, and some shouldn't. Recall stores not just browser history, but also data that users type into the browser with only very coarse control over what gets stored. While the data is stored in encrypted format, this stored data represents a new vector of attack for cybercriminals and a new privacy worry for shared computers.
"Microsoft is also once again playing gatekeeper and picking which browsers get to win and lose on Windows – favoring, of course, Microsoft Edge. Microsoft's Edge allows users to block specific websites and private browsing activity from being seen by Recall. Other Chromium-based browsers can filter out private browsing activity but lose the ability to block sensitive websites (such as financial sites) from Recall.
[9]Microsoft Build 2024 looks like it's more about AI fluff than developer stuff
[10]Microsoft smartens up Edge for Business with screenshot blocking, logo branding, more
[11]Microsoft AI Studio opens for business, with a nod to safety
[12]Intel, AMD take a back seat as Qualcomm takes center stage in Microsoft's AI PC push
"Right now, there's no documentation on how a non-Chromium based, third-party browser, such as Firefox, can protect user privacy from Recall. Microsoft did not engage our cooperation on Recall, but we would have loved for that to be the case, which would have enabled us to partner on giving users true agency over their privacy, regardless of the browser they choose."
Jake Moore, Global Cybersecurity Advisor at ESET, noted that while the feature is not on by default, its use "opens up another avenue for criminals to attack."
Moore warned that "users should be mindful of allowing any content to be analysed by AI algorithms for a better experience."
Cybersecurity expert Kevin Beaumont was scathing in his [13]assessment of the technology, writing: "In essence, a keylogger is being baked into Windows as a feature."
AI expert Gary Marcus was [14]blunter : "F^ck that. I don't want my computer to spy on everything I ever do."
To add to Microsoft's woes, a spokesperson for the UK's Information Commissioner's Office said today: "We expect organisations to be transparent with users about how their data is being used and only process personal data to the extent that it is necessary to achieve a specific purpose. Industry must consider data protection from the outset and rigorously assess and mitigate risks to people's rights and freedoms before bringing products to market.
"We are making enquiries with Microsoft to understand the safeguards in place to protect user privacy."
At present, Windows Recall feels like it was put together with insufficient thought.
Microsoft has said that "Recall is a key part of what makes Copilot+ PCs special."
However, as Microsoft has pointed out, it remains in preview. Enterprises are unlikely to go anywhere near it until the privacy and security questions it raises have been answered. The GDPR aspect alone makes it a non-starter for all but the most determined of organizations.
Microsoft's customers and Windows enthusiasts alike have been clamoring for something in the operating system to make all the AI hype worthwhile. But, in its current form, Windows Recall is not it. ®
Get our [15]Tech Resources
[1] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=2&c=2Zk4WouXUke1LxC6MDK6Y0wAAANY&t=ct%3Dns%26unitnum%3D2%26raptor%3Dcondor%26pos%3Dtop%26test%3D0
[2] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Zk4WouXUke1LxC6MDK6Y0wAAANY&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[3] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33Zk4WouXUke1LxC6MDK6Y0wAAANY&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[4] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=4&c=44Zk4WouXUke1LxC6MDK6Y0wAAANY&t=ct%3Dns%26unitnum%3D4%26raptor%3Dfalcon%26pos%3Dmid%26test%3D0
[5] https://gs.statcounter.com/browser-market-share/desktop/worldwide
[6] https://www.microsoft.com/en-gb/windows/copilot-plus-pcs?r=1#faq2
[7] https://support.microsoft.com/en-gb/windows/recall-and-your-data-d404f672-7647-41e5-886c-a3c59680af15
[8] https://pubads.g.doubleclick.net/gampad/jump?co=1&iu=/6978/reg_software/aiml&sz=300x50%7C300x100%7C300x250%7C300x251%7C300x252%7C300x600%7C300x601&tile=3&c=33Zk4WouXUke1LxC6MDK6Y0wAAANY&t=ct%3Dns%26unitnum%3D3%26raptor%3Deagle%26pos%3Dmid%26test%3D0
[9] https://www.theregister.com/2024/05/22/microsoft_build_2024/
[10] https://www.theregister.com/2024/05/21/microsoft_edge_business/
[11] https://www.theregister.com/2024/05/21/microsoft_ai_studio_opens_for/
[12] https://www.theregister.com/2024/05/21/qualcomm_windows_microsoft/
[13] https://doublepulsar.com/how-the-new-microsoft-recall-feature-fundamentally-undermines-windows-security-aa072829f218
[14] https://x.com/GaryMarcus/status/1792950631197798549
[15] https://whitepapers.theregister.com/
It begs the obvious question...
Why?
What is the point?
Re: It begs the obvious question...
Exactly - I cannot think of one use case for this feature
Re: It begs the obvious question...
I can think of lots of use cases, none of which involve the user being spied on, but plenty of benefit for Microsoft.
Re: It begs the obvious question...
For clarity, and because the edit window has now passed, "none of which involve the user being spied on" would have been better written as, "none of which are to the benefit of the user, who is being spied on". All those use cases do indeed involve spying on the user.
Re: It begs the obvious question...
It also begs the question of how aggressively they'll have to monetise this to make any money from it. We've already seen Amazon, Apple and Google shrink from AI assistants because they were a money pit.
Re: It begs the obvious question...
Thumbs up for spelling "monetise" the English way, thumbs down for using the non-word monetise in the first place.
Re: It begs the obvious question...
monetise is a perfectly cromulent word: the OED has a citation for it (well, monetize, 'cause OED) from 1867.
Re: It begs the obvious question...
You're one of those people who thinks we're not supposed to pronounce the 'é in forté, aren't you?
Re: It begs the obvious question...
I don't agree with the poster you are replying to, but I also don't agree with you, as the correct English pronunciation of forté is fɔːteɪ (forte, the Italian variant, used often in musical notation is pronounced fɔːti ). If you speak US English, you might pronounce it fɔːt , but you'll also likley pronounce the name of a couple of well known Middle-Eastern countries as Eye Rack and Eye Ran, so forgive me if I don't take lessons on the pronunciation of my native language (or of loan-words from neighbouring countries) from you.
edit - I wrote all the above before re-reading your post and realising I'd misread it quite egregiously. Rather than ninja-edit or delete my own post, I will now correct myself (as I wish many other posters would)
You are, of course, totally correct, and the poster to whom you were replying remains wrong, possibly doubly so...
I hang my head in shame at my lack of reading comprehension, and present myself for punishment to the Headmaster.
Re: It begs the obvious question...
My American friends and relations pronounce “niche” as “nitch”, which - whilst normal by their local standards - always makes me twitch. Or should that be “twiche”?
Re: It begs the obvious question...
Language be a changin.
Re: It begs the obvious question...
You get your points for knowing why it's pronounced that way.
Re: It begs the obvious question...
What is the point?
Helping train MS's AI, at our expense and risk.
AI expert Gary Marcus was blunter: "F^ck that. I don't want my computer to spy on everything I ever do."
I'm no AI expert, but wot he said. Plus I'm currently running 3 displays at 3440x1440 and wondering how this 'feature' will cope with this. It's bad enough that the snipping tool insists on saving a screenshot every time, even though there's a 'save to disk' option if we really wanted to save stuff. Maybe MS is also getting commission from SSD vendors to burn through write cycles as well.
Re: It begs the obvious question...
Obviously this "feature" will fill up your HD like there is no tomorrow. Just multiply a JPG of your screen's resolution by time ("every few seconds" they said) and you will know in how many hours your disk will be full. "Buy a new, better one! Here are some exciting opportunities from our partners!"
"At present, Windows Recall feels like it was put together with insufficient thought"
Like almost everything M$ has ever done, from hiding file extensions by default to forcing massively time-consuming updates on users at unpredictable times. The concept of a computer being an unobtrusive tool has apparently never ranked high with them.
Re: "At present, Windows Recall feels like it was put together with insufficient thought"
You can imagine some techy getting excited with his idea that "We can already take screenshots when the user asks. Why not do it automatically and just save the results?" Or maybe just doing it and showing the higher ups. The question is why there weren't any grown-ups in the room to realise it was a Really Bad Idea?
Re: "At present, Windows Recall feels like it was put together with insufficient thought"
Like almost everything M$ has ever done..
.. has demonstrated they, and a lot of other big tech companies are in serious need of a thorough ethical cleansing. A 'feature' like this should never have got this far, if there wasn't a total disregard for user privacy.
Re: "At present, Windows Recall feels like it was put together with insufficient thought"
Smells like Microsoft's internal feature market. It's like a Battle Royal of useless shit, the team that can generate the most shit gets to stay employed, doubly so if it allows data mining of customers.
Re: "At present, Windows Recall feels like it was put together with insufficient thought"
Microsoft is going completely berserk over this AI thing, stuffing it in every place the sun shines and doesn't shine. I hear copy-paste is being AI-ed and Paint too is getting an AI makeover.
Re: "At present, Windows Recall feels like it was put together with insufficient thought"
It's the new "metaverse". Once they realise that it is, in practice, pretty useless, except for a few niche use cases, they'll bin it. They just need to get through the sunk cost fallacy stage first.
TOTAL RECALL
– we remember it for you wholesale.
Re: TOTAL RECALL
Wait, this is Microsoft, not Elon, so no "get your ass to Mars".
"a snapshot of a user's active screen every few seconds"
I'm starting to regret the days where 640KB was enough for everyone.
According to the Beeb the ICO is already taking an interest.
And according to the Reg, too. See the fine article.
Features like that
are a damn good reason NOT to buy a Windows computer. They claim it will be optional... yeah, right. So much crap, bloat and downright dangerous stuff you've got to turn off in Windows before you can safely use it.
Re: Features like that
Is there enough you can "turn off" in Windows? I don't think you can control what is being gathered and sent when you want safe usage. This is the OS that will fail when the internet is unavailable. You may guess why...
I would trust my brother "Kuato" who lives inside my stomach, before trusting M$.
Start the reactor, Bearshark.
As this seems to be a screen grab of everything and only in private browser sessions seem to be excluded, what about the other million apps you can run on your computer?
Definitely no corporate secrets, passwords, plans, patent records etc etc that would be exposed.
I would bet this "feature" is going to be attacked by nation States ASAP.
Also also, how long before they start 'offering' to backup Recall in the cloud, or governments try to require you to have scanning for csam or anything else that they think can use to push mass surveillance onto thier citizens with?
Obvious parallel is obvious
The phrase "Big Brother is Watching You" entered the English vocabulary with the publication of George Orwell's 1984, with the sinister connotations it now holds, but imagine if the book had never been published, or didn't have "Big Brother" as an entity, and how innocent the phrase would sound. Indeed, in the book, the phrase is supposedly reassuring, and a signifier of the constant propaganda that inhabitants of Airstrip One are fed.
In the future, then, will the phrase, "Windows Recall is Helping You" enter the vocabulary in the same way? Something that is presented as a useful feature for the user, such as Big Brother's protective gaze, has a true nature that is much more sinister? "All data is held on your device" for now, but we all know the heavyweight "AI" processing will take place in a data centre somewhere, and I struggle to believe that those snapshots won't be sent there, even transiently, for analysis, and processing.
The underlying benefit for Microsoft of doing this? Data analysis. Profiling. Targeted advertising. Does anyone honestly think that three-letter agencies wouldn't want their go on this fairground ride as well? We'd be naïve to think that they aren't already salivating at the thought. As for the potential to sell that data on, especially in jurisdictions without adequate provisions for privacy (*cough* the US *cough*), well, how long before that valuable profiling is used for political targeting, not just plain old advertising? It'll be Cambridge Analytica all over again, but with richer, higher quality data.
So, it's a hard nope from me.
They missed an opportunity
They should have named this feature "Panopticon."
Re: They missed an opportunity
They'd only be kicking themselves that "Palantir" was already taken...
How much data stored?
With a fairly modest 1600x 1200 screen, I presume the screen capture will need to be fairly good quality to be useful, so assume 100kB per screen. Every 3 seconds, that's about 1GB for only 8 hours of use. That'll take quite a while to fill a decent hard disk, but it still feels like a total waste of storage - this recall feature I mean not Windows generally... or do I?
Ok so probably plenty more compression that could be done to squeeze it down a fair bit more.
Re: How much data stored?
Just did a test on my laptop. Screen is 1920 x 1080. JPG compression is set to 75%. Resulting file size is 250KB (using Paint.net).
My home PC has two screens, both which are much larger in pixel weight. I'm guessing that that GB limit would be reached way before noon.
Don't want it. Period.
Re: How much data stored?
Maybe it is possible to use difference between frames for compression, similar to video for each screenshot. That would reduce storage requirements considerably.
Either way, it is not innovative, just creepy.
Re: How much data stored?
How many processor cores is that going to keep busy? Are Microsoft going to pay for the wear and tear on my storage (even modern flash memory has a limited number of rewrites before it fails)? What about the increased electricity usage? While they're at it, they can chip in for the air conditioning in summer.
Re: How much data stored?
Well, obviously, the local store is only a staging process to The Cloud!
That will have room for years of tracking history.
Already Red US states will be licking their lips
for women looking for abortion clinics.
Surveillance
AI in combination with on-computer surveillance for "unlawful content" will lead to a surveillance-state never seen before in the history of mankind. Even the DDR didn't have anything near this level of control and oversight. It's downright scary that politicians are even contemplating this.
Microsoft could well its first victim since I'm convinced users will flee Windows for Linux once this becomes a reality. They'll have sunk a $2 trillion company with their antics.
If Apple too is sunk when China invades Taiwan the U.S. will be a whoooolotta poorer.
Re: Surveillance
MS have been boiling the frog for a while now and they know how to do it. Corporations will get a group policy to turn it off, everyone else will get slurped, much like today.
another useful feature
Whether it is marketed as security observability oversight recovery etc it is like many of the other overreaching features (UEFI / systemd / secure boot / IME / etc) rife with potential for exploitation
Total Recall
This is one of the best examples I ran into lately of a solution desperately looking for a problem it could solve.
Looking from my professional environment, this is something absolutely NOT wanted.
Totally Illegal
Maybe this time they will hit the level where the US government, at least the federal one, declares the usage of Windows 11 as illegal on federal computers.
Which might result in a special build of Windows 11 which is, by default, free of all that cloud shit (like the server versions usually are). Everything available to activate if needed of course, but not constantly pushed into my face. I would even switch my OS language for that.
Remote work
Great tech for companies that want to spy on their employees...
Not that new...
Having done forensics on Windows, there is a shocking amount of user history data stashed on a system. Taking screen shots & making them available in a user interface is new.
Recording copious amounts of user activity is not new to Windows 11. It just hasn't been obvious to the masses before.
Satnad: "It's all done locally"
And then uploaded as telemetry...
Oh hell no! Even if I'm able to disable this shit, that's not good enough - I want to be able to eradicate every last shred of it from my computer. Oh, and Microsoft, yeah - that's right - MY COMPUTER! Just because it's running your increasingly unacceptable bag o'shite OS, doesn't make it yours.
Seriously thinking it's time to switch to Linux, especially now that it's a viable gaming platform.
You have to be pulling my purple plonker
What is the fucking point? If I buy or build a PC it is for a specific purpose not for some overblown OS company to decide what to do with it.
I have a couple of machines for coding, a self-built gaming rig and a gaming lappy. My devices only run specific s/w for specific tasks, I don't need advanced bullshit functionality or so-called ai telling me what it thinks I want to do. I know what I want to do. Fuck'em.
Domestic Abuse
Imagine what an abuser could do with access to their partner's history.