Much has been written about the terrifying privacy implications of Microsoft’s new Windows Recall “feature”.
Talking to even one person who works in cybersecurity, political advocacy, domestic abuse prevention, LGBTQ+ support, etc. would reveal the full, naked horror this sort of technology enables. To build from that, I’d like to suggest another potential negative consequence: discrimination based on ability.
Autonomy
As specialized knowledge workers we are granted, and take for granted a lot autonomy we are given over our work devices. This includes the applications we can install, how we use them, and even the larger processes that shape us needing them.
Even folks who work in more restrictive environments such as finance, healthcare, and government may forget this. The reality is most non-digital experience creation jobs that require use of a computer are incredibly restricted and inflexible, with a narrow range of acceptable software and ways to use said software.
Unlike contemporary employee monitoring solutions, Recall captures a level of detail that is effectively a realtime video playback of not only everything you do on your device, but also how you go about doing it.
Through the lens of a mandatory work install, Recall can track the precise movements and actions of employee behavior when interfacing with work systems. This easily opens up the potential for creating efficiency-tracking mechanisms—especially automated, machine vision-powered approaches.
Discrimination
I encourage you to be the villain. Think of jobs like customer support, call centers, data entry, and the manual intervention training that AI-powered systems demands.
Now think of physical disability, neurodiversity, cognitive considerations, and cultural factors. What happens if you don’t precisely use a computer in the exact way a system deems as the most efficient?
Disabled people have a difficult enough time getting employment, and this could be another obstacle in the way towards that. And even if they manage to get hired, it is also easy to envision a scenario where they, and people like them are then curve fit fired. There may also be far less of the kind of financial cushion the salary of a cushy tech job provides.
Power
Now think of the power dynamics at play for the types of jobs where this sort of scenario would play out.
A lot of these sorts of jobs are entry-level, where supply tends to be far, far greater than demand. This is likely only going to continue as more and more of these kinds of roles are automated away—while simultaneously making it worse for those who remain.
What incentives does this sort of system have to provide accommodation? What sorts of systems would this discrimination implicitly create and reinforce?
Then there’s also how this would this add to the existing pressures that surround deciding whether or not to disclose disability status in a professional capacity—those hard fought, hard won protections might not be as effective as you think they are.
But what about personal use?
Since this post is shared to social media, I should also pre-empt an obvious talking point: Windows Recall may be helpful for people with memory and executive function issues.
The consideration here is one of consent.
Opting into this sort of feature, or being informed that this feature is enabled by default while onboarding, is a lot different than having it imposed on you. This does not even begin to address notions of technological literacy and how that translates to making a truly informed choice.
Outside of personal use, the threat of losing your income and health insurance will deter people from dissenting.
There’s also the fact that this isn’t as secure as people are saying it is. To that point, there is a public, proof-of-concept exfiltration tool live on GitHub right now. Microsoft also has had some, ah, notable issues with their security practices. They also have a history of trying to force your files into the cloud.
Hammering nails down
Another more abstract concern with automated efficiency monitoring is the chilling effect it would have on our ability to improve our working conditions for ourselves.
All the little hacks and cheats and tweaks would be off the table, in that the omnipresent surveillance would punish exploration and deviation. Extrapolating on this, it would also extend to how we think about our overall approach to labor.
Latibulating
I am not a fan of Windows Recall the same way I am not a fan of most AI. I am also powerless to stop it.
It is a rushed solution that force fits a technology into people’s daily lives without communicating the full ramifications of its risks. This is pressure from the top manifesting, coating every surface area possible with an environment-murdering mass speculative bet nested inside competitive catch-up.
The worst people are making the worst things in the worst way for the worst reasons. Tech long ago shed the pretense of caring about ethics, much less talking to a single humanities major. Similarly, Microsoft can’t be trusted.
This is not a good faith feature. If I can dream up this entirely plausible scenario, imagine what other horrible, terrible things are possible.