Today’s bosses have unparalleled opportunities to monitor workers’ emotions — and punish those workers for expressing anything short of cheeriness.
Call it the right to frown. Technological advances are giving employers the ability to monitor workers more closely than a human manager ever could. (RichVintage / Getty Images)
For employers, the pandemic’s acceleration of remote work presents a dilemma: How can they keep an eye on their employees when they’re no longer physically present at the workplace? Remote work appears to have little if any effect on employee productivity, but that hasn’t stopped employers from angling for new methods for keeping workers under their thumb.
Fortunately for them, the market offers a host of solutions.
The employer might install software on work devices that monitors mouse usage to ensure workers are at their desks. But some half of large companies were already using such software in 2018, and workers have found ways to circumvent it. Mouse jigglers, for instance, artificially mimic mouse usage.
So maybe the employer wants something more advanced — software like Time Champ, which claims its system is particularly hard to evade as it “includes various features such as mouse click tracking, keystroke recording, random screenshot capturing, screen recording and even live video surveillance.” Or he could try geolocation tracking, or even a tool that uses artificial intelligence (AI) to analyze employee text messages (one firm that specializes in the latter boasts Walmart, Delta, T-Mobile, Chevron, and Starbucks among its clients).
If these options aren’t enough, a recent filing with the US patent office suggests even more invasive advances for employee surveillance are coming soon. The December 2023 filing from Snap, a social media firm, seeks to patent “emotion recognition for workplace analytics.”
Taking a call-center agent as an example of the technology’s uses, which Snap describes as “workforce optimization,” the company’s patent application notes that call centers “typically stress that their employees stay emotionally positive when talking to customers, smile, and respond with respect and gratitude to whatever question a customer could ask.” By contrast, “an angry and distressed employee may not provide high quality service to a customer, which ultimately may cause reduction in revenue, negative customer feedbacks, and loss of clients, among other factors.”
Specifying that the software would be restricted to periods when a customer-service agent is video conferencing with customers — though it’s not hard to imagine such surveillance being used beyond this parameter — Snap’s technology uses a “virtual face mesh” to detect emotions such as “tiredness, negative attitude to work, stress, anger, disrespect, depression, frustration, embarrassment, irritation, sadness, indifference, confusion, annoyance, and so forth.”
No matter that human emotions are notoriously varied and culturally specific — there is plenty of evidence that technologies that attempt to categorize emotions based on facial movements do not have much success. Or that the high rate of stress and turnover at call centers suggests an urgent need to improve the job. With Snap’s tool, a call-center agent yawning would translate to a negative reading, while a smile would register as a positive.
The boss could store such data for future analysis: as Snap helpfully explains, this information might be used for “workforce optimization, laying-off underperforming employees, promoting those employees that have positive attitudes towards customers, or making other administrative decisions.”
Enough Is Enough
Employers’ desire to control employees will never be sated, and they will never run out of companies hawking the latest methods. The acceleration of this trend during the pandemic raises a question: What are the consequences of allowing employers to deploy ever-more invasive methods of high-tech surveillance and control?
Sociologist Arlie Hochschild coined the phrase “emotional labor” to describe the work required of employees in customer-facing roles (flight attendants were her key example), and workers have always had to pretend to be content around the boss, no matter their actual mood. But with new surveillance technologies, we can now imagine workers forced to keep a grin glued to their face for the entirety of a shift to avoid being flagged as a candidate for layoffs.
With new surveillance technologies, we can now imagine workers forced to keep a grin glued to their face for the entirety of a shift to avoid being flagged as a candidate for layoffs.
All of this has real consequences for workers’ right to organize. If an algorithm is monitoring your every move, word, and even facial expression, good luck finding a way to talk to your coworkers about the workplace problems that were making you grimace in the first place.
Workers are starting to make this argument themselves. Amazon workers at STL8, a fulfillment center outside of St Louis, Missouri, recently filed an unfair labor practice (ULP) charge with the National Labor Relations Board (NLRB) arguing that the company’s plethora of surveillance technologies in itself constitutes a violation of their rights. As the Guardian reported, the charge alleges that Amazon has “maintained intrusive algorithms and other workplace controls and surveillance which interfere with Section 7 rights of employees to engage in protected concerted activity.”
Amazon is on the cutting edge when it comes to workplace surveillance. Every second of a warehouse worker’s shift is tracked, with workers alerted when they’re falling behind Amazon’s grueling productivity expectations, urged to pick up the pace. Managers monitor this data, intervening when a worker is too slow and, if they don’t improve, terminating them.
When employees start to organize, as they have been doing in growing numbers recently, the company’s monitoring tools are there to catch it: Amazon’s human resources department monitors employee message boards, and the company uses software to track possible organizing threats. As one recent job listing put it, Amazon uses intelligence analysts to monitor “labor organizing threats.”
“We don’t want this becoming the norm,” Wendy Taylor, a packer at Amazon’s STL8 and a member of the shop’s organizing committee, told the Guardian. “We have an injury crisis and we’re being watched and you feel like you’re in prison. We just want people to know we have a right to organize, the right to form a union and the right to have a safer work environment.”
As NLRB general counsel Jennifer Abruzzo wrote in a memo on the need to protect workers from AI-enabled monitoring of organizing activities, “It is the Board’s responsibility ‘to adapt the Act to changing patterns of industrial life.’ An employer’s right to oversee and manage its operations with new technologies is ‘not unlimited in the sense that [it] can be exercised without regard to any duty which the existence of rights in others may place upon [the] employer.’”
Call it the right to frown. Technological advances are giving employers the ability to monitor workers more closely than a human manager ever could, and as Snap’s patent suggests, it will only get worse. The right to organize is not meaningful if employers are granted an unlimited right to surveil workers.