The fine-graining of data collection and increasing isolation of consumers is leading to surveillance pricing, a new trend where corporations exploit personal information to set individualized prices for each person.
Walmart application view. (Andrew Harrer /
Bloomberg via Getty Images)
Six years ago, I was at a conference at the University of Chicago, the intellectual heart of corporate-friendly capitalism, when my eyes found the cover of the Chicago Booth Review, the business school’s flagship publication. “Are You Ready for Personalized Pricing?” the headline asked. I wasn’t, so I started reading.
The story looked at how online shopping, persistent data collection, and machine-learning algorithms could combine to generate the stuff of economists’ dreams: individual prices for each customer. It even recounted an experiment in 2015, where online employment website ZipRecruiter essentially outsourced its pricing strategy to two University of Chicago economists, Sanjog Misra and Jean-Pierre Dubé.
ZipRecruiter had previously charged businesses one fixed monthly price for its job-screening services. Misra and Dubé took the information that ZipRecruiter asked prospective clients on an introductory registration screen about their location, industry, and employee benefits. In the initial part of the experiment, ZipRecruiter assigned a random price to each business. The researchers could then see which attributes correlated with a willingness to pay higher prices. “There were enough things that people involved had disclosed at the registration stage that were associated with their price sensitivities that we could build a pricing algorithm around it,” Dubé told me in an interview.
Sure enough, when ZipRecruiter deployed the algorithm to deliver tailored prices based on the questions customers answered, profits went up 84 percent over the old system. Misra became an adviser to ZipRecruiter. The algorithm isn’t used today, but remnants of the pricing strategy remain: ZipRecruiter’s FAQ page promises to “customize” what it charges based on particular attributes.
Businesses have always wanted to maximize what they can induce people to pay, trying to walk right up to the limit before a customer says no. But everyone has a different pain point, and companies were deterred from purely individualizing what they charge, because of publicly posted prices and consumer anger over the unfairness of being charged differently for the same product.
Today, the fine-graining of data and the isolation of consumers has changed the game. The old idiom is that every man has his price. But that’s literally true now, much more than you know, and it’s certainly the plan for the future.
“The idea of being able to charge every individual person based on their individual willingness to pay has for the most part been a thought experiment,” said Lina Khan, chairwoman of the Federal Trade Commission, a federal agency that oversees antitrust law and consumer protection. “And now . . . through the enormous amount of behavioral and individualized data that these data brokers and other firms have been collecting, we’re now in an environment that technologically it actually is much more possible to be serving every individual person an individual price based on everything they know about you.”
The old idiom is that every man has his price. But that’s literally true now and it’s certainly the plan for the future.
Economists soft-pedal this emerging trend by calling it “personalized” pricing, which reflects their view that tying prices to individual characteristics adds value for consumers. But Zephyr Teachout, who helped write anti-price-gouging rules in the New York attorney general’s office, has a different name for it: surveillance pricing.
“I think public pricing is foundational to economic liberty,” said Teachout, now a law professor at Fordham University. “Now we need to lock it down with rules.”
When Theory Becomes Reality
“How much?” is a question shoppers ask at yard sales and flea markets across the country. Vendors size them up and try to find the sweet spot: a price both sides will accept, one that tips customers to say yes while making the transaction profitable. In the 1800s, this was the process behind virtually every retail sale. Without fixed prices, sales clerks would haggle with shoppers, aiming for that sweet spot. Some would get a discount; others would overpay.
John Wanamaker opened his flagship department store in an abandoned railroad depot in Philadelphia in 1876 with a novel idea: affixing a price tag to each item. It was a big step in a nation that was fed up with differential pricing. One of the reasons railroad monopolies inspired such Progressive Era fury had to do with the rates they charged for transporting and storing crops. Large shippers got volume discounts, and yeoman farmers were stuck paying more, which they condemned as price discrimination.
The populist Granger movement farmers established led to the “just and reasonable” rate regulation in the Interstate Commerce Act of 1887, which prohibited any special rates, rebates, or preferential treatment for any shipper, product, or destination. “When you read Ida Tarbell, she says on the railroads everyone is talking about price discrimination . . . allowing companies to pick a different private price for each person,” Teachout said.
But public prices didn’t extinguish the dream for private ones, and businesses innovated. Some industries set expectations for differential prices, like airlines after deregulation. Airfares primarily hinged on the supply of available tickets, but midweek trips — likely for business on the company’s dime — cost more. Negotiations at car dealerships followed the flea market model, where salespeople could pick up clues from customers, like how someone’s residence could provide an approximation of household income.
Eventually, companies longed for real information to inform prices. Catalina Marketing used a shopper’s basket of purchases to generate real-time coupons attached to the checkout receipt. Catalina would later combine knowledge from purchases with information gathered from the credit card used to pay. Then, loyalty cards gave retailers a graph of a customer’s full shopping history, along with crude demographic characteristics like addresses and phone numbers. That allowed grocery stores to make even smarter offers.
The results were surprisingly robust. A 1996 paper by three economists and statisticians found that “rather short purchase histories can produce a net gain in revenue from target couponing which is 2.5 times the gain from blanket couponing,” the authors wrote. Even a single purchase history improved revenue on coupons by 50 percent, according to the study.
Business school academics who study personalized pricing have appeared ebullient about its possibilities. And there are many academics to choose from. There are courses taught at MIT on using data to “improve” pricing, complete with a movie-style trailer; Harvard has an entire department called the Pricing Lab, which analyzes data and conducts experiments, like the Billion Prices Project.
Their theory is that an individualized price is better for the consumer. The ZipRecruiter experiment found that 60 percent of consumers in the sample paid less with personalized prices. But making things cheaper isn’t really what economists mean by “better for the consumer.”
If you really want or need something, with personalized pricing you will pay more. Some might call exploiting the human impulse of desire unfair; Dubé and other economists economists call it an efficient allocation of resources.
“Using personalized pricing,” Dubé explained, “while it’s true some people are going to pay higher prices, I could vastly increase the set of people who are actually going to be able to buy.” In other words, someone who really wants that snazzy handbag will be charged $300, and someone who wants it less will get it offered to them for $200 or even $150. In the end, more people get a handbag.
This is about the willingness to pay, not the ability to pay. If you really want or need something, and the seller knows it, with personalized pricing you will pay more. Some might call exploiting the human impulse of desire unfair; Dubé and other economists call it an efficient allocation of resources.
“Theory is a playground for the mind but not reflective of public-policy preferences,” said Lee Hepner, legal counsel with the American Economic Liberties Project. He thinks that economists often deflect from the true purpose of this type of pricing. “The literature even acknowledges that personalized pricing is a transfer of wealth from consumer to the seller. Writ large, the goal and endgame is to maximize revenue.”
The World After E-Commerce
There have been two binding constraints for true personalization: the quality and quantity of data collected, and the mechanism for giving individual prices to people who shop where price tags are publicly displayed. Step by step, these constraints are being defeated, and a new frontier on pricing is becoming available.
E-commerce really served both ends. Instead of being out in the world, people shop from home, unaware of any uniform price. And data that can be grabbed over the internet dwarfs what’s available on a loyalty card. It includes your IP address, the devices you use, your phone number, email, pinpoint demographics, and a comprehensive graph of everything you’ve ever done on the internet, from purchases to searches to websites visited to emails to social media posts and much, much more. And if the retailer doesn’t get all that information, they could always just buy it.
The biggest online retailer and hoarder of purchasing data immediately tried to exploit this circumstance. In 2000, Amazon varied its prices randomly for top DVDs and MP3 players, as a blind experiment to see what price points worked. But users compared notes in chat rooms and figured it out. One shopper deleted browser cookies and got served lower prices. Jeff Bezos had to apologize and Amazon actually sent out thousands of refunds. The company claims to this day that they never differentiate prices.
Other companies have also been caught. In 2012, travel site Orbitz steered Mac users to pricier hotels, after learning that they tended to spend $20 to $30 more per night. The Wall Street Journal found out and Orbitz stopped. The Princeton Review was charging more for SAT prep to ZIP codes that contained a high percentage of Asians; the company stopped asking for ZIP codes. (It now appears to get the information it needs for differential pricing from a user’s IP address.)
But it’s a big internet, with billions of prices. The same year as the Orbitz article, the Journal also found the same Swingline stapler charged somewhat differently on Staples.com on two computers just a few miles apart. Forbes found similar results in 2014. The largest restaurant franchise operator in America, Flynn Restaurant Group, employs data scientists to develop pricing models that correspond to individual locations.
Perhaps the main reason to suspect that there are thousands of examples of surveillance pricing that haven’t been caught is the explosion of so-called “pricing consultants,” who nudge everyone who sells a widget online to sign up for AI-powered services to extract data, analyze it for insights, and deploy prices that are contoured to particular customers.
Perhaps the main reason to suspect that there are thousands of examples of surveillance pricing that haven’t been caught is the explosion of so-called ‘pricing consultants.’
Ninetailed, a data-driven personalization and experimentation technology, helpfully explains “why you should use personalized pricing,” arguing that “it can help you build rapport with your customers” by giving them a customized experience. Catala Consulting focuses on hotels, also praising personalization’s ability to build loyalty and increase profitability. And yes, uber-consultant McKinsey is circling around this concept too, probably winning an euphemism award by calling price discrimination “digital pricing transformations.”
The most detailed of these I found comes from a business-to-business consultant called Cortado Group, which calls personalized pricing “a cornerstone of modern business strategy.” It offered a “compelling real-world example” of an unnamed “e-commerce powerhouse” that used browsing and purchasing history and even the amount of time spent on product pages to “craft pricing models” that “rewrite the growth curve.” Cortado says that offers from the e-commerce company were “tailored for customers with a history of consistent purchases,” and that prices for occasional shoppers were “adjusted” to boost sales. There is a hint of caution: “Striking the right balance between personalization and customer privacy is an ongoing challenge.”
I asked Cortado who this e-commerce powerhouse was. They never got back to me.
Overblown hype is endemic to digital marketing, of course. But there can’t be this many consultants going on about surveillance pricing if none of it was happening. And while the standard justification of increasing access and value works in a lab, in the real world it plays out in ways that would probably offend people, if they knew what was happening.
In the story about Staples offering different prices for estimated geolocations, the Journal wrote: “Areas that tended to see the discounted prices had a higher average income than areas that tended to see higher prices.” In the consumer loan context, reduced ability to pay — a lower creditworthiness — is correlated with higher prices. A study of broadband internet offers to 1.1 million residential addresses showed the worst deals given to the poorest people.
In America, it’s always been expensive to be poor. The classic study of the subject, The Poor Pay More, published by the sociologist David Caplovitz in 1963, recounted how sellers took advantage of the limited knowledge, limited options, and limited time of lower-income consumers to price-gouge on everything. What’s new is that personalized price algorithms now reduce this process to a science, not just for the poor but for everyone.
The App Knows Everything
Dubé, the University of Chicago economist, suggested to me that advances in privacy laws, particularly from the General Data Protection Regulation in Europe and copycat legislation in a dozen countries, make it harder to collect enough data to truly personalize prices. He conceded that the algorithms have improved but he insisted that Google’s elimination of cookies and Apple’s Ask App Not to Track standard for the iPhone make persistent tracking less available.
Jeff Chester begs to differ. “The system is in place to deliver personalized pricing,” said Chester, who runs the Center for Digital Democracy.
The digital surveillance we know about comes from platform companies like Meta, Google, and Amazon, which have built colossal advertising business lines out of social media, search, and e-commerce. But what’s emerging is even more invasive, and more primed to find customers in unusual places, where they will have no idea what the common price might be.
You might be aware that fast-food companies like McDonald’s have begun pushing customers to their apps. Deals on McDonald’s app are extremely good, at least for now: $1 breakfast sandwiches, 20 percent off any purchase above $5. That’s because McDonald’s, whose CEO has talked on earnings calls recently about a “street-fighting mentality” in winning customers, wants to burrow into phones, where they can access more personal data and get people hooked on an app where specific prices can be customized to the user.
What McDonald’s is doing is almost a throwback, kind of a high-tech loyalty card for the digital age. Worldwide, 150 million active members are now on the McDonald’s app, which is run by a company called Plexure that specializes in “personalized mobile engagement.” McDonald’s has a nearly 10 percent stake in Plexure, which also works with IKEA, 7-Eleven, White Castle, and more.
A Plexure slide presentation viewed by the Prospect stresses the power of customized engagement. It starts with using a cheap offer to entice users to purchase through the mobile app. After that, various factors go into the process of “deep personalization”: Time of day, food preferences, ordering habits, financial behaviors, location, weather, social interactions, and “relevance to key moments i.e. pay day.”
It doesn’t take much brainpower to devise ways to exploit this data. If the app knows you get paid every other Friday, it can make your meal deal $4.59 instead of $3.99 when you have more money in your pocket. If it knows you usually grab an Egg McMuffin before class on Wednesday, or that you always only have an hour to eat dinner between your first and second job, it can increase the price of that promotion. If it knows it’s cold out, it can raise the price of hot coffee; on a scorcher, it can up the price of a McFlurry. And the app gets smarter as you agree to or turn down those offers in real-time.
If the app knows you get paid every other Friday, it can make your meal deal $4.59 instead of $3.99 when you have more money in your pocket
It doesn’t sound like much, but with 300 million customer interactions across its range of apps every day, Plexure can magnify tiny price shifts into real money. The company promises that using its app strategy will increase the frequency of orders by 30 percent and the size of orders by 35 percent. Domino’s just attributed its strong first-quarter earnings, with income increasing by 20 percent over last year, to its loyalty program. Grocery stores like Walmart and Kroger have also gotten into this, leveraging purchasing history with digital targeting. And improving artificial intelligence can just make this all move faster.
Like everything else on the internet, the goal is to addict the user. Plexure and other digital marketers talk openly about the “dopamine rush” that getting a targeted deal can release. And the reason those deals are so smart about their subjects has to do not only with Plexure’s tracking of data from within the app, but the agglomeration of that with everything else in your digital, and even non-digital, life.
McDonald’s uses an “identity graph” provided by a company called LiveRamp, which combines multiple levels of data associated with an individual. That includes email, social media, and browser activity; behavior on streaming video sites or other smart devices; your subscriptions and app downloads; and histories from travel, retail, financial, auto, and even medical partners.
It’s hard to put into words how powerful this can be: an app with predictive capabilities that far exceed your own brain. “Individualized pricing is certainly one expression of surveillance capitalism’s information asymmetries,” said Shoshana Zuboff, author of the 2019 book The Age of Surveillance Capitalism. “You can easily see that the companies have a nearly infinite inferential opportunity to know and predict their customers.”
Dubé conceded that digital retail advertising is “exploding,” but he sees it as nothing more than a bargain between retailers and consumers. “[The retailer] says, I’m going to pay you. How am I going to pay you, I’m going to give you discounts,” he told me. “But in exchange for those discounts, I’m allowed to track you.” He considers it “a fair and equitable form of consent.”
But to Chester, the point is that companies are setting up the architecture to use rampant spying to set prices. The framework gets around data privacy protections because so much of it is “first-party” data collected directly by companies, which as Dubé argues, provides the consent needed for online promotions.
“They want [customers] to say, ‘Of course I want the discount and loyalty points.’ So you’re consenting,” Chester explained. “And not only do they have geolocation and other information. But because they have consent, they’re able to leverage that data … They want your permission to freely continue to target you.”
Can They Keep a Secret?
Your phone apps are not public storefronts. Nobody else sees the offers you’re getting. The coupons may not be the same as someone else’s and may change depending on your behaviors and habits. That’s how businesses can personalize prices.
Another method is through a smart TV, a prospect that has taken streaming companies, television manufacturers, and advertisers by storm.
In January, at the Consumer Electronics Show in Las Vegas, Disney announced the “future of entertainment and advertising” with a variety of new technologies, including one that reads scenes in Disney’s vast online library and allows companies to match the mood with ads. One of the most powerful features, Gateway Shop, “allows consumers to access personalized offers for purchase from a retailer without leaving the viewing environment.” In other words, viewers will soon be able to see an individualized offer for a product they just saw in a program, and can send it to their computer or phone for seamless purchase.
Amazon has a similar concept: “dynamic ad insertion.” At specific markers in a streaming broadcast, Amazon Web Services can place personalized ads tailored to a household, with special offers. With this in place, you could get served an ad with a different flavor of soda than the house down the street, one predicted to be more to your liking.
Most streaming media and tech companies have bought into these experiments. Walmart and NBCUniversal’s Peacock streaming platform made a deal to display “shoppable ads,” where you can buy items featured in the Bravo show Below Deck Mediterranean through your remote or with a QR code. Roku has a similar deal with Shopify for ads that enable purchases through a smart TV.
Kroger and other grocers, which have huge pipelines of first-party data, have inked deals with streaming companies. 84.51°, a media company Kroger acquired in 2015 (Kroger renamed it based on the coordinates of its corporate headquarters in Cincinnati), boasts of “leveraging data from over 62 million households in the U.S.” Albertsons has similar data reach and partnerships with tech firms. This is the reason why Chester opposes the Kroger and Albertsons merger, which would combine the data of what are really two media companies, bringing their datasets to a new, narrow marketplace where they don’t have to be concerned with posted prices.
People have underestimated the role advertising and marketing plays in the media system. It’s the key underpinning.
The proposed merger between Walmart and Vizio, a smart-TV manufacturer, makes a ton more sense in the context of being able to deliver targeted ads that people can buy direct from the television. Vizio’s SmartCast system has the features of a streaming media site; it serves advertising to viewers based on their data profiles. Vizio is under a federal consent decree right now for collecting user viewing histories without consent.
It’s very clear that Walmart wants to link up Vizio’s capabilities with its retail media arm, Walmart Connect, for the purposes of direct-to-consumer advertising over the smart TV. A letter from 19 groups opposing the merger notes, “Acquiring Vizio will enable Walmart to further grow its business lines that rely on extracting, monetizing, and exploiting consumer data.”
If you thought prices on an app were shrouded in secrecy, prices paid in the home, through your TV set, through your Alexa speaker, through your smart refrigerator, will be even more inscrutable. “We’re talking about a seamless link between platforms, brands, ad agencies, and retailers,” Chester explained. “People have underestimated the role advertising and marketing plays in the media system. It’s the key underpinning.”
Policy Comes for Surveillance Pricing
Surveillance pricing is done in the dark because companies know there would be some degree of anger if a product with one price suddenly had 330 million. Any kind of discrimination only works well in secret, before too many people understand the implications. “What people fundamentally want as a public-policy goal is predictable pricing,” Hepner said. “If people were aware that they were paying differently, they would be upset.”
Dubé thinks the concern over advertising promotions is misplaced. “If I literally tell you, the price of a six-pack is $1.99, and then I tell someone else the price of a six-pack for them is $3.99, this would be deemed very unfair if there was too much transparency on it,” he said. “But if instead I say, the price of a six-pack is $3.99 for everyone, and that’s fair. But then I give you a coupon for $2 off but I don’t give the coupon to the other person, somehow that’s not as unfair as if I just targeted a different price.”
But the fact that one person can get that coupon and the other cannot is the point. If people understood that, they would see it as a form of discrimination. One way to stop companies from engaging in that discrimination is to simply reveal it out in the open. Several academic papers warn companies of going too far with personalization, of causing backlash. The problem with a sunlight-only approach, however, is that companies have listened, and learned how to shroud pricing strategies in neutral or even consumer-friendly language, and to keep them far from any public sphere.
Teachout believes that corporate hesitancy to roll out surveillance pricing in a widespread fashion gives policymakers an opportunity. “This turns the public open market into private fiefdoms, and puts people at the whim of algorithms,” she said. “It’ll be a major battle for what the Ubers of the next generation will call the right to price. But now’s the time to do it before it’s embedded in every price interaction.”
One question that raises is how you actually prohibit surveillance pricing, which includes two considerations: the collection of personal information, and the exploitation of that information to set differential prices. Protecting against the former would involve data privacy rules; protecting against the latter is more standard price regulation. Teachout sees the need for a mix of both.
Some targeted advertising is already banned, particularly to children; the FTC is in the midst of a case against Meta about that. And some privacy rules, like those protecting personal health information, are already in place. There are disclosure regimes where companies must tell consumers how their personal data is being used; those could be extended to price discrimination.
It’ll be a major battle for what the Ubers of the next generation will call the right to price. But now’s the time to do it before it’s embedded in every price interaction.
But policy decision-making thus far has been uneven. The bipartisan deal on a federal data privacy standard announced last month specifically exempts first-party advertising, the very process that is being exploited to deliver personalized prices. On the other hand, the Department of Transportation has announced a privacy review of major US airlines, another industry with access to significant data that it could sell or monetize. If an airline is planning to insert ads or offers based on an identity graph into the seatback video screen of a passenger, the review should catch that.
FTC chair Khan has pursued innovative efforts to label data broker collection and sales as an unfair practice. The data broker is of course only one of the many entities sharing data with one another, but the efforts create an opening to manage, limit, or even ban the identity graph on fairness grounds.
“The FTC under Lina Khan understands the system that has emerged,” said Chester. “The fact that it is trying to regulate commercial surveillance shows it understands the potential for consumer harm.” But he thinks it will take more than just regulating data flows. “You have to say no, NBC and Kroger and Walmart can’t work together to do offers and services and tools that allow you to manipulate the consumer.”
Credit Scores Were Just the Start
There’s something almost existential about the prospect of surveillance pricing. If sellers know when your wages increase, it’s almost not worth it to get a better job; the money will be extracted away by smart pricing. If every purchasing decision comes with doubt about whether you paid more than your friends and relatives, or more because of the time of day or a personal routine picked up on by the seller, you may spend a lot of time second-guessing your life choices. And if your every habit is intuited so well by marketers, it calls into question what agency you have in the matter.
We already have a unique identifier that follows you around as you try to navigate your financial life. It’s called the credit score, the distillation of a life of purchase histories. The origins date to the nineteenth century as a high-level form of gossip: a company called the Mercantile Agency hired a network of correspondents, who compiled markedly subjective, often biased information on people seeking credit, which was eventually distilled into a number and used by lenders. The credit score’s transformation of rumor into fact mirrors the transformation of someone’s mindless web scrolling and social media likes into an identity graph that can determine the prices they pay.
The feeling behind that, the humiliation attached to your financial picture with a scarlet FICO score, is perhaps best expressed in fiction. Gary Shteyngart’s novel, Super Sad True Love Story, set in a New York City of the near future, envisions a world with Credit Poles, lamppost-like structures that display the financial worthiness of everyone who walks by. It depicts a repressive government merged entirely with big business, and also expresses the power dynamics inherent in exposing people’s deepest secrets, in public, for all to see.
Shteyngart gets a lot of this right, and the technology underpinning the Credit Pole may be coming to a phone or TV set near you. But Super Sad True Love Story ends in revolution. Corporate America has figured out that not everything should be on display. If they have their way, only their algorithms get to see the Credit Pole scores; only they know what goes into your personal price. Whether this becomes reality depends on whether policymakers open the backroom door, and reveal the whirlwind of activity going on inside.
You can subscribe to David Sirota’s investigative journalism project, the Lever, here.