We see the world not as it is, but as we are. In the domain of data, economists need to rethink what information they use to portray ground truth, and to reimagine what truth they wish to record. The field suffers from a “déformation professionelle,” viewing the economy through the lens of a “small data” world they have long known. But in a “big data” universe—when the variety, frequency, and granularity of data sources (and features to measure) are vastly more numerous—a new mindset is required.  

To get a flavor of what such a collision looks like between more information and traditional thinking, consider a bit of history from the field of health care. 

In 1990 General Electric released an update to the software for its Signa magnetic resonance imaging (MRI) machines, used for medical scans. Engineers had uncovered a flaw in the system that compressed how it showed tissue containing lipids, or fat. But when the more accurate images became available, many radiologists rebelled. They were unaccustomed to seeing the better scans and felt more comfortable assessing the older ones. There were fears of misdiagnoses owing to new images. GE was forced to add a feature to the MRI machines that let radiologists see the old scans—labeled “classic,” in a nod and a wink to the debacle over the launch of “new Coke” a few years earlier.

An MRI scan is pictorial, informational. It’s not the thing itself. In this way, it’s a bit like economic data, such as growth, unemployment, inflation, and the like. The radiologists in the 1990s preferred the information that was less accurate because they had become accustomed to using compressed scans; their skills were largely honed to work within those constraints. They resisted better images. Is there a risk that today’s economists are vulnerable to the same mental trap?

Galaxy of data

Consider the galaxy of data and AI all around us today, and how novel it is. A quarter-century ago, most things in life did not have a computer chip or connect to a network. It was a bygone age of letters, subway tokens, travel alarm clocks, and credit card transactions that required a signature on a carbon paper form after going through an imprinter, known as a zip-zap machine. Your sleep and exercise weren’t tracked by your wristwatch. Your cordless phone didn’t recognize your face; your bank didn’t verify your voice signature. Cars without satnav systems meant drivers relied on badly refolded maps. Don’t be wistful: The point is that the digitalization of society means that activities that could never be easily rendered into data now are.

This offers the possibility to understand the economy in ways that are more accurate, a better reflection of ground truth, the actual thing being measured. Reporting can happen much faster, perhaps in quasi real time, and in ways that are more granular, down to small segments or even individuals, which older methods were incapable of—instead compressing information like a pre-1990 MRI scan. Accuracy, speed, and details improve. Moreover, what gets measured can itself change, leading to new ways to understand the world (and by doing so, hopefully improve it).

Yet the entities compiling the information will come from the private sector, since it is generating the data in its operations. For example, satellite imagery can track farm yields. Job posting sites can identify which urban areas are growing faster than others, while home sale sites can show which are in decline. In many instances, firms find themselves in the middle of data flows from others’ operations. The payroll processor ADP handles one in six US workers: Its monthly jobs report is used by economists to supplement data from the US Bureau of Labor Statistics.

Alternative indicators

Such alternative indicators (or “alt-data”) may not be compiled using the academically rigorous methods of state statistical agencies. Harnessing the data will require a shift in thinking by today’s practitioners—who may need to reconceive their responsibility, from generating information to working with the private sector to bolster and validate the data’s integrity so that it can be used for broader purposes. It is an echo of the field’s origins. 

The term statistics derives from the German “Statistik,” coined in the mid-1700s to mean the “science of the state.” Such metrics may be based on inference: generalizing from what is easily measurable to reach conclusions about what is hard to learn. Because it was often expensive or impossible to count the things themselves, the accepted practice was to find proxies and extrapolate. This approach characterized stats’ earliest days. “The city of Dublin in Ireland appears to have more chimneys than Bristol, and consequently more people,” wrote William Petty at the start of an essay on “political arithmetick” in the 1680s to estimate populations.

Today, developed economies spend billions of dollars a year to produce reliable economic and social indicators. To the high priests and priestesses of official metrics, it is a holy calling, a mark of civilization. “Knowledge is power: Statistics is democracy,” famously stated Olavi Niitamo, who led Statistics Finland from 1979 to 1992.

Data is only a simulacrum of what it aims to quantify, qualify, and record. It is an abstraction, never the thing itself, just as a map is not territory and a weather simulation won’t get you wet. Data contains an “information quotient” of what it depicts. As the world changes, so too must the statistics with which social scientists take the measure of man. Despite worldly philosophers embracing more serious methods to establish a dismal science, informal proxies and extrapolations are still used. 

Anecdata

Alan Greenspan, the Federal Reserve chairman from 1987 to 2006, is infamous for embracing “anecdata”—a cross between anecdote and data—to get a leg up on official indicators. As a young economist, among the data he scrutinized were sales of men’s underwear. In his thinking, it is an economic bellwether: the sort of thing people cut back on when belts tighten.

His successors at the Fed followed his lead. At the start of the financial crisis in 2008, just days after Lehman Brothers’ collapse, Janet Yellen, then president of the San Francisco Federal Reserve Bank, warned of a nasty economic downturn during a Federal Open Market Committee meeting. “East Bay plastic surgeons and dentists note that patients are deferring elective procedures,” she reported, according to transcripts released five years later. “Reservations are no longer necessary at many high-end restaurants.” Her colleagues laughed.

How did the statistical agency do? In the fourth quarter of 2008, the first figure released for the US was a decline in GDP of 3.8 percent. That was quickly revised a month later to a drop of 6.2 percent. In the final revision, in July 2011, it was recalculated as having fallen by 8.9 percent—the largest downward revision of GDP on record, and more than twice as bad as first reported. Perhaps alternative indicators would have helped.

The new data sources might have done a faster and better job than existing indicators, and with more detail. For example, ADP, the payroll firm, could have spotted a decline in new employees and a slowdown in pay raises. Google searches related to home purchases may have slowed precipitously. Likewise, professional job listing sites like LinkedIn and Indeed have a lens on recruitment ads—not only those that are posted, but those that are pulled. (That data is used by investors since it’s an early predictor of business wobbles and analyst downgrades, and thus stock prices.)

Tool for transparency

During crises, official metrics may fail because of reporting lags. Alt-data flourished at the outset of the COVID-19 pandemic. GPS in Apple and Android phones measured a decline in visits to retailers—and revealed which places disobeyed lockdown orders. Likewise, during the US government shutdown in October 2025, statistical agencies could not release data so the private sector filled the breach. Employment trends were provided by ADP and Carlyle, a private equity fund managing 277 companies with 730,000 employees.

Alt-data holds governments accountable. Argentina’s official inflation data became so ridiculous in the early 2010s that The Economist used figures from PriceStats instead, a company founded by two economists from Harvard Business School and the Massachusetts Institute of Technology. It tracks changes in 800,000 daily prices from among 40 million products in 25 economies. As questions are asked about the integrity of US data after the head of the Bureau of Labor Statistics was fired by President Donald Trump in August 2025, following a negative jobs report, alt-data can be an independent tool for transparency.

The explosion of new data sources and techniques is especially important in developing economies, which lack the institutional capacity, funds, skills, and political will to collect, analyze, and report statistics. With creative thinking, private sector data can be transformative. For example, many developing economies cannot afford meteorological equipment in remote areas to measure weather events like rainfall, for advance notice of flooding. But mobile operators have cell phone towers across the countryside. These towers are constantly communicating with each other for network information and to hand off traffic. Yet the strength of the signal weakens in rain—making them useful to measure rainfall. More such originality is needed to overcome data gaps in poor places. 

However, creating more accurate, granular, and timely indicators means little if there is no way to use them effectively. “Unless we concurrently increase the speed of implementation, ‘big data’ is of limited use,” said Greenspan in an interview I conducted over email in 2014. 

Brave new world

Moreover, the stakes are even higher than the need to improve what exists or fill in known gaps. The datafication of activities that have never been rendered into the form of data offers a unique opportunity to learn new things about the world. Society is just at the outset of a major transformation in understanding. 

An early dimension of this transformation is LinkedIn’s “economic graph.” It measures the work activities of 1.2 billion people, 67 million companies, 15 million jobs, 41,000 skills, and 133,000 schools. Many countries use it to answer questions like “What skills are growing quickest, what places are gaining and losing jobs, how hard are mid-career transitions by occupation, and in which industries and countries are women in more senior leadership roles?” This information could never be tracked, analyzed, and compared until now.

Though such deep analysis of people’s personal information may seem to threaten privacy, it need not. Advanced data processing techniques—with space-age names like federated learning, homomorphic encryption, secure multiparty computation, and differential privacy—allow for analysis of encrypted data, so the actual record is not visible to the data processor. The system is still in its infancy since it’s hard to pull off. But companies and statistical offices are already experimenting with it.

Of course, there are limits to using corporate “data in the wild.” It’s often in the form of data exhaust—that is, generated as a by-product of a company’s regular business activities. Hence it will contain the biases of that environment. Carlyle’s firms accepted a private equity owner (so perhaps were not the strongest); LinkedIn probably has more professionals than working-class members (so perhaps skews wealthier); ADP is silent on the gray economy of nannies, house cleaners, car washers, and the like (whose numbers may be even stronger signals of economic health).

Furthermore, alt-data can’t be relied on entirely if it may not always be around. For example, the US software company Intuit produced a small-business index based on aggregated data from its QuickBooks accounting software. But in 2015 it discontinued the reports—before relaunching them with a different, more robust methodology in 2023. So the future won’t be based solely on alternative data, but on complementary official and unofficial sources. Still, this is a brave new world. 

Modern metrics

And that brings us back to the MRI. Magnetic resonance imaging dates to 1974, when it was patented by Raymond Damadian of the State University of New York as a noninvasive way to detect cancer. That same year marked a brutal recession in the US, which inspired a Yale University economist and former White House advisor, Arthur Okun, to create a new indicator to account for its toll on individuals, not the abstract unit of the economy as a whole. 

His Economic Discomfort Index—later dubbed the “misery index”—became a staple of US politics. Ronald Reagan used it to defeat President Jimmy Carter for the presidency in 1980. But it is simply the sum of the unemployment and inflation rates. A modern metric for the AI era is easy to imagine.

It would gather all the ways people might express their misery, from shifting spending patterns—not buying fewer things (a blunt number) but actually switching from eating steak to ramen. Likewise, missed utility bills and overdue car payments. Then, incidents of road rage, erratic driving, and fender benders—not in aggregate but tracked down to the person. Apple watches can track the quality of sleep and stress during the day. Closed-circuit TV cameras in streets, shops, and offices that have facial-recognition capability can record individuals’ emotions. Toilets with biosensors can track users’ levels of hormones such as cortisol and epinephrine that spike in moments of anxiety.  

This comes as close to ground truth as it gets. Such sci-fi stats probably strike many people as true misery: The privacy implications are frightening, even if the data, in theory, could be anonymized. Armed with such information, doesn’t the state have a duty to intervene to help individuals and protect society? “After such knowledge, what forgiveness?” lamented T. S. Eliot.

Such alt-data won’t happen soon, if ever. Paradigms shift one funeral at a time. And a “techlash” is gathering force as the public grows wary of unbridled data use and the optimism of the early internet ebbs. Ideally, today’s social scientists have the care, ethics, and flexibility of mind to harness the best of AI and big data and prevent their shortcomings. After all, radiologists no longer need the “classic” view for MRI scans.