The Pulse of the Planet

Source details

The variety, frequency, and granularity of data sources have increased dramatically, creating a “big data” universe that requires a new mindset compared with the traditional “small data” world.
Benefits of new data sources:
- Greater accuracy and a closer reflection of “ground truth.”
- Faster reporting, potentially in quasi real time.
- Much higher granularity, down to small segments or individuals.
Cautionary parallels:
- Professionals can become accustomed to compressed or imperfect information (the MRI “classic” vs. updated scans), resisting better data because skills and practices are tuned to older signals.

Private-sector data sources and examples:
- Payroll processor ADP handles one in six US workers and provides a monthly jobs report used to supplement official statistics.
- LinkedIn’s “economic graph” measures 1.2 billion people, 67 million companies, 15 million jobs, 41,000 skills, and 133,000 schools.
- PriceStats tracks changes in 800,000 daily prices from among 40 million products in 25 economies.
- Carlyle provided employment trends during the US government shutdown, managing 277 companies with 730,000 employees.
- Intuit’s QuickBooks-based small-business index was discontinued in 2015 and relaunched with a different methodology in 2023.
Historical performance and potential improvements:
- Official US GDP first reported a 3.8 percent decline for Q4 2008, revised to a 6.2 percent decline a month later, and finally recalculated as an 8.9 percent decline in July 2011—highlighting large reporting and revision lags where alt-data might have identified downturns earlier.
- During the COVID-19 pandemic, smartphone GPS data measured declines in retail visits and adherence to lockdowns, demonstrating how alt-data can fill gaps during crises.

Strengths:
- Can fill analytical gaps, especially in developing economies lacking meteorological equipment or statistical capacity (e.g., using cell-tower signal degradation to measure rainfall).
- Acts as an independent tool for transparency when official data integrity is questioned.
Limits and biases:
- Often generated as “data exhaust” and carries biases inherent to the private-sector environment (examples: LinkedIn skewing toward professionals; ADP omitting the gray economy).
- Private-sector data may disappear or change methodologies (e.g., Intuit), so alt-data should complement, not replace, official statistics.
- Practical and ethical constraints on collecting hyper-granular personal metrics (health sensors, facial recognition, biosensors), including privacy concerns and potential state overreach.

Privacy-preserving analytics:
- Emerging techniques—federated learning, homomorphic encryption, secure multiparty computation, and differential privacy—allow analysis without exposing raw personal records; these techniques are nascent but under experimentation by companies and statistical offices.
Institutional shifts required:
- Statistical practitioners may need to reconceive their role from solely generating official information to partnering with the private sector to validate and bolster data integrity.
- Speed of implementation matters: “Unless we concurrently increase the speed of implementation, ‘big data’ is of limited use,” Kenneth Greenspan (interview, 2014) observed.

Data is always an abstraction, not the thing itself; it contains an “information quotient” that can rise as measurement improves.
The expansion of measurable phenomena opens the possibility of creating new metrics (for example, a modernized “misery” metric that aggregates behavioral, financial, and biometric signals), but such metrics raise profound privacy and ethical questions.
Public sentiment and “techlash” may limit unbridled data collection; paradigms and adoption of new metrics can be slow and contested.

Combine official and alternative data sources to leverage complementary strengths and mitigate weaknesses.
Foster partnerships between statistical agencies and private firms to validate, standardize, and preserve the integrity of alt-data.
Invest in privacy-preserving analytic techniques and experiment with their deployment to enable useful analysis without exposing individual records.
Prioritize increasing the speed of implementation for data-driven insights so that faster, more granular indicators can be actionable in policymaking.
Promote creative use of existing private-sector infrastructure (for example, mobile network data) to fill measurement gaps in resource-constrained settings.

F&D Magazine — Kenneth Cukier, December 2025

The Pulse of the Planet
- The Pulse of the Planet (Markdown version)
- The Pulse of the Planet (PDF)