## Use of Big Data in Producing Statistics

## Source details

**Canonical URL:** [Use of Big Data in Producing Statistics](https://www.imf.org/-/media/files/capacity-developement/d4d/june-2024/04-d4d-big-data.pdf)

## Other formats

- [Markdown version](/-/media/files/capacity-developement/d4d/june-2024/04-d4d-big-data.pdf.md)
- [Structured JSON version](/-/media/files/capacity-developement/d4d/june-2024/04-d4d-big-data.pdf.json)

---

### What is Big Data?
- Big Data encompasses new data sources, usually by-products, as opposed to survey-based data sources which are designed for production of official statistics.
- Examples of Big Data: Satellite images, vessel traffic, postings in social networks, search volumes, ...
- Broadly, nontraditional data sources include administrative data (value added tax, social security, personal and corporate income tax, customs...)
- Characteristic dimensions highlighted:
  - Very large (volume)
  - Quickly produced (velocity)
  - Different in types (variety)

*Source: IMF Statistics Department presentation by Mr. Seung Mo Choi, Deputy Division Chief, Data Governance and Services Division — JUNE 19, 2024, D4D FUND STEERING COMMITTEE MEETING.*

### Advantages of Big Data for Macroeconomic Statistics
- Timeliness
- Granularity of the analysis
- Filling the areas where official statistics are lacking

*Sources referenced in the presentation: Choi and Iyer (2022); Duval and others (2022); Huang and others (2019).*

### Leapfrog opportunity in macroeconomic statistics
- Increasing adaptation of technologies in statistical compilation programs, including:
  - Digitized administrative data
  - Primary data from the private sector
- Transforming such data into primary data to produce macroeconomic statistics (and policy-relevant climate indicators) requires a specific set of technologies and skills.
- Traditional surveys are costly and response rates are falling.
- Demand for capacity development (CD) to explore use of Big Data for official statistics in LLMICs is expected to grow.

*Source: IMF Statistics Department presentation — JUNE 19, 2024.*

### A Big Data Center to support IMF membership
- Objectives:
  - Support responsible and impactful use of Big Data.
  - Position the Center as an internationally renowned Centre.
  - Support statistical innovation for the IMF and its membership.

*Source: IMF Statistics Department presentation — JUNE 19, 2024.*

### Role of D4D-II in the use of Big Data
- D4D-II would assist countries in integrating Big Data into official statistics programs.
- Integration of Big Data sources into tailored CD will support both the macroeconomic and environmental and climate change statistics workstreams.

Module-level activities described:
- MODULE 1 — Addressing Data Needs to Empower Policymaking
  - Support countries in processing available Big Data sources to produce high-frequency indicators.
  - The better use of and increased access to administrative data and Big Data to address source data gaps.
- MODULE 5 — Environmental and Climate Change Statistics
  - Help LLMICs to use Big Data including to obtain estimates of relevant macro critical indicators.
  - Big Data sources will also be harnessed to compile climate indicators for several purposes including to support the IMF Resilience and Sustainability Trust.

*Source: IMF Statistics Department presentation — JUNE 19, 2024.*

### Examples of Big Data applications promoted by D4D-II
- MODULE 1 — Automatic Identification System (AIS) Data and Trade Nowcasting
  - AIS Data (on vessel traffic) have been used in tracking international trade in real time.
  - PortWatch aims to release AIS-derived real-time trade estimates to monitor disruptions from shocks, assessing their regional and global spillovers through port-to-port linkages.
- MODULE 5 — Physical risk indicators and global data sets
  - Physical risk indicators combine information on hazards, exposures, and vulnerability to help policymakers better understand the risk climate hazards pose to populations and the economy.
  - Global data sets can be combined with national and regional data on specific vulnerabilities to support designing appropriate adaptation policy and allocating investments in resilient infrastructure where needed.

*Source: IMF Statistics Department presentation — JUNE 19, 2024.*

### Pilot workshop: “Big Data for Macroeconomic Statistics” for African countries
- Jointly organized by AfDB Data Innovation Lab and the IMF Big Data Center.
- Contents will include:
  - Key technologies and advanced geospatial analysis techniques and methodologies.
  - AIS data for trade nowcasting (based on the IMF PortWatch platform).
  - NO2, nightlights, vegetation indices, and Google Trends for GDP nowcasting.
  - Advanced natural language processing tools and machine learning models to analyze text data from social media, forums, and customer feedback.
- Participants will have the opportunity to present and share knowledge about their use cases with Big Data sources.
- Participation details:
  - 30 representatives of statistics agencies and central banks from 14 African countries will be invited to attend the workshop.
  - September 23-27, 2024, in Kigali, Rwanda.

---


_Source: https://www.imf.org/-/media/files/capacity-developement/d4d/june-2024/04-d4d-big-data.pdf_