## Section 3 — Econometric methodology of the granular instrumental variable approach

## Source details

**Canonical URL:** [Section 3 — Econometric methodology of the granular instrumental variable approach](https://www.imf.org/-/media/files/publications/wp/2024/english/wpiea2024077-print-pdf.pdf)

## Other formats

- [Markdown version](/-/media/files/publications/wp/2024/english/wpiea2024077-print-pdf.pdf.md)
- [Structured JSON version](/-/media/files/publications/wp/2024/english/wpiea2024077-print-pdf.pdf.json)

---

### A New Data-Set
- Sample coverage and scope:
  - Annual data from 1960 to 2021 for 20 agricultural, energy, and mineral commodities.
  - Sample selection follows Alvarez et al. (2023) criteria (top 10 most traded commodities by USD value of exports 2019, BACI data) with additions from US/UK critical minerals lists and palm oil; exclusions due to data availability or market segmentation (silicon, sunflower seeds, tobacco, titanium, natural gas).
  - Added commodities for food importance: bananas, bovine, tea, and cereals (cereals is the calorie-weighted average of wheat, maize, soybeans, and rice based on global production).

- Commodities in the sample:
  - Food and beverages: Bananas, Bovine, Cocoa, Coffee, Maize, Palm Oil, Rice, Soybeans, Sugar, Tea, Wheat, and Cereals.
  - Raw agricultural materials: Cotton and Rubber (natural).
  - Energy: Crude oil and Coal.
  - Minerals: Aluminum, Copper, Lead, Tin, and Zinc.

- Data sources:
  - Agricultural production and consumption: Food and Agricultural Organization (FAO, 2023).
  - Crude oil and coal: International Energy Agency (IEA, 2024).
  - Refined consumption of aluminum, copper, lead, tin, zinc: Stuermer (2017) until 1994, spliced to World Bureau of Metals Statistics (WBMS, 2024) for 1995–2021.
  - Production: British Geological Survey (2023) for aluminum, lead (refined), copper (mined) for 1960–2021; Bems et al. (2023) for tin and zinc 1960–1994, spliced to WBMS (2024) for 1995–2021.
  - Price data: World Bank (2024) and Schwerhoff and Stuermer (2019).
  - Inflation adjustment: US consumer price index from US Bureau of Labor Statistics (2024).

- Data quality screening algorithm (applied to agricultural and energy commodities; mineral series checked by hand; crude oil exempt from criterion 4):
  1. All observations are larger than zero in levels.
  2. Log changes of all observations are within the 10th and the 90th percentile of the distribution.
  3. Less than 20 zero entries in log changes.
  4. The country is above the 25th percentile in terms of its volume of consumption (or production).

### Methodology
- Goal:
  - Estimate supply and demand elasticities using the granular instrumental variable (GIV) method (Gabaix and Koijen, 2020), exploiting country-specific idiosyncratic shocks to production and consumption as exogenous instruments.

- Model setup (in log-differences, deviations from steady state):
  - Demand equation: y^d_it = φ^d p_t + λ_i η^d_t + u^d_it, with factor decomposition λ^d_i η^d_t = Σ_{f=1}^r λ^d,f_i η^d,f_t.
  - Supply equation: y^s_it = φ^s p_t + λ_i η^s_t + u^s_it, with factor decomposition λ^s_i η^s_t = Σ_{f=1}^r λ^s,f_i η^s,f_t.
  - Idiosyncratic shocks u^d_it ∼ N(0, σ^2_{d,u}) and u^s_it ∼ N(0, σ^2_{s,u}), assumed country-specific and mutually independent.
  - Special case: single year fixed effect implies λ^d_i η^d_t = η^d_t and λ^s_i η^s_t = η^s_t.

- Construction of granular instruments (GIV algorithm):
  1. Panel regression for consumption (i ∈ I_d) and production (i ∈ I_s): y^d_it = α_i + δ_t + ε_it; save residuals ε̂_it. Time fixed effects δ_t capture price and other common factors.
  2. Extract country-specific components from ε̂_it following Bai (2009): ε̂_it = Λ F_t + u_it. Save residuals û_it and estimated factors F̂_t. (STATA module "regife" by Gomez (2021) used for interactive fixed effects estimation.)
  3. Construct share-weighted average of estimated idiosyncratic shocks:
     - z^k_t = Σ_{i=1}^{I_k} ω^k_i û^k_it with k ∈ {d, s}, where ω^k_i is the time-invariant share of country i in total production (consumption) over the sample.
     - If one common factor with homogeneous loadings (year fixed effect), instrument reduces to z^k_t = Σ_{i=1}^{I_k} [ω^k_i − 1/I_k] y^k_it.

- Instrument variations:
  - Use consumption-based GIV z^d_t, production-based GIV z^s_t, and their difference z^d_t − z^s_t (preferred per Gabaix and Koijen (2020)).
  - Create three variations of each instrument: (a) single time fixed effect (skip step 2), (b) one common factor with heterogeneous loadings, (c) two common factors with heterogeneous loadings.
  - Total of nine GIVs in total for each commodity.

- Regression analysis and identification:
  - Define unweighted average: y^k_Et = (1/I_k) Σ_{i=1}^{I_k} y^k_it.
  - Estimate local projections with IVs for horizons h = 0, 1, 2, ..., 5:
    - y^k_{E,t+h} = δ^k_h(L) y^k_Et + β^k_h(L) p_t + φ^k_h p_t + ε^k,IV_{i,t+h}, where y^k_{E,t+h} ≡ ln(Y^k_{i,t+h}) − ln(Y^k_{i,t−1}), δ_h(L) and β_h(L) are polynomials in lag operator L = 5.
  - Instrument p_t with contemporaneous and lagged GIVs: pair {z^k_t, L.z^k_t} — nine such pairs per commodity per side.
  - Instrument selection:
    - Choose instrument pair with the strongest first-stage measured by F-score, conditional on the 2nd stage elasticity having the expected sign; if wrong sign, move to next-best IV pair.

### Granularity in Commodity Markets
- Market concentration and identification:
  - Identification requires granularity, manifesting as high market concentration measured by the Herfindahl-Hirschman Index (HHI) for production and consumption.
  - Example: palm oil production concentrated in Indonesia with production HHI = 0.4, roughly 80 times higher than the HHI if all 195 countries had equal market shares; implies idiosyncratic shock in Indonesia likely affects global prices and can serve as an instrument.

- Common vs idiosyncratic factors (Bai (2009) decomposition of panel residuals):
  - Common factors generally more relevant than idiosyncratic factors in driving fluctuations for both commodity-consumption and commodity-production on average across commodities.
  - Possible explanation: global supply chains transmit shocks as common factors on the production side (e.g., shipping, trade networks).
  - Common factors have particularly increased in industrial commodities production over the last decade.
  - Common factors across countries have become more important for consumption over time (food and industrial commodities), possibly due to increased synchronization of the global business cycle.
  - For food commodities, idiosyncratic shocks are more relevant for production than for consumption; agricultural production is more exposed to country-specific events (droughts, floods, temperature anomalies, pests).

### Results
- Main stylized finding: Commodities are mostly inelastic.

- Supply side findings:
  - Metals, especially copper and zinc, tend to have the lowest one-year supply elasticities (close to zero).
  - Agricultural commodities generally have the highest one-year supply elasticities.
  - Example: Cereals supply elasticity ~ 0.6 → a 10 percent increase in prices raises output by 6 percent within a year.
  - Distinction within agriculture:
    - Perennial crops (coffee, cocoa, palm oil, rubber) have smaller supply elasticities due to multi-year production lags (palm oil trees, coffee trees, cocoa trees typically take at least two, three, and five years respectively to bear fruit).
  - Energy commodities (crude oil, coal) have supply elasticities between minerals and agricultural commodities.

- Demand side findings:
  - Demand elasticities vary by commodity characteristics rather than commodity type.
  - Rice: price elasticity of demand close to zero; likely reflects that only about 10 percent of rice production is traded internationally and rice prices are typically subsidized.
  - Elasticities above 0.4 for tea, cotton, and wheat.
  - Crude oil and coal: demand elasticities below 0.2.
  - Copper and zinc: demand elasticities close to zero.
  - Lead and tin: demand elasticities between 0.2 and 0.3.

- Estimation context and outputs:
  - Impulse response functions (IRFs) show change in quantity supplied (blue) or demanded (red) due to a 1 percent increase in prices over years 0–5; 90 percent confidence bands shown.
  - Table 2 (annex) provides detailed local projection estimates at different horizons and indicates whether baseline specification used consumption-based IV, production-based IV, or the difference IV.

- Comparison with literature:
  - Results are consistent with prior ranges summarized by Fally and Sayre (2018) for many commodities; half of the estimates are new commodities in the literature.
  - Notable differences:
    - Higher supply elasticity for coal relative to prior literature.
    - Rice short-term demand elasticity not statistically different from zero and at the upper bound of Fally and Sayre (2018)’s range.
    - Soybean short-term demand point-estimate higher than literature.
    - Wheat elasticity toward the low-end of the literature range.
    - Long-run copper supply elasticity within literature range; copper demand appears more elastic than previous estimates.

- Dynamics over horizons:
  - Commodities tend to become more responsive as markets adjust over multi-year horizons.
  - Heterogeneity across commodity types:
    - Agricultural commodities: supply responses relatively flat over five years for many crops; some show significant peaks 2–3 years after shocks (notably perennial multi-year crops: rubber, coffee, cocoa).
    - Metals and energy: supply elasticities generally upward sloping over the horizon, significant particularly for copper.
    - Demand multipliers: generally imprecisely estimated; metals show the largest increases in multipliers over the horizon; most agricultural commodities do not show larger demand multipliers over time.
  - Overall patterns:
    - Demand and supply for agricultural goods are generally more responsive to shocks than minerals and energy commodities.
    - Agricultural commodities display smaller price volatility compared to metals and energy commodities.
    - Agricultural commodities see the least increase in responsiveness after a couple of years; mineral commodities show strong increases in responsiveness over time.

- Commodity price volatility (period comparison):
  - Volatility is the standard deviation of log-differences in monthly prices over respective periods.
  - Price indices referenced: Base metal, Iron ore, Food, Cereal, Wheat, Coal, Natural gas; crude oil = IMF average petroleum spot price.
  - Periods compared: 1990–2019 versus 2020–2023 (figure indicates increased volatility for some series in 2020–2023 relative to 1990–2019).

### 5.3 Comparison to Other Identification Methods
- Methodological comparison:
  - Caldara et al. (2019) identify monthly oil supply shocks at the global level and then attribute them to outages in individual countries; the GIV approach in this paper identifies shocks at the annual frequency and at the country-level.
  - Major differences arise in the frequency and scope of episodes between the two approaches.

- Key comparative findings:
  - When monthly supply shocks identified by Caldara et al. (2019) lead to a supply shock at the annual frequency, the narrative shocks are broadly similar to those identified using the GIV.
  - Example — U.S.A., September 2005 (hurricane Rita):
    - Actual decline (monthly): -18.9 percent
    - Narrative approach (global negative supply shock, Caldara et al. (2019)): -1.3 percent
    - Actual change (annual, our data): -5.2 percent
    - GIV approach (idiosyncratic residual, annual): -5.3 percent
  - Example — Iran, 1987 (war with Iraq):
    - Actual decline (September): -22 percent
    - Annual average in 1987: increased by 12 percent
    - GIV picks up the annual increase as an idiosyncratic shock
  - Some countries/episodes are not considered in the GIV algorithm due to data inconsistency over the entire sample length (example: Ecuador in 1987).

- Sources of differences between instruments:
  - Differences often relate to the temporary nature (at most two months) and subsequent rebound of monthly declines, which may not translate into annual declines. Illustrative episodes of temporary shocks with later rebound: Iran Jan 1985, Nigeria Jun 1985, Qatar Apr 1986, UAE Aug 1990.
  - Some differences stem from a drop after a spike in production (examples: Saudi Arabia Sept 1986, Iran Sept 1987, UAE Jan 1988).
  - Caldara et al. (2019) also compare their shock series with those identified in Kilian (2008); the comparison shows notable differences and indicates that the literature is not settled on shock identification.

- Aggregate assessment:
  - Overall, the results show that the GIV method identifies similar shocks to the narrative approach when data are comparable at the monthly and annual frequency.

*Source — wpiea2024077-print-pdf - Section 3 presents the econometric methodology of the granular instrumental variable ap-*

### Section 3 presents the econometric methodology of the granular instrumental variable ap-

### wpiea2024077-print-pdf - Section 3 presents the econometric methodology of the granular instrumental variable ap-

### A New Data-Set
- Sample coverage and scope:
  - Annual data from 1960 to 2021 for 20 agricultural, energy, and mineral commodities.
  - The sample selection follows Alvarez et al. (2023) criteria (top 10 most traded commodities by USD value of exports 2019, BACI data) with additions from US/UK critical minerals lists and palm oil; exclusions due to data availability or market segmentation (silicon, sunflower seeds, tobacco, titanium, natural gas).
  - Added commodities for food importance: bananas, bovine, tea, and cereals (cereals is the calorie-weighted average of wheat, maize, soybeans, and rice based on global production).

- Commodities in the sample:
  - Food and beverages: Bananas, Bovine, Cocoa, Coffee, Maize, Palm Oil, Rice, Soybeans, Sugar, Tea, Wheat, and Cereals.
  - Raw agricultural materials: Cotton and Rubber (natural).
  - Energy: Crude oil and Coal.
  - Minerals: Aluminum, Copper, Lead, Tin, and Zinc.

- Data sources:
  - Agricultural production and consumption: Food and Agricultural Organization (FAO, 2023).
  - Crude oil and coal: International Energy Agency (IEA, 2024).
  - Refined consumption of aluminum, copper, lead, tin, zinc: Stuermer (2017) until 1994, spliced to World Bureau of Metals Statistics (WBMS, 2024) for 1995–2021.
  - Production: British Geological Survey (2023) for aluminum, lead (refined), copper (mined) for 1960–2021; Bems et al. (2023) for tin and zinc 1960–1994, spliced to WBMS (2024) for 1995–2021.
  - Price data: World Bank (2024) and Schwerhoff and Stuermer (2019).
  - Inflation adjustment: US consumer price index from US Bureau of Labor Statistics (2024).

- Data quality screening algorithm (applied to agricultural and energy commodities; mineral series checked by hand; crude oil exempt from criterion 4):
  1. All observations are larger than zero in levels.
  2. Log changes of all observations are within the 10th and the 90th percentile of the distribution.
  3. Less than 20 zero entries in log changes.
  4. The country is above the 25th percentile in terms of its volume of consumption (or production).

### Methodology
- Goal: Estimate supply and demand elasticities using the granular instrumental variable (GIV) method (Gabaix and Koijen, 2020), exploiting country-specific idiosyncratic shocks to production and consumption as exogenous instruments.

- Model setup (supply and demand in log-differences, deviations from steady state):
  - Demand equation: y^d_it = φ^d p_t + λ_i η^d_t + u^d_it, with factor decomposition λ^d_i η^d_t = Σ_{f=1}^r λ^d,f_i η^d,f_t.
  - Supply equation: y^s_it = φ^s p_t + λ_i η^s_t + u^s_it, with factor decomposition λ^s_i η^s_t = Σ_{f=1}^r λ^s,f_i η^s,f_t.
  - Idiosyncratic shocks u^d_it ∼ N(0, σ^2_{d,u}) and u^s_it ∼ N(0, σ^2_{s,u}), assumed country-specific and mutually independent.
  - Special case: single year fixed effect implies λ^d_i η^d_t = η^d_t and λ^s_i η^s_t = η^s_t.

- Construction of granular instruments (GIV algorithm):
  1. Panel regression for consumption (i ∈ I_d) and production (i ∈ I_s): y^d_it = α_i + δ_t + ε_it; save residuals ε̂_it. Time fixed effects δ_t capture price and other common factors.
  2. Extract country-specific components from ε̂_it following Bai (2009): ε̂_it = Λ F_t + u_it. Save residuals û_it and estimated factors F̂_t. (STATA module "regife" by Gomez (2021) used for interactive fixed effects estimation.)
  3. Construct share-weighted average of estimated idiosyncratic shocks:
     - z^k_t = Σ_{i=1}^{I_k} ω^k_i û^k_it with k ∈ {d, s}, where ω^k_i is the time-invariant share of country i in total production (consumption) over the sample.
     - If one common factor with homogeneous loadings (year fixed effect), instrument reduces to z^k_t = Σ_{i=1}^{I_k} [ω^k_i − 1/I_k] y^k_it.
  - Instrument variations:
    - Use consumption-based GIV z^d_t, production-based GIV z^s_t, and their difference z^d_t − z^s_t (preferred per Gabaix and Koijen (2020)).
    - Create three variations of each instrument: (a) single time fixed effect (skip step 2), (b) one common factor with heterogeneous loadings, (c) two common factors with heterogeneous loadings.
    - Total of nine GIVs in total for each commodity.

- Regression analysis and identification:
  - Define unweighted average: y^k_Et = (1/I_k) Σ_{i=1}^{I_k} y^k_it.
  - Estimate local projections with IVs for horizons h = 0, 1, 2, ..., 5:
    - y^k_{E,t+h} = δ^k_h(L) y^k_Et + β^k_h(L) p_t + φ^k_h p_t + ε^k,IV_{i,t+h}, where y^k_{E,t+h} ≡ ln(Y^k_{i,t+h}) − ln(Y^k_{i,t−1}), δ_h(L) and β_h(L) are polynomials in lag operator L = 5.
  - Instrument p_t with contemporaneous and lagged GIVs: pair {z^k_t, L.z^k_t} — nine such pairs per commodity per side.
  - Instrument selection: choose instrument pair with the strongest first-stage measured by F-score, conditional on the 2nd stage elasticity having the expected sign; if wrong sign, move to next-best IV pair.

### Granularity in Commodity Markets
- Market concentration and identification:
  - Identification requires granularity, manifesting as high market concentration measured by the Herfindahl-Hirschman Index (HHI) for production and consumption.
  - Example: palm oil production concentrated in Indonesia with production HHI = 0.4, roughly 80 times higher than the HHI if all 195 countries had equal market shares; implies idiosyncratic shock in Indonesia likely affects global prices and can serve as an instrument.

- Common vs idiosyncratic factors (based on Bai (2009) decomposition of panel residuals):
  - Common factors generally more relevant than idiosyncratic factors in driving fluctuations for both commodity-consumption and commodity-production on average across commodities.
  - Possible explanation: global supply chains transmit shocks as common factors on the production side (e.g., shipping, trade networks).
  - Common factors have particularly increased in industrial commodities production over the last decade.
  - Common factors across countries have become more important for consumption over time (food and industrial commodities), possibly due to increased synchronization of the global business cycle.
  - For food commodities, idiosyncratic shocks are more relevant for production than for consumption; agricultural production is more exposed to country-specific events (droughts, floods, temperature anomalies, pests).

### Results

- Main stylized finding: Commodities are mostly inelastic
  - Supply side:
    - Metals, especially copper and zinc, tend to have the lowest one-year supply elasticities (close to zero).
    - Agricultural commodities generally have the highest one-year supply elasticities.
    - Example: Cereals supply elasticity ~ 0.6 → a 10 percent increase in prices raises output by 6 percent within a year.
    - Distinction within agriculture:
      - Perennial crops (coffee, cocoa, palm oil, rubber) have smaller supply elasticities due to multi-year production lags (palm oil trees, coffee trees, cocoa trees typically take at least two, three, and five years respectively to bear fruit).
    - Energy commodities (crude oil, coal) have supply elasticities between minerals and agricultural commodities.
  - Demand side:
    - Demand elasticities vary by commodity characteristics rather than commodity type.
    - Rice: price elasticity of demand close to zero; likely reflects that only about 10 percent of rice production is traded internationally and rice prices are typically subsidized.
    - Elasticities above 0.4 for tea, cotton, and wheat.
    - Crude oil and coal: demand elasticities below 0.2.
    - Copper and zinc: demand elasticities close to zero.
    - Lead and tin: demand elasticities between 0.2 and 0.3.
  - Estimation context:
    - Impulse response functions (IRFs) show change in quantity supplied (blue) or demanded (red) due to a 1 percent increase in prices over years 0–5; 90 percent confidence bands shown.
    - Table 2 (annex) provides detailed local projection estimates at different horizons and indicates whether baseline specification used consumption-based IV, production-based IV, or the difference IV.

- Comparison with literature:
  - Results are consistent with prior ranges summarized by Fally and Sayre (2018) for many commodities; half of the estimates are new commodities in the literature.
  - Differences noted:
    - Higher supply elasticity for coal relative to prior literature.
    - Rice short-term demand elasticity not statistically different from zero and at the upper bound of Fally and Sayre (2018)’s range.
    - Soybean short-term demand point-estimate higher than literature.
    - Wheat elasticity toward the low-end of the literature range.
    - Long-run copper supply elasticity within literature range; copper demand appears more elastic than previous estimates.

- Dynamics: Supply and demand become more responsive over time
  - Commodities tend to become more responsive as markets adjust over multi-year horizons.
  - Heterogeneity across commodity types:
    - Agricultural commodities: supply responses relatively flat over five years for many crops; some show significant peaks 2–3 years after shocks (notably perennial multi-year crops: rubber, coffee, cocoa).
    - Metals and energy: supply elasticities generally upward sloping over the horizon, significant particularly for copper.
    - Demand multipliers: generally imprecisely estimated; metals show the largest increases in multipliers over the horizon; most agricultural commodities do not show larger demand multipliers over time.
  - Overall patterns:
    - Demand and supply for agricultural goods are generally more responsive to shocks than minerals and energy commodities.
    - Agricultural commodities display smaller price volatility compared to metals and energy commodities.
    - Agricultural commodities see the least increase in responsiveness after a couple of years; mineral commodities show strong increases in responsiveness over time.

- Commodity price volatility (period comparison):
  - Volatility is the standard deviation of log-differences in monthly prices over respective periods.
  - Price indices referenced: Base metal, Iron ore, Food, Cereal, Wheat, Coal, Natural gas; crude oil = IMF average petroleum spot price.
  - Periods compared: 1990–2019 versus 2020–2023 (figure indicates increased volatility for some series in 2020–2023 relative to 1990–2019).

*Italic: Source — wpiea2024077-print-pdf - Section 3 presents the econometric methodology of the granular instrumental variable ap-*

### 5.3    Comparison to Other Identification Methods

### 5.3    Comparison to Other Identification Methods

### Methodological comparison
- Caldara et al. (2019) identify monthly oil supply shocks at the global level and then attribute them to outages in individual countries; the GIV approach in this paper identifies shocks at the annual frequency and at the country-level.
- Major differences arise in the frequency and scope of episodes between the two approaches.

### Key findings from the comparison
- The comparison is reassuring: when monthly supply shocks identified by Caldara et al. (2019) lead to a supply shock at the annual frequency, the narrative shocks are broadly similar to those identified using the GIV.
- Example — U.S.A., September 2005 (hurricane Rita):
  - Actual decline (monthly): -18.9 percent
  - Narrative approach (global negative supply shock, Caldara et al. (2019)): -1.3 percent
  - Actual change (annual, our data): -5.2 percent
  - GIV approach (idiosyncratic residual, annual): -5.3 percent
- Example — Iran, 1987 (war with Iraq):
  - Actual decline (September): -22 percent
  - Annual average in 1987: increased by 12 percent
  - GIV picks up the annual increase as an idiosyncratic shock
- Some countries/episodes are not considered in the GIV algorithm due to data inconsistency over the entire sample length (example: Ecuador in 1987).

### Sources of differences between instruments
- Differences often relate to the temporary nature (at most two months) and subsequent rebound of monthly declines, which may not translate into annual declines. Illustrative episodes of temporary shocks with later rebound: Iran Jan 1985, Nigeria Jun 1985, Qatar Apr 1986, UAE Aug 1990.
- Some differences stem from a drop after a spike in production (examples: Saudi Arabia Sept 1986, Iran Sept 1987, UAE Jan 1988).
- Caldara et al. (2019) also compare their shock series with those identified in Kilian (2008); the comparison shows notable differences and indicates that the literature is not settled on shock identification.

### Aggregate assessment
- Overall, the results show that the GIV method identifies similar shocks to the narrative approach when data are comparable at the monthly and annual frequency.

*Source: Section 5.3, "Comparison to Other Identification Methods," from the provided IMF working paper content.*

---


_Source: https://www.imf.org/-/media/files/publications/wp/2024/english/wpiea2024077-print-pdf.pdf_