Alfred Kammer's Remarks at the ECB Forum: "Lessons from recent experiences in macroeconomic forecasting"
IMF News, June 28, 2023
Source details
- Canonical URL
- Alfred Kammer's Remarks at the ECB Forum: "Lessons from recent experiences in macroeconomic forecasting"
Other formats
Bibliographic details
- Published: June 28, 2023
The challenge of macroeconomic forecasting
- Macroeconomic forecasting is difficult because economies are "complex social systems."
- Forecast accuracy:
- Short horizons (same year mainly) "tend to do reasonably well."
- Moving out to "a 1.5 to two-year horizon both our and others’ forecasts deteriorate very fast (Celasun et al., 2021)."
- Higher GDP volatility in a country → larger forecast errors.
- "Once we control for GDP growth volatility, our forecasts for advanced economies are no better than those of low-income countries."
- Main WEO forecasting approach:
- The World Economic Outlook (WEO) is "published quarterly" and is a bottom-up exercise aggregating forecasts from "190 country teams."
- Country teams use common assumptions for key global variables (commodity prices, benchmark interest rates) but retain substantial discretion on how much economies are affected by common factors and on embedding country-specific information and tools.
- Empirical finding on common vs idiosyncratic factors:
- Domestic factors tend to be "more important than foreign ones" on average.
- Ex-post evaluations find that "common components are not sufficiently taken into account."
- Celasun et al. (2021) find that growth projections for large economies and countries' own terms of trade forecasts can help predict growth forecast errors for a significant share of countries—implying inefficiencies in using available information.
Adaptations during the pandemic and energy crisis
- Recent shocks:
- Pandemic and the "massive volatility in energy markets since the Russian invasion of Ukraine" disrupted standard macro relationships (e.g., Okun’s law; the relationship between household disposable income and consumption).
- Gas prices "decoupled significantly from oil prices when Russia cut gas flows to Europe" in 2022.
- Forecast errors during large common shocks:
- During the GFC and pandemic, errors were "very large and one-sided," indicating dominance of common shocks.
- Spring 2020 centralized approach to the pandemic:
- Three immediate challenges in March 2020: infections clustered in few countries; uncertain severity of mobility restrictions; little historical data on activity response.
- IMF Research Department engaged with epidemiologists and public health experts and provided centralized guidance on "how many effective working days would be lost in the coming quarters."
- Country teams translated lost days into GDP declines depending on economic structures (contact-intensive share), added international spillovers and offsetting policy support.
- Result: the "same year" forecast made in the Spring of 2020 for the euro area was "off only by about 1 percentage point."
- Inflation underestimation over the past "1.5 years":
- Even after shocks materialized, quantifying their effects on inflation was challenging; "forecast errors were very large."
- Two specific model shortcomings and responses:
- Granularity: using international oil prices as a proxy for overall energy prices failed when gas decoupled from oil; IMF now "project inflation in a much more disaggregated way" (see McGregor and Toscani (2022)). A re-specified Phillips-curve based inflation projection estimated until 2019 produces "significantly better out of sample forecasts" but still "misses the most recent inflation surge by a meaningful margin."
- Non-linearities: models trained on historical data struggled to capture very large input price movements (e.g., a "500 percent increase"); such nonlinear responses are "challenging to gauge in real time."
Lessons learned and forward-looking guidance
- Three guiding principles for IMF forecasting going forward (as summarized by Kammer):
- Be balanced:
- "Emphasize top-down forecasting approaches when common forces are strong, but allow for idiosyncrasies as well."
- Enhanced role for top-down guidance is likely to persist post-pandemic, while retaining room for country-specific factors.
- Be nimble:
- "Be ready to continuously develop and add new tools."
- Monitor and enhance tools, exploit underused data sources and incorporate new data, "including big data in a flexible way."
- Be modest:
- Focus on avoiding misses that would cause "major policy mistakes" rather than marginal variation around the modal forecast.
- Acknowledge that macroeconomic data (notably GDP) are "prone to large revisions."
- "De-emphasizing point estimates and focusing on forecast ranges" is important when forecasting informs policy.
- Role of scenarios and cross-checks:
- Scenario analysis is important to implement policies to avoid downside scenarios and for contingency planning.
- Use multiple tools jointly and cross-check results.
- Example: in addition to disaggregated inflation components, the IMF now examines prices from the "income" side—wage growth, import prices, and profit share behavior consistent with projections (see Hansen et al., 2023).
- Current key uncertainties and cautions:
- With inflation having peaked, there is uncertainty whether model disinflation paths are trustworthy given dissipation of supply shocks and monetary tightening.
- Uncertainty on the level of slack in the economy; many standard relationships "are not yet fully normalized"; wage and price inflation exceed the estimation range of models.
- Historical tendency: "we tend to overcompensate for large overpredictions of growth by then turning overly pessimistic"—so avoid turning overly pessimistic about disinflation paths.
- Forecasters’ active role:
- An ongoing interaction between existing tools, new/refined tools, and data sources is needed to bring in considerations not embedded in models.
- Concrete empirical and methodological points reiterated:
- Forecast horizons: "same year" vs "a 1.5 to two-year horizon" performance differences.
- WEO process: "190 country teams" preparing forecasts under common assumptions.
- Pandemic forecasting benchmark: Spring 2020 euro area "off only by about 1 percentage point."
- Pseudo out-of-sample inflation forecast window referenced: "2020q1 until 2023q1."
- Recent model literature cited for disaggregated inflation and income-side checks: McGregor and Toscani (2022); Hansen et al., 2023.
Alfred Kammer, Remarks at the ECB Forum on Central Banking, June 28, 2023.
Content in this bundle
- Lessons from Recent Experiences in Macroeconomic Forecasting