## wpiea2022052-print-pdf

## Source details

**Canonical URL:** [wpiea2022052-print-pdf](https://www.imf.org/-/media/files/publications/wp/2022/english/wpiea2022052-print-pdf.pdf)

## Other formats

- [Markdown version](/-/media/files/publications/wp/2022/english/wpiea2022052-print-pdf.pdf.md)
- [Structured JSON version](/-/media/files/publications/wp/2022/english/wpiea2022052-print-pdf.pdf.json)

---

### Introduction: motivation, contribution, and structure
- Need for high-frequency indicators
  - The Covid crisis highlighted the need for a systematic analysis of high-frequency indicators of economic activity.
  - Example: in 2020 the economy in Europe recorded in the second quarter its largest contraction since World War II, while the first official GDP growth estimates of the second quarter for European countries were published only in mid-July—too late to assess the magnitude of the shock and calibrate policy responses.
  - Core question: whether a variety of higher-frequency data from different sources could be systematically analyzed to provide sufficiently accurate and timely information to inform policymaking.
- Data availability and novel indicators
  - New information from non-official sources (market surveys, sentiment indicators, Google search results, air quality, etc.) can quickly register sharp changes and supplement standard official indicators.
  - The paper compiles datasets of both standard variables and new ones, including Google search and air quality, and constructs country-specific databases for selected European economies.
- Modeling advances and methods
  - Dynamic factor models (DFM) have been prominent nowcasting tools for 20+ years.
  - Machine learning (ML) methods have recently been deployed alongside DFMs for nowcasting.
  - The paper applies standard DFMs and several ML algorithms to nowcast GDP growth and compares performance across heterogeneous European economies during normal and crisis times.
- Contribution
  - Motivates and compiles datasets of standard and novel variables (e.g., Google search and air quality), including country-specific databases and transformations.
  - Employs DFMs and several ML algorithms to nowcast GDP growth.
  - Compares performance of alternative methods across a heterogeneous group of European economies, during normal and crisis episodes.
- Paper structure (parts)
  - Part 2: literature overview focusing on DFMs and machine learning methods.
  - Part 3: discussion of data and statistical treatment.
  - Part 4: non-technical summary of DFMs and selected ML approaches.
  - Part 5: main results and comparisons, including during normal and crisis episodes.
  - Part 6: conclusion.

### Data selection, clusters, and transformations
- Data scope and exclusions
  - Datasets combine official and non-official, hard and soft, and proxy data published at different frequencies.
  - The dataset excludes detailed sectoral data (uses only headline figures) and variables available only since 2020.
  - Google Trends, Oxford Stringency Indices and air quality data are the only high (weekly) frequency variables included.
  - Total number of variables ranges between 20 to 60 depending on the economy and data availability.
- Data clusters included
  - Firm and production data; Surveys and forward-looking indicators; Labor market; Financial variables; Prices; Foreign trade; External environment; Economy-specific variables; Additional indicators (air quality index, hotel occupancy, Oxford Stringency Indices, Google search frequency).
- Transformations before modeling
  - All variables with frequency higher than quarterly (mostly weekly and monthly) are converted to quarterly for ML models; DFMs exploit mixed-frequency data via bridge equations and Kalman filter.
  - Time series are seasonally adjusted and detrended; many quarterly variables are converted to year-over-year growth rates; some variables (e.g., unemployment rate, survey outcomes) are kept in levels or transformed to year-over-year differences.
- Appendix 1: country specific databases and transformation indicators for each variable.

### Dynamic Factor Models (DFMs): formulation and estimation
- Core idea
  - A large number of economic variables (N) are described by a small number of common factors (r), with r < N.
- Measurement equation
  - yi,t = λi,1 f1,t + · · · + λi,r fr,t + ei,t, for i = 1, …, n
  - Common component: Σr j=1 λi,j fj,t; ei,t captures idiosyncratic movements.
- Variable reduction approaches
  - (i) naïve: extract top factors (largest eigenvectors) to jointly explain about 80 percent of variation.
  - (ii) structured: split dataset into clusters using prior expertise and extract one factor per cluster.
- State-space representation and dynamics
  - Factors: fj,t = aj fj,t−1 + uj,t,   Uj,t ~ N(0, σu,j2)
  - Idiosyncratic components: ei,t = ρi ei,t−1 + εi,t,   εi,t ~ N(0, σε,t2)
  - Measurement equation + transition equations form the state space model.
- Estimation
  - Parameters estimated iteratively using Kalman smoother and expectation maximization (EM) algorithm.
  - Initialization:
    - Step 1: EM computes principal components; estimate parameters by OLS treating principal components as true factors.
    - Step 2: update factor estimates via Kalman smoother.
    - Iterate until convergence to obtain maximum likelihood estimates.
- Methodological lineage
  - Follows NY Fed block structure modeling and estimation framework and EM algorithm approach cited.

### Machine Learning (ML) methods: motivation, types, and limitations
- Motivation and advantages
  - Flexibility to capture complex nonlinear relationships.
  - Variable selection capabilities to limit overfitting.
  - Use of cross-validation to fine-tune hyperparameters using pseudo out-of-sample performance.
- Limitations and considerations
  - Interpretability: ML often viewed as a black box; trade-off between performance and interpretability.
  - Missing data: some ML algorithms are less robust than DFMs; toolkit uses a naïve combination of deleting rows or columns with missing data.
  - Focus: ML techniques applied in this study are for prediction only.
- Algorithms applied (brief)
  - LASSO, Ridge, Elastic Net: regularized linear regressions to avoid overfitting; face linearity limitations.
  - Support Vector Machine (SVM): kernel-based hyperplane methods; performance depends on kernel and regularization.
  - Random Forest (RF): ensemble of regression trees; flexible but limited at predicting extreme/outlier events.
  - Neural Network (NN): multi-layer nonlinear mapping; highly flexible but significantly limits interpretability.

### Assessment of predictive performance: protocol and metrics
- Backtesting protocol (pseudo-real-time)
  - In each period t: explanatory variables available up to quarter t; target variable available only up to t−1.
  - Estimate model parameters using data up to t−1, produce one-step-ahead prediction for t, compute forecast error vs actual.
- Predictive accuracy statistics (definitions preserved)
  - RMSE: RMSE = sqrt( (∑(yt − ŷt)2) / N )
  - MAE: MAE = (∑ |yt − ŷt|) / N
  - MDA: MDA = (1/N) ∑ 1[ssign(yt − yt−1) = ssign(ŷt − yt−1)], where 1A(x) = {1 if x ∈ A; 0 if else} and ssign(x) = {−1 if x < 0; 0 if x = 0; 1 if x > 0}
- Use of metrics
  - RMSE emphasizes large errors; MAE is less sensitive to outliers; MDA measures direction-of-change accuracy.
  - Metrics inform model aggregation weights (example: inverse RMSEs used as weights).
- Subsample analysis
  - Performance assessed over: (i) full sample, (ii) normal times (2015 to 2019Q4), and (iii) COVID-19 period (2020Q1 up to 2021Q4).

### Empirical results and key findings (summary)
- General performance claims
  - Both DFM and ML models add value, especially during times of heightened volatility.
  - Except for Ireland and Malta, the DFM and ML models outperform AR(1) in a large majority of cases on the whole sample (Table 2).
  - AR(1) performs quite well during relatively stable growth and frequently outperforms other nowcasting models (Table 3).
  - During the COVID-19 pandemic (Table 4):
    - Ridge regression and Lasso models could significantly reduce the average forecast errors ranging from 40 to 75 percent relative to the AR(1) benchmark.
    - For DFM, the forecast errors are about 50 percent lower than the AR(1) benchmark across all countries.
- Model heterogeneity across countries and periods
  - No one-size-fits-all model: best model differs across countries and time.
  - DFMs tend to perform relatively well during normal times and were among best performers for Austria, Hungary, and Portugal pre-pandemic; DFM performed worse than AR(1) for Malta and Poland pre-pandemic.
  - ML methods often performed better at capturing turning points during the COVID period (notably 2020Q2-2020Q3).
  - Example country-method performance snippets:
    - Ridge Regression and DFM perform well for Poland and Hungary, respectively, during normal times but underperform during COVID-19 period.
    - GPReg generates the lowest RMSEs for Ireland and Portugal during normal times; LinReg performs best for Poland pre-pandemic.
    - Several ML methods appeared to perform better at capturing turning points during COVID compared to both AR(1) and DFM.
- Visualization
  - Figure 1 categorizes examples of model performances as "The Good", "The (Not So) Bad", and "The Ugly".

### Operationalization: the integrated, automated nowcasting tool
- Core capabilities
  - Automatically (i) collects and treats datasets; (ii) applies a suite of DFM and ML models to perform backtests (charts and accuracy indicators); (iii) re-estimates models whenever new data are available and produces a nowcast of current quarter GDP growth; (iv) generates aggregated outputs across methods and subsamples.
  - Tool can be applied to any country once variables are selected and subject to data availability.
- Implementation details
  - Developed in Matlab; composed of three major components: Data Warehouse, Backtest, and Nowcasting Tracker.
  - Reports Generator is a Matlab script that serves as the main user interface and reads an Excel "Spec File".
- Component functions
  - Data Warehouse: fetches historical data (Haver, Google) and applies transformations specified in the Spec File; may invoke R scripts where necessary.
  - Backtest: performs out-of-sample pseudo-nowcasts on an expanding window; produces plots and accuracy metrics.
  - Nowcasting Tracker: produces current-quarter nowcasts for selected methods; saves table and plot of GDP nowcasts.
- Modularity and extensibility
  - Each nowcasting method has a standalone function/script; users can add or remove methods.
  - Standardized outputs facilitate extension by users familiar with Matlab.
- Execution flow
  - Data Warehouse fetches and transforms data → Backtest runs expanding-window out-of-sample nowcasts → Nowcasting Tracker produces current-quarter nowcasts → Reports Generator integrates outputs and reads Spec File.

### Nowcasting variables by country: inventory and transformations (overview)
- Inventories are standardized and grouped by thematic categories: National accounts; Housing and construction; Labor; Manufacturing; Retail and consumption; International trade; Financial; Surveys; Other.
- Frequency abbreviations: Q (quarterly), M (monthly), D (daily).
- Transformation labels used include: YoY percent change, Percent change, Level, Change (difference), Hp-filtered cyclical components, and others as listed per indicator.
- Country inventories (selected examples, transformations preserved exactly)
  - Austria highlights: Real gross domestic product — Q — YoY percent change; Industrial production index: construction — M — YoY percent change; Google: keywords — M — Level; Nitrogen dioxide emissions — M — Level; Apple Mobility Trends Index: walking — D — Level.
  - Hungary highlights: Real gross domestic product — Q — YoY percent change; Domestic turnover series — M — Level or YoY percent change; Google: keywords — M — Level; Nitrogen dioxide emissions — M — Level.
  - Ireland highlights: Real gross domestic product — Q — YoY percent change; Total job postings — D — Level; Stock price index, ISEQ — M — Percent change; Google: change in visits relative to baseline: retail & recreation — D — Level; Nitrogen dioxide emissions — M — Level.
  - Malta highlights: Real gross domestic product — Q — YoY percent change; Real gross capital formation — Q — Level; Google: keywords — M — YoY change (difference); Nitrogen dioxide emissions — M — YoY change (difference).
  - Poland highlights: Real gross domestic product — Q — YoY percent change; HICP: core inflation — M — YoY percent change; Poland: policy rate — M — Level; Apple Mobility Trends Index: walking — D — Level; Nitrogen dioxide emissions — M — YoY percent change.
  - Portugal highlights: Real gross domestic product — Q — YoY percent change; Oxford government response index; Oxford stringency index; Oxford risk of openness index: community understanding — D — Level; Google: keywords; Nitrogen dioxide emissions — M — Level.
- Appendix 1 presents country specific databases and indicates the transformation applied to each variable.

### Sample results — Hungary (selected backtest metrics and nowcast tracker)
- Backtest periods reported
  - 2016Q1 to 2021Q2
  - 2016Q1 to 2019Q4
  - 2020Q1 to 2021Q2
- Reported metric triples (MFE, RMSE, MDA) by model (selected rows preserved exactly)
  - DFM
    - 2016Q1 to 2021Q2: MFE -0.202, RMSE 2.893, MDA 0.81
    - 2016Q1 to 2019Q4: MFE -0.063, RMSE 0.507, MDA 0.733
    - 2020Q1 to 2021Q2: MFE -0.574, RMSE 5.477, MDA 1
  - Lasso_SEm
    - 2016Q1 to 2021Q2: MFE -0.129, RMSE 1.591, MDA 0.857
    - 2016Q1 to 2019Q4: MFE -0.132, RMSE 1.134, MDA 0.8
    - 2020Q1 to 2021Q2: MFE -0.121, RMSE 2.419, MDA 1
  - Ridge_SEm
    - 2016Q1 to 2021Q2: MFE -0.194, RMSE 1.621, MDA 0.714
    - 2016Q1 to 2019Q4: MFE -0.231, RMSE 1.109, MDA 0.6
    - 2020Q1 to 2021Q2: MFE -0.097, RMSE 2.519, MDA 1
  - AR1
    - 2016Q1 to 2021Q2: MFE 0.571, RMSE 5.374, MDA 0.714
    - 2016Q1 to 2019Q4: MFE 0.204, RMSE 0.862, MDA 0.667
    - 2020Q1 to 2021Q2: MFE 1.547, RMSE 10.193, MDA 0.8
  - Selected ML and NN examples (preserved)
    - GPReg: 2016Q1 to 2021Q2: MFE 0.477, RMSE 3.866, MDA 0.81
    - NNdeep: 2016Q1 to 2021Q2: MFE 0.281, RMSE 2.315, MDA 0.762
    - RFor: 2016Q1 to 2021Q2: MFE -0.282, RMSE 4.089, MDA 0.762
- Nowcasting Tracker — selected historical RMSE and Nowcast ranking (selected rows preserved)
  - Lasso_SEm — Nowcast 6.89 — Historical RMSE 1.69
  - ElNet_SEm — Nowcast 6.89 — Historical RMSE 1.69
  - LinReg — Nowcast 5.15 — Historical RMSE 1.70
  - Ridge_SEm — Nowcast 6.89 — Historical RMSE 1.70
  - DFM — Nowcast 5.17 — Historical RMSE 2.96
  - GPReg — Nowcast 10.73 — Historical RMSE 4.02
- Interpretation: the Nowcasting Tracker ranks models by historical RMSE (lower is better) and reports their current-quarter GDP predictions; backtest charts compare historical outturns with model out-of-sample predictions.

### Summary, conclusions, and practical guidance
- Data and methods applied
  - Compiled datasets for six European economies covering 1995Q1-2021Q4, including both normal times and the COVID period; datasets include standard variables and newer indicators such as Google Trends and air quality.
  - Applied dimension-reduction methods and representative DFM following New York Fed tradition; several ML algorithms introduced.
- Overall empirical conclusions (verbatim numeric claims preserved)
  - Strong majority of applied methods outperform AR(1) benchmark.
  - ML methods reduced average forecast errors up to 75 percent versus AR(1); DFM reduced forecast errors by half versus AR(1) across all countries.
- Caveats and practitioner guidance
  - No single method dominates across all countries and episodes.
  - DFMs tend to perform better during normal times but were less effective at predicting the sharp contraction in 2020Q2 compared with ML methods.
  - ML methods performed strongly at identifying turning points but may overvalue minor changes in explanatory variables during normal times.
  - Practitioners should carefully analyze backtesting results and apply judgment when selecting which models to trust for nowcasting.

*IMF Working Papers — Nowcasting GDP: A Scalable Approach Using DFM, Machine Learning and Novel Data, Applied to European Economies (content as provided).*

### Introduction............................................................................................................

### Introduction

### Contents overview
- Introduction ......................................................................................................................................................................... 4
- Literature Review ............................................................................................................................................................... 5
- Data ....................................................................................................................................................................................... 7
- Models and Methodology ................................................................................................................................................ 9
- DFMs ........................................................................................................................................................................... 10
- Machine Learning ....................................................................................................................................................... 11
- Assessment of Predictive Performance .................................................................................................................. 12
- Results ............................................................................................................................................................................... 14
- Summary and Conclusion ............................................................................................................................................. 18
- Appendices ....................................................................................................................................................................... 20

### Appendices and supporting materials
- 1. Nowcasting Variables by Country ....................................................................................................................... 20
- 2. Sample Results—Hungary ................................................................................................................................... 27
- 3. An Integrated Tool ................................................................................................................................................. 32
- 4. Machine Learning Algorithms .............................................................................................................................. 34
  - 1. Regularized Regression methods ............................................................................................................ 34
  - 2. Support Vector Machine ............................................................................................................................ 34
  - 3. Random Forest ........................................................................................................................................... 36
  - 4. Neural Network ........................................................................................................................................... 38
- 5. Indicators of predictive accuracy: Models by Country ...................................................................................... 40
- References ........................................................................................................................................................................ 41

### Figures and tables referenced
- FIGURES
  - Figure 1. Examples of Model Performances .......................................................................................................... 16
- TABLES
  - Table 1. A Brief Introduction to ML Algorithms ...................................................................................................... 12
  - Table 2. Full Sample- Model Performance (RMSE) Until 2021Q1 ..................................................................... 17
  - Table 3. Pre-COVID Samples- Model Performance (RMSE) Until 2019Q4...................................................... 17
  - Table 4. During COVID-19 Sample- Model Performance (RMSE) Between 2020Q1-2021Q1..........          18

### Glossary (selected acronyms)
- AR  Auto Regressive
- CNN  Convolutional Neural Network
- COVID Coronavirus Disease
- CPI  Consumer Price Index
- DFM  Dynamic Factor Model
- ECB  European Central Bank
- EM Expectation Maximization
- GDP Gross Domestic Product
- GPReg Gaussian Process Regression
- IMF International Monetary Fund
- LASSO Least Absolute Shrinkage and Selection Operator
- LinReg  Linear Regression
- LSTM Long Short-Term Memory
- MAE Mean Absolute Error
- MDA Mean Directional Accuracy
- MIDAS Mixed Data Sampling
- ML  Machine Learning
- NN  Neural Network
- OECD Organization for Economic Co-operation and Development
- OLS Ordinary Least Squares
- ReLU Rectifier Linear Unit
- RF  Random Forest
- RMSE Root Mean Squared Error
- RNN Recursive Neural Networks
- SVM Support Vector Machine
- VAR Vector Autoregressive Model
- WEI Weekly Economic Index

*IMF WORKING PAPERS Nowcasting GDP: A Scalable Approach Using DFM, Machine Learning and Novel Data, Applied to European Economies*

### Introduction

### Introduction

### Need for high-frequency indicators
- The Covid crisis highlighted the need for a systematic analysis of high-frequency indicators of economic activity.
- In normal times, conventional monthly and quarterly aggregate indicators (e.g., industrial production or gross domestic product) generally suffice; during crisis times businesses, market analysts, and policymakers need higher-frequency information.
- Example: in 2020 the economy in Europe recorded in the second quarter its largest contraction since World War II, while the first official GDP growth estimates of the second quarter for European countries were published only in mid-July—too late to assess the magnitude of the shock and calibrate policy responses.
- Question motivating the work: whether a variety of higher-frequency data from different sources could be systematically analyzed to provide sufficiently accurate and timely information to inform policymaking.

### Data availability and novel indicators
- Over the last decade, data availability has become less of a constraint in advanced countries thanks to new technologies and more efficient data collection and compilation.
- New information from non-official sources (market surveys, sentiment indicators, Google search results, air quality, etc.) is frequently available and can quickly register sharp changes in economic conditions, supplementing standard official indicators.
- The paper compiles datasets of both standard variables and new ones, including Google search and air quality, and constructs country-specific databases for selected European economies.

### Modeling advances and methods
- For more than 20 years, models based on dynamic factor methodology (DFM) have been prominent nowcasting tools to assess the state of the economy in the current period or near future.
- Machine learning (ML) methods have recently gained popularity and have been deployed in nowcasting alongside DFMs.
- The paper applies standard DFMs and several ML algorithms to nowcast GDP growth and compares the performance of alternative methods across a heterogeneous group of European economies during normal and crisis times.

### Contribution of this paper
- Motivates and compiles datasets of standard and novel variables (e.g., Google search and air quality), including country-specific databases and transformations.
- Employs DFMs and several ML algorithms to nowcast GDP growth.
- Compares the performance of alternative methods across a heterogeneous group of European economies, during normal and crisis episodes.

### Structure of the paper
- Part 2: literature overview focusing on DFMs and machine learning methods.
- Part 3: discussion of data that could inform an economy’s position and their statistical treatment.
- Part 4: non-technical summary of DFMs and selected machine learning approaches.
- Part 5: main results and comparisons across alternative techniques, including during normal and crisis episodes.
- Part 6: conclusion.

### Literature and methodological context (summary)
- Nowcasting models build on the idea that business cycle fluctuations can be captured by a small number of factors (Sargent and Sims (1977); Stock and Watson (1989)).
- Early state-space models used Kalman Filter for latent common factors; later work extended to high-dimensional datasets (Giannone et al. (2005); Watson (2005); Doz et al (2012)).
- DFMs were implemented for nowcasting at major institutions (Giannone, Reichlin, and Small (2008); ECB (2008); IMF implementations cited).
- DFMs have been widely used during the COVID-19 pandemic and augmented with novel indicators (e.g., Google mobility, Weekly Economic Index).
- Machine learning methods have grown in use because (i) new data sources (e.g., Google Trends) and (ii) many improved algorithms (textbook references).
- Empirical studies report ML methods often outperform traditional benchmarks (AR(1), simple DFMs) in nowcasting and short-term prediction across various country studies and datasets.

### Data selection and transformations
- Nowcasting exploits diverse timely information available before official releases; data selection and transformation are key.
- High-dimensional “big data” datasets can include explanatory variables close to, or exceeding, the number of observations.
- Risks: overfitting and excessive collinearity leading to large variance in estimated coefficients; careful variable selection remains critical.
- The dataset follows standard nowcasting practice and contains a wide range of official and non-official, hard and soft, and proxy data published at different frequencies.
- Exclusions and special considerations:
  - The dataset does not include detailed sectoral data and uses only headline figures (e.g., industrial production included only at aggregate level).
  - The dataset excludes variables available only since 2020 because many high-frequency proxies have very short time series that do not allow historical training and testing.
  - Google trends, Oxford Stringency Indices and air quality data are the only high (weekly) frequency variables included.
- Total number of variables ranges between 20 to 60 depending on the economy and data availability.
- Data clusters included:
  - Firm and production data (industrial output, housing and construction indicators).
  - Surveys or forward-looking indicators (business surveys, economic sentiment, consumer confidence).
  - Labor market variables (unemployment, wages, employment).
  - Financial variables (stock prices, various interest rates, stock of credit and deposits).
  - Prices (producer price index, CPI, house prices, commodity price index).
  - Foreign trade (exports and imports volumes and values).
  - External environment (openness to trade, reliance on Global Value Chain, main trading partners’ economic variables).
  - Other economy-specific variables (reliance on tourism, commodity export, climate vulnerabilities, presence of domiciled multinational companies).
  - Additional indicators (air quality index, hotel occupancy rates, Oxford Stringency Indices, frequency of Google searches for specific words).
- Transformations before modeling:
  - All variables with frequency higher than quarterly (mostly weekly and monthly) must be transformed into quarterly data. DFMs exploit mixed-frequency data via bridge equations and Kalman filter; ML models convert weekly and monthly data to quarterly so all final variables are quarterly.
  - Time series must be seasonally adjusted and detrended. Generally, transforming quarterly variables in levels into year-over-year growth rates addresses both seasonality and non-stationarity. A smaller number of variables (e.g., unemployment rate, survey outcomes) are kept in levels or transformed to year-over-year differences.

### Models and methodology focus
- Methods need to (i) extract information from a large number of variables and (ii) address issues from large datasets (collinearity, overfitting), while handling incomplete and mixed-frequency data and processing new information in real time.
- The paper focuses on DFMs and tools relying on machine learning.

*IMF Working Papers — Nowcasting GDP: A Scalable Approach Using DFM, Machine Learning and Novel Data, Applied to European Economies — Introduction*

### Appendix 1 presents country specific databases and indicates the transformation applied to each variable.

### wpiea2022052-print-pdf - Appendix 1 presents country specific databases and indicates the transformation applied to each variable.

### Dynamic Factor Models (DFMs)
- Core idea:
  - A large number of economic variables (N) show similar developments over the business cycle and thus can be described by a small number of common factors (r), i.e., r<N.
- Measurement and structure:
  - Observed variables: y1,t, …, yn,t driven by unobserved dynamic factors f1,t, …, fr,t and idiosyncratic errors e1,t, …, en,t:
    - yi,t = λi,1 f1,t + · · · + λi,r fr,t + ei,t, for i = 1, …, n (1)
  - Common component: Σr j=1 λi,j fj,t represents the part explained by common factors; ei,t captures idiosyncratic movements.
- Variable reduction approaches:
  - (i) naïve: extract top factors (largest eigenvectors) to jointly explain about 80 percent of variation.
  - (ii) structured: split dataset into clusters using prior expertise and extract one factor (most relevant eigenvector) per cluster.
- State-space representation and dynamics:
  - Factors and idiosyncratic components are modeled as Gaussian autoregressive processes to capture serial correlation and persistence:
    - fj,t = aj fj,t−1 + uj,t,   Uj,t ~ N(0, σu,j2) (2)
    - ei,t = ρi ei,t−1 + εi,t,   εi,t ~ N(0, σε,t2) (3)
  - Equations 1, 2, and 3 form a state space model: Equation 1 = measurement equation; Equations 2 and 3 = transition equations.
- Estimation:
  - Parameters estimated iteratively using the Kalman smoother and the expectation maximization (EM) algorithm.
  - Initialization:
    - Step 1: EM computes principal components; model parameters estimated by OLS treating principal components as true common factors.
    - Step 2: Using parameters from step 1, updated factor estimates obtained via Kalman smoother (Giannone et al. (2008); Doz et al. (2011)).
    - Iterate the two steps until convergence to obtain maximum likelihood estimates.
- Methodological lineage:
  - The dynamic factor model used closely follows the NY Fed block structure modeling and estimating framework (Bok et. al (2017); Banbura et al (2013), Chapter 4).
  - The expectation maximization algorithm follows Banbura and Modungo (2010).

### Machine Learning (ML) methods: motivation and characteristics
- Reasons for wide use:
  - Flexibility: effective at capturing patterns and complex/nonlinear relationships.
  - Variable selection: actively limits overfitting by selecting suitable indicators.
  - Cross-validation: fine-tunes model parameters by partitioning sample into training/testing subsamples and gauging pseudo out-of-sample performance.
- Limitations and considerations:
  - Interpretability: research scarce; ML often seen as a black box with a trade-off between performance and interpretability.
  - Missing data: some ML algorithms are less robust to missing observations than DFM. Toolkit approach used here: a naïve combination of deleting rows or columns with missing data.
  - Focus of study: ML techniques applied here are for prediction only.
- Brief descriptions of ML algorithms (as provided):
  - LASSO, Ridge, Elastic Net:
    - Modified linear regressions with regularizations to enhance prediction accuracy; avoid dimensionality and overfitting but face linearity challenge.
  - Support Vector Machine (SVM):
    - Constructs hyperplanes (kernel-based); can overcome linear regression drawbacks but performance depends on kernel and regularization selection and may limit interpretability.
  - Random Forest (RF):
    - Ensemble of regression trees; non-parametric and flexible; limited at predicting extreme/outlier events and lacks interpretability.
  - Neural Network (NN):
    - Multi-layer non-linear mapping with neurons applying functions across layers; very flexible and powerful for prediction but significantly limits interpretability.

### Assessment of predictive performance: methodology and metrics
- Backtesting protocol (pseudo-real-time):
  - In each period t: explanatory variables available up to quarter t; target variable available only up to t−1.
  - Estimate model parameters using data up to t−1, produce one-step-ahead out-of-sample prediction for time t, then compute forecast error by comparing predicted vs actual target.
- Predictive accuracy statistics used (definitions preserved):
  - Root mean squared error (RMSE):
    - RMSE = sqrt( (∑(yt − ŷt)2) / N )
  - Mean absolute error (MAE):
    - MAE = (∑ |yt − ŷt|) / N
  - Mean Directional Accuracy (MDA):
    - MDA = (1/N) ∑ 1[ssign(yt − yt−1) = ssign(ŷt − yt−1)], where 1A(x) = {1 if x ∈ A; 0 if else} and ssign(x) = {−1 if x < 0; 0 if x = 0; 1 if x > 0}
- Use of metrics:
  - RMSE places higher weight on large errors via squaring.
  - MAE assigns equal weight to all errors and is less sensitive to outliers.
  - MDA measures probability that model forecasts the correct direction of change.
  - Metrics inform model aggregation weights; example: inverse RMSEs used as weights to create weighted averages of predictions.
- Subsample analysis:
  - Performance assessed over: (i) full sample, (ii) normal times (2015 to 2019Q4), and (iii) period of heightened volatility triggered by COVID-19 (2020Q1 up to 2021Q4).

### Empirical results and key findings
- Data and scope:
  - Applied DFM and various ML models to a set of six European countries over different sample periods; tables present RMSEs for full sample up to 2021Q1, pre-COVID up to 2019Q4, and COVID period 2020Q1 to 2021Q1.
- Main findings (verbatim numeric and qualitative claims preserved):
  - Both DFM and ML models add value, especially during times of hightened volatility.
    - Except for Ireland and Malta, the DFM and ML models outperform AR(1) in a large majority of cases on the whole sample (Table 2).
    - AR(1) performs quite well during relatively stable growth and frequently outperforms other nowcasting models (Table 3).
    - During the COVID-19 pandemic (Table 4):
      - Ridge regression and Lasso models could significantly reduce the average forecast errors ranging from 40 to 75 percent relative to the AR(1) benchmark.
      - For DFM, the forecast errors are about 50 percent lower than the AR(1) benchmark across all countries.
  - Model performance varies across sub-samples (time periods):
    - Ridge Regression and DFM perform well for Poland and Hungary, respectively, during normal times but underperform during the COVID-19 period.
    - RMSEs during normal times (pre-COVID) are lower than during the COVID period.
    - Pre-pandemic (Table 3): DFM performs relatively well and is among best performers for Austria, Hungary, and Portugal; DFM performed worse than AR(1) for Malta and Poland.
    - ML methods: Gaussian Process Regression (GPReg) generates the lowest RMSEs for Ireland and Portugal during normal times; Linear Regression (LinReg) performs best for Poland.
    - During COVID (Table 4): several ML methods appeared to perform better at capturing turning points (notably 2020Q2-2020Q3) compared to both AR(1) and DFM.
  - Cross-country heterogeneity:
    - No one-size-fits-all model; best model differs across countries and time.
    - Example: DFM generates the lowest RMSE for Malta; Ridge and LASSO perform best for Austria, Hungary, and Poland; LASSO does not track well actual GDP growth for Malta and Ireland.
- Visualization:
  - Figure 1 provides illustrative examples of model performances categorized as "The Good", "The (Not So) Bad", and "The Ugly" (chart-based illustrations).

### Operationalization: automated nowcasting tool
- The integrated, fully-automated nowcasting tool:
  - Automatically (i) collects and treats datasets; (ii) applies a suite of DFM and ML models to perform backtests (charts and forecasting accuracy indicators); (iii) re-estimates models whenever new data are available and produces a nowcast of current quarter GDP growth; (iv) generates aggregated outputs across methods and subsamples.
  - Tool can be applied to any country once variables are selected and subject to data availability (Annex [2]).

### Summary and conclusion: implications and guidance
- Data and methods:
  - Compiled datasets for six European economies covering 1995Q1-2021Q4, including normal times (1995-2019) and COVID period (2020Q1-2021Q4). Datasets include standard variables and newer indicators such as Google Trends and air quality.
  - Applied dimension-reduction methods to avoid collinearity and overfitting; representative DFM follows New York Fed tradition; several ML algorithms introduced.
- Overall performance:
  - Strong majority of applied methods outperform AR(1) benchmark.
  - ML methods reduced average forecast errors up to 75 percent versus AR(1); DFM reduced forecast errors by half versus AR(1) across all countries.
- Caveats and practical guidance:
  - No single method dominates across all countries and episodes.
  - DFMs tend to perform better during normal times but were less effective at predicting the sharp contraction in 2020Q2 compared with ML methods.
  - ML methods performed strongly at identifying turning points but may overvalue minor changes in explanatory variables during normal times.
  - Practitioners should carefully analyze backtesting results and apply judgment when selecting which models to trust for nowcasting.

*Source: IMF Working Paper content (Appendices and main sections as provided).*

### 1. Nowcasting Variables by Country

### 1. Nowcasting Variables by Country

### Overview
- The document lists standardized indicator inventories used for nowcasting GDP across multiple European countries.  
- Indicators are grouped by thematic categories: National accounts, Housing and construction, Labor, Manufacturing, Retail and consumption, International trade, Financial, Surveys, Other.  
- Each indicator entry preserves its Frequency and Transformation as specified (e.g., Q, M, D, YoY percent change, Level, Change (difference), Hp-filtered cyclical components).

### Austria — Indicator inventory
- National accounts
  - Real gross domestic product — Q — YoY percent change
  - Real final consumption expenditure — Q — YoY percent change
  - Real gross capital formation — Q — YoY percent change
  - Real exports of goods & services — Q — YoY percent change
  - Real imports of goods & services — Q — YoY percent change
- Housing and construction
  - Residential property prices — Q — Level
  - Industrial production index: construction — M — YoY percent change
- Labor
  - Labor force participation rate — Q — Level
  - Employment rate — Q — Level
  - Unemployment rate — Q — Level
  - Productivity per employee: industry incl. construction — M — YoY percent change
- Manufacturing (selected)
  - Domestic industrial new orders — M — YoY percent change
  - Industrial production index: total industry excl. construction — M — YoY percent change
  - Germany: industrial production excl. construction — MYoY percent change
  - Germany: capacity utilization: manufacturing — Q — Level
  - Euro area: manufacturing new orders — M — Level
  - United States: industrial production index — MYoY percent change
- Retail and consumption
  - Retail trade value excl. autos & motorcycles — M — YoY percent change
  - Retail trade volume excl. autos & motorcycles — M — YoY percent change
  - New passenger car registrations — M — YoY percent change
- International trade
  - Merchandise trade: exports — M — YoY percent change
  - Merchandise trade: imports — M — YoY percent change
  - Balance of payments: services credit — M — YoY percent change
  - Balance of payments: services debit — M — YoY percent change
- Financial
  - Domestic loans to consumers — M — YoY percent change
  - Households' overnight deposits — M — YoY percent change
- Surveys
  - Industrial confidence indicator — M — Level
  - Purchasing Managers Index: Manufacturing — M — Level
  - Germany: Ifo business climate — M — Level
  - Euro area: Purchasing Managers Index: composite employment — M — Level
  - Euro area: business climate indicator — M — Level
- Other
  - Tourist arrivals — M — YoY percent change
  - Tourists' overnight stays — M — YoY percent change
  - Google: keywords — M — Level
  - Nitrogen dioxide emissions — M — Level
  - Apple Mobility Trends Index: walking — D — Level
  - Google: change in visits relative to baseline: retail & recreation — D — Level

### Hungary — Indicator inventory (sample)
- National accounts
  - Real gross domestic product — Q — YoY percent change
  - Real final consumption expenditure — Q — YoY percent change
  - Real gross capital formation — Q — YoY percent change
  - Real exports of goods & services — Q — YoY percent change
  - Real imports of goods & services — Q — YoY percent change
- Labor
  - Unemployment rate, 25-74 years — M — Level
- Manufacturing (selected)
  - Domestic turnover: manufacturing — M — Level
  - Domestic turnover: consumer goods — M — Level
  - Domestic turnover: capital goods — M — YoY percent change
  - Domestic turnover: nondurable consumer goods — M — YoY percent change
  - Domestic turnover: durable consumer goods — M — YoY percent change
  - Domestic turnover: intermediate goods — M — Level
- Retail and consumption
  - New car registrations — M — Level
- Financial
  - Stock price index, BUX — M — Level
- Surveys (selected)
  - Business confidence index — M — Level
  - Economic sentiment index — M — YoY percent change
  - Consumer confidence index — M — YoY percent change
  - Retail trade survey: volume of stocks — M — YoY percent change
  - Manufacturing PMI and component series — M — various transformations (Level, YoY percent change)
  - Factors limiting building activity: demand, financial constraints, labor shortage, equipment shortage, other factors — M — YoY percent change
  - Industry: volume of export order books — M — Level
  - Services: expected demand, next 3 months — M — Level
- Other
  - Google: keywords — M — Level
  - Nitrogen dioxide emissions — M — Level

### Ireland — Indicator inventory (selected)
- National accounts (extensive GVA and sectoral series) — Q — YoY percent change
  - Real gross domestic product; Real private consumption expenditure; Real gross fixed capital formation; Real exports of goods & services; Real imports of goods & services; Real government consumption expenditure; Real modified total domestic demand; Real gross value added (GVA) and sectoral GVAs (agriculture; industry excluding construction; manufacturing; building and construction; distribution, transport, hotels, and restaurants; information and communication; financial and insurance activities; real estate activities; professional, admin, and support services; public admin, education, and health; arts, entertainment, and other services)
- Prices
  - Harmonized inflation — M — YoY percent change
  - HICP: Total excluding energy, food, alcohol, and tobacco — M — YoY percent change
  - Residential property price index — M — YoY percent change
  - West Texas intermediate crude oil spot price (Cushing, OK) — M — Percent change
  - Exchange rate: US dollar/euro (average) — M — Percent change
  - Exchange rate: UK Pound sterling/euro (average) — M — Percent change
- Labor
  - Employed — Q — YoY percent change
  - Unemployment rate, Covid-19 adjusted, upper bound — M — Level
  - Unemployment rate, Covid-19 adjusted, lower bound — M — Level
  - Unemployment — M — Percent change
  - Pandemic unemployment payment recipients — M — Level
  - Participation rate — Q — YoY percent change
  - Average hourly earnings — Q — YoY percent change
  - Public sector employment including semi-state bodies — Q — YoY percent change
  - Total job postings — D — Level
- Fiscal
  - Central government: tax revenue and tax components — M — YoY percent change
- Manufacturing and international indicators (selected)
  - Industrial production (modern/traditional), Euro area and US manufacturing/industrial series — M — Percent change or YoY percent change
- Financial
  - Stock price index, ISEQ — M — Percent change
  - 3-month interbank market rate — M — Level
  - 10-year government bond yield — M — Level
  - Money supply: M3, growth rate — M — Level
  - Private sector loan growth — M — Level
  - Various international equity and commodity indices — M — Percent change
- Surveys and Other
  - PMI composite output; consumer sentiment indices; Apple Mobility Trends Index: walking — D — Level; Moovit public transit index — D — Level; Google: change in visits relative to baseline: retail & recreation — D — Level; Composite indicator of systemic stress — D — Level; Google: keywords — M — Level; Nitrogen dioxide emissions — M — Level

### Malta — Indicator inventory (selected)
- National accounts
  - Real gross domestic product — Q — YoY percent change
  - Real private consumption expenditure — Q — YoY percent change
  - Real government consumption expenditure — Q — YoY percent change
  - Real gross capital formation — Q — Level
  - Real exports of goods & services — Q — Level
  - Real imports of goods & services — Q — Level
- Housing and construction
  - Building permits: residential buildings excl. community residences — Q — Level
  - Construction output — Q — Level
- Labor
  - Employment rate — Q — Level
  - Unemployment rate, NSA — Q — YoY percent change
  - Hours worked: industry excl. construction — Q — YoY percent change
  - Hours worked: construction — Q — Level
  - Unemployment rate, SA — Q — Level
- Fiscal
  - Tax revenue — M — YoY percent change
- Manufacturing
  - Industrial turnover — Q — YoY percent change
  - Industrial production — Q — YoY percent change
- Retail and consumption
  - Retail trade: confidence indicator — Q — YoY percent change
  - Wholesale & retail trade & repair of autos & motorcycles, volume — Q — YoY percent change
- International trade
  - Merchandise trade: exports/imports — Q — YoY percent change
  - Services trade, value — Q — YoY percent change
- Financial
  - Other MFI loans to households & individuals for house purchase — Q — Hp-filtered cyclical components
  - Term premium — M — YoY percent change
- Surveys (selected)
  - Industrial confidence indicator — M — YoY percent change
  - Services confidence indicator; Consumer confidence indicator; Economic sentiment indicator — M — YoY percent change
  - Retail trade: order expectations — M — YoY percent change
  - Euro area indicators — M — YoY percent change
- Other
  - Departing tourists; Departing tourists: total expenditure; Departing tourists: total nights — Q — YoY percent change
  - Stock of licensed motor vehicles — D — YoY percent change
  - Google: keywords — M — YoY change (difference)
  - Nitrogen dioxide emissions — M — YoY change (difference)

### Poland — Indicator inventory (selected)
- National accounts
  - Real gross domestic product — Q — YoY percent change
  - Real final consumption expenditure; Real government consumption expenditure; Real private consumption expenditure; Real gross capital formation; Real gross fixed capital formation; Real exports/imports of goods & services — Q — YoY percent change
  - EU27: real gross domestic product — Q — YoY percent change
  - United States: real gross domestic product — Q — YoY percent change
  - United States: IHS Markit monthly real GDP — M — YoY percent change
- Housing and construction
  - Construction & assembly production — M — YoY percent change
- Prices
  - HICP: core inflation — M — YoY percent change
  - West Texas intermediate crude oil spot price (Cushing, OK) — M — MYoY percent change
  - Poland: JPMorgan Real Broad Effective Exchange Rate Index, CPI B — M — Level
- Labor
  - Unemployment rate — M — Level
  - Average paid employment: enterprise sector — M — YoY percent change
  - Germany: unemployment rate — M — Level
- Manufacturing (selected)
  - EU27: industrial production; Industrial new orders; United States: industrial production; Industrial production excl. construction; Industrial production: mining & quarrying; Industrial production: manufacturing — M — YoY percent change
  - Germany: industrial production series and manufacturing orders, volume — M — YoY percent change
- Retail and consumption
  - Wholesale trade enterprises; Retail sales, constant prices — M — YoY percent change
- International trade
  - Balance of payments: exports — M — YoY percent change
  - Balance of payments: imports — M — YoY percent change
- Financial
  - MSCI share price index with net dividends, local currency — M — YoY percent change
  - Money supply: M3 — M — YoY percent change
  - Poland: policy rate — M — Level
- Surveys (selected)
  - PMI: manufacturing — M — Level
  - Euro area and global PMI and confidence indicators — M — Level
  - Germany: ifo business climate index, expectations, production activity — M — various transformations
- Other
  - Apple Mobility Trends Index: walking — D — Level
  - Google: change in visits relative to baseline: transit stations; retail & recreation — D — Level
  - Tourist arrivals — M — YoY percent change
  - Google: keywords — M — YoY percent change
  - Nitrogen dioxide emissions — M — YoY percent change

### Portugal — Indicator inventory (selected)
- National accounts
  - Real gross domestic product — Q — YoY percent change
  - Real final consumption expenditure; Real government consumption expenditure; Real exports/imports of goods & services — Q — YoY percent change
  - Gross value added: construction — Q — YoY percent change
- Housing and construction
  - Housing price index — Q — Level
  - Construction output — M — YoY percent change
- Labor
  - Participation rate; Employment rate; Unemployment rate — Q — YoY percent change
  - Gross monthly earnings, 3-month moving average — M — YoY percent change
  - Real labor productivity per person — Q — YoY percent change
- Manufacturing (selected)
  - Industrial new orders; Industrial production — M — YoY percent change
  - Germany: industrial production excl. construction; capacity utilization; manufacturing orders — M/Q — Level or YoY percent change
  - EU27 and United States: industrial production series — M — YoY percent change
- Retail and consumption
  - Retail trade; Retail sales volume; Total industry sales; New car registrations — M — YoY percent change
- International trade
  - Merchandise trade: exports/imports; Services exports/imports; Services trade value — M — YoY percent change
- Financial
  - OMFI loans to private individuals for consumption; Overnight deposits — M — YoY percent change
- Surveys
  - Economic sentiment indicator; Manufacturing confidence; Germany: ifo business climate — M — Level
  - Euro area PMI: composite employment; Euro area: business climate indicator — M — Level
- Other
  - Number of travelers; Overnight stays in tourist accommodation establishments — M — YoY percent change
  - Oxford government response index; Oxford stringency index; Oxford risk of openness index: community understanding — D — Level
  - Google: keywords; Nitrogen dioxide emissions — M — Level

### Standardization note
- Frequency abbreviations used in the inventories: Q (quarterly), M (monthly), D (daily).
- Transformation labels include YoY percent change, Percent change, Level, Change (difference), Hp-filtered cyclical components, and others as listed per indicator.

---

### 2. Sample Results — Hungary (November 2011 example)

### Backtest: purpose and periods
- The Backtest assesses forecasting power of individual models on historical data over three periods:
  - 2016Q1 to 2021Q2
  - 2016Q1 to 2019Q4
  - 2020Q1 to 2021Q2
- Metrics provided: MFE (Mean Forecast Error), RMSE (Residual Mean Squared Error), and MDA (Mean Directional Accuracy).
- Charts (not reproduced here) compare actual GDP growth with out-of-sample predictions by individual models.

### Backtest: reported metric triples (MFE, RMSE, MDA) by model and period
- DFM
  - 2016Q1 to 2021Q2: MFE -0.202, RMSE 2.893, MDA 0.81
  - 2016Q1 to 2019Q4: MFE -0.063, RMSE 0.507, MDA 0.733
  - 2020Q1 to 2021Q2: MFE -0.574, RMSE 5.477, MDA 1
- ElNet_SEm
  - 2016Q1 to 2021Q2: MFE -0.141, RMSE 1.60, MDA 0.81
  - 2016Q1 to 2019Q4: MFE -0.133, RMSE 1.132, MDA 0.733
  - 2020Q1 to 2021Q2: MFE -0.162, RMSE 2.444, MDA 1
- ElNet_SE1
  - 2016Q1 to 2021Q2: MFE -0.196, RMSE 1.662, MDA 0.714
  - 2016Q1 to 2019Q4: MFE -0.248, RMSE 1.146, MDA 0.6
  - 2020Q1 to 2021Q2: MFE -0.059, RMSE 2.575, MDA 1
- Lasso_SEm
  - 2016Q1 to 2021Q2: MFE -0.129, RMSE 1.591, MDA 0.857
  - 2016Q1 to 2019Q4: MFE -0.132, RMSE 1.134, MDA 0.8
  - 2020Q1 to 2021Q2: MFE -0.121, RMSE 2.419, MDA 1
- Lasso_SE1
  - 2016Q1 to 2021Q2: MFE -0.201, RMSE 1.691, MDA 0.762
  - 2016Q1 to 2019Q4: MFE -0.255, RMSE 1.183, MDA 0.667
  - 2020Q1 to 2021Q2: MFE -0.059, RMSE 2.599, MDA 1
- Ridge_SEm
  - 2016Q1 to 2021Q2: MFE -0.194, RMSE 1.621, MDA 0.714
  - 2016Q1 to 2019Q4: MFE -0.231, RMSE 1.109, MDA 0.6
  - 2020Q1 to 2021Q2: MFE -0.097, RMSE 2.519, MDA 1
- Ridge_SE1
  - 2016Q1 to 2021Q2: MFE -0.176, RMSE 1.632, MDA 0.714
  - 2016Q1 to 2019Q4: MFE -0.234, RMSE 1.108, MDA 0.6
  - 2020Q1 to 2021Q2: MFE -0.021, RMSE 2.548, MDA 1
- RFor
  - 2016Q1 to 2021Q2: MFE -0.282, RMSE 4.089, MDA 0.762
  - 2016Q1 to 2019Q4: MFE 0.248, RMSE 0.65, MDA 0.667
  - 2020Q1 to 2021Q2: MFE -1.695, RMSE 7.758, MDA 1
- SVMachine
  - 2016Q1 to 2021Q2: MFE -0.186, RMSE 1.827, MDA 0.714
  - 2016Q1 to 2019Q4: MFE -0.284, RMSE 1.342, MDA 0.6
  - 2020Q1 to 2021Q2: MFE 0.075, RMSE 2.728, MDA 1
- LinReg
  - 2016Q1 to 2021Q2: MFE -0.262, RMSE 1.707, MDA 0.81
  - 2016Q1 to 2019Q4: MFE -0.297, RMSE 1.333, MDA 0.733
  - 2020Q1 to 2021Q2: MFE -0.168, RMSE 2.438, MDA 1
- GPReg
  - 2016Q1 to 2021Q2: MFE 0.477, RMSE 3.866, MDA 0.81
  - 2016Q1 to 2019Q4: MFE 0.305, RMSE 0.759, MDA 0.733
  - 2020Q1 to 2021Q2: MFE 0.934, RMSE 7.298, MDA 1
- NN
  - 2016Q1 to 2021Q2: MFE 0.341, RMSE 3.685, MDA 0.619
  - 2016Q1 to 2019Q4: MFE 0.119, RMSE 0.792, MDA 0.467
  - 2020Q1 to 2021Q2: MFE 0.931, RMSE 6.937, MDA 1
- NNdeep
  - 2016Q1 to 2021Q2: MFE 0.281, RMSE 2.315, MDA 0.762
  - 2016Q1 to 2019Q4: MFE 0.368, RMSE 0.881, MDA 0.667
  - 2020Q1 to 2021Q2: MFE 0.047, RMSE 4.194, MDA 1
- NNLSTM (NNlstm)
  - 2016Q1 to 2021Q2: MFE -0.073, RMSE 3.351, MDA 0.714
  - 2016Q1 to 2019Q4: MFE 0.174, RMSE 0.736, MDA 0.6
  - 2020Q1 to 2021Q2: MFE -0.721, RMSE 6.304, MDA 1
- NNbilstm
  - 2016Q1 to 2021Q2: MFE -0.081, RMSE 4.024, MDA 0.571
  - 2016Q1 to 2019Q4: MFE 0.269, RMSE 0.989, MDA 0.4
  - 2020Q1 to 2021Q2: MFE -1.015, RMSE 7.535, MDA 1
- NNconv
  - 2016Q1 to 2021Q2: MFE -0.363, RMSE 2.374, MDA 0.762
  - 2016Q1 to 2019Q4: MFE -0.313, RMSE 1.544, MDA 0.667
  - 2020Q1 to 2021Q2: MFE -0.499, RMSE 3.782, MDA 1
- AR1
  - 2016Q1 to 2021Q2: MFE 0.571, RMSE 5.374, MDA 0.714
  - 2016Q1 to 2019Q4: MFE 0.204, RMSE 0.862, MDA 0.667
  - 2020Q1 to 2021Q2: MFE 1.547, RMSE 10.193, MDA 0.8

(Note: MFE ... Mean Forecast Error, RMSE ... Residual Mean Squared Error, MDA... Mean Directional Accuracy)

### Nowcasting Tracker — selected historical RMSE and Nowcast ranking
- Nowcast Historical RMSE (selected rows as presented)
  - Lasso_SEm — Nowcast 6.89 — Historical RMSE 1.69
  - ElNet_SEm — Nowcast 6.89 — Historical RMSE 1.69
  - LinReg — Nowcast 5.15 — Historical RMSE 1.70
  - Ridge_SEm — Nowcast 6.89 — Historical RMSE 1.70
  - Ridge_SE1 — Nowcast 6.89 — Historical RMSE 1.72
  - ElNet_SE1 — Nowcast 6.90 — Historical RMSE 1.75
  - Lasso_SE1 — Nowcast 6.90 — Historical RMSE 1.78
  - SVMachine — Nowcast 5.63 — Historical RMSE 1.90
  - NNconv — Nowcast 9.98 — Historical RMSE 2.27
  - NNdeep — Nowcast 18.65 — Historical RMSE 2.40
  - DFM — Nowcast 5.17 — Historical RMSE 2.96
  - NN — Nowcast 33.94 — Historical RMSE 3.26
  - NNlstm — Nowcast 11.99 — Historical RMSE 3.79
  - RFor — Nowcast 8.62 — Historical RMSE 3.86
  - NNbilstm — Nowcast 9.97 — Historical RMSE 3.91
  - GPReg — Nowcast 10.73 — Historical RMSE 4.02

- The Nowcasting Tracker ranks individual models by RMSE (lower RMSE indicates better predictive accuracy on historical data) and presents current-quarter GDP growth predictions by individual models; the nowcasting chart highlights historical GDP growth and the values expected for the current quarter by the six most accurate models (chart not reproduced here).

*IMF Working Papers — Nowcasting GDP: A Scalable Approach Using DFM, Machine Learning and Novel Data, Applied to European Economies*

### 3. An Integrated Tool

### 3. An Integrated Tool

### Overview
- The authors developed an integrated tool that automatically:
  - (i) collect and treat the data set;
  - (ii) applies a suit of DMF and ML models to the dataset to generate backtest results (graphs and quantitative indicators), and
  - (iii) nowcast GDP growth for the current quarter for each method and aggregating all the methods.
- The tool can be applied to any country, and is automated once variables to form the dataset have been selected.
- The tool is developed in Matlab and is composed of three major components: the Data Warehouse, the Backtest, and the Nowcasting Tracker.
- All three components are integrated by the Reports Generator, which is a Matlab script that serves as the main user interface (in addition to an Excel-based file that the script reads from).

### Implementation and Components
- Reports Generator
  - A Matlab script that serves as the main user interface and reads an Excel-based file (the “Spec File”).
  - Orchestrates the Data Warehouse, Backtest, and Nowcasting Tracker.

- Data Warehouse
  - A Matlab script that fetches historical data from various data sources such as Haver and Google.
  - Where necessary, R scripts are also invoked for retrieving from certain data sources.
  - The Matlab script receives instructions (what Haver series to fetch, what frequency the data is in, what data transformation to perform, etc.) from the Excel “Spec File,” which is part of the user interface and serves as input to the Reports Generator.

- Backtest
  - Performs out-of-sample pseudo-nowcasts on an expanding window of historical data.
  - Produces plots and accuracy metrics for users to compare performance of nowcasting methods against one another and against actual historical outturns.
  - Out-of-sample pseudo-nowcasts assume that for a certain historical quarter t = t’, GDP data is available up to t’-1, and try to predict the GDP for quarter t’.
  - The backtest result for a given nowcasting method combines n such out-of-sample nowcasts for t = t’, t’-1, ..., t’-n+1, where n depends on historical data availability and the user’s specification.

- Nowcasting Tracker
  - Produces actual nowcasts for the current quarter using all or some of the methods tested in the backtest.
  - Generates and saves a table and a plot of GDP nowcasts for the user’s information.
  - Enables users to form comprehensive judgement based on each method’s backtest performance and their knowledge on the economy concerned.

### Modularity and Extensibility
- All nowcasting methods, including the DFM and various machine learning algorithms, have their respective standalone functions or scripts and are invoked as needed in the Backtest and Nowcasting Tracker.
- The toolkit’s modularity and standardized output format allow users with some familiarity with the toolkit structure and the Matlab language to:
  - Add customized nowcasting methods to the toolkit.
  - Remove existing methods from the toolkit easily.
- Users normally do not need to modify or carefully read the Matlab programs associated with the three underlying components unless they would like to add their own nowcasting method.

### Execution Flow and Outputs
- Typical end-to-end execution:
  - Data Warehouse fetches and transforms historical data according to the Spec File.
  - Backtest runs out-of-sample pseudo-nowcasts across an expanding window and produces performance plots and accuracy metrics.
  - Nowcasting Tracker produces current-quarter nowcasts using selected methods and saves a summary table and plot.
  - Reports Generator integrates the components and reads the Excel Spec File as the primary user input.
- Outputs intended for user decision-making:
  - Backtest comparison charts and quantitative indicators for method selection.
  - Saved table and plot of GDP nowcasts for the current quarter to inform comprehensive judgement.

*IMF Working Paper No. WP/22/52 — 3. An Integrated Tool*

---


_Source: https://www.imf.org/-/media/files/publications/wp/2022/english/wpiea2022052-print-pdf.pdf_