## Introduction

## Source details

**Canonical URL:** [Introduction](https://www.imf.org/-/media/files/publications/wp/2024/english/wpiea2024206-print-pdf.pdf)

## Other formats

- [Markdown version](/-/media/files/publications/wp/2024/english/wpiea2024206-print-pdf.pdf.md)
- [Structured JSON version](/-/media/files/publications/wp/2024/english/wpiea2024206-print-pdf.pdf.json)

---

### Context and motivation
- Japan experienced inflation above the two-percent target for more than two years since April 2022, the first time in three decades.
- Inflation drivers evolved from cost-push factors initially to increasingly demand-side factors, with services inflation above 2 percent.
- Forecasting challenges noted:
  - The Bank of Japan (BOJ) repeatedly revised upward its quarterly economic outlook; the level and persistence of core inflation surprised the BOJ.
  - Many central banks and the World Economic Outlook under-projected inflation in 2021–22.
  - Structural changes from the pandemic (e.g., persistent supply chain constraints, steepening Phillips curves) were not captured by prevailing models.
  - Three decades of disinflation in Japan complicate forecasters’ tasks.
- Rationale for machine learning (ML):
  - Traditional forecasting models assume a pre-specified stochastic relationship and are limited by specification.
  - ML models discover mappings between inputs and outputs using rigorous training and testing algorithms, enabling:
    - Incorporation of a wider range of variables via regularization and handling of non-linear relationships.
    - Emphasis on out-of-sample predictive performance and model selection.
    - Rapid re-training to adapt to changing data patterns.

### Scope of the paper
- First application of ML to inflation forecasting in Japan targeting core inflation excluding fresh food and energy (VAT rate hikes excluded as one-off policy effects).
- Forecasting horizons: one-month, three-month and six-month ahead to maximize training samples with inflationary pressure.
- Models considered: LASSO, Elastic Net, Random Forest, XGBoost.
- Benchmarks: random walk (RW) and autoregressive (AR) univariate models.
- Key empirical result (back testing, post 2022):
  - Among six models, LASSO performs best for core inflation forecasts.
  - Average RMSE of LASSO forecasts: about 0.1, 0.2 and 0.3 percentage point for 1-month, 3-month, and 6-month ahead inflation forecasts, respectively.
  - Formal tests indicate LASSO performs significantly better than the benchmark models.
  - Forecasting errors of LASSO appear smaller than professional forecasters’ errors in 2022–23:
    - Consensus forecast bias: averaging -0.6 percentage points for 3-month ahead inflation and -1.2 percentage points for 6-month ahead inflation.
  - Semi-structural BOJ model and forecasters calibrated to pre-pandemic data missed inflation in 2022 and 2023.

### Methodology

### Forecasting framework
- Model specification:
  - π_{t+h} = F_h(x_t) + u_{t+h}, h = 1,...,H, t = 1,...,T.
    - π_{t+h} is core inflation (excluding fresh food and energy) in month t+h.
    - x_t = (x_{1t},...,x_{nt})′ contains covariates and possible lags of π_t.
    - F_h(.) is the mapping for horizon h (h = 1, 3, or 6).
  - Direct forecasts used (covariates are not predicted).
- Forecast equation and training window:
  - π̂_{t+h|t} = F̂_{h,t−R_h+1:t}(x_t), where F̂ is estimated using data from time t−R_h+1 to t and R_h is the window size (varies by horizon and number of lags).
- Features:
  - More than twenty numerical features including economic activity indicators, exchange rates, import prices and tourist arrivals.

### Training, validation, and selection
- Monthly data since 2012 used for cross validation and tuning.
- Time-split K-fold cross-validation to preserve temporal dependency: the k-th split treats the first k folds as training and the (k+1)-th as test; successive training sets are supersets of previous ones.
- Hyperparameters chosen to minimize average RMSE across test folds.
- Back-testing period: July-December 2023. RMSE computed over this period for 1-, 3-, and 6-month ahead forecasts; model with lowest RMSE selected.
- Rolling-window framework (Giacomini and White, 2006): expands the cross-validation sample by one period with each new data point and shifts the back-testing sample forward by one period to adapt to recent data and attenuate structural-break effects.

### Machine learning methods and implementation

### Models considered
- Linear ML models:
  - LASSO (Least Absolute Shrinkage and Selection Operator)
  - Elastic Net
- Tree-based ML models:
  - Random Forest (RF)
  - XGBoost (gradient boosting implementation)

### Implementation and tuning
- Software and libraries: Python with sklearn and xgboost; BayesSearchCV from skopt for tree-based hyperparameter tuning.
- Cross-validation: TimeSeriesSplit with 10 folds (n_splits=10).
- Hyperparameter tuning:
  - Tree-based models: Bayesian Search algorithm (BayesSearchCV); for XGBoost, n_iter=30 per optimization.
  - LASSO: linear_model.LassoCV from sklearn (α selected by built-in cross-validation).
  - Elastic Net: linear_model.ElasticNetCV from sklearn with l1_ratio grid [.1, .5, .7, .9, .95, .99].
- Rolling-window retraining examples:
  - One-month ahead forecast in July 2023 uses data until June 2023.
  - Three-month ahead forecast in July 2023 uses data until April 2023.

### Benchmarks and evaluation metric
- Benchmarks:
  - Random walk (RW): π̂_{t+h|t} = π_t.
  - Autoregressive (AR) of order p: π̂_{t+h|t} = φ̂_{0,h} + φ̂_{1,h}·π_t + … + φ̂_{p,h}·π_{t-p+1}.
- Evaluation metric:
  - RMSE_{m,h} = √(1/(T−T0+1) ∑_{t=T0}^{T} ê_{t,m,h}^2 ), where ê_{t,m,h} = π_t − π̂_{t,m,h}.
  - RMSE calculated for each horizon during back testing (July – December 2023).

### Potential predictors and data transformations
- Feature categories (more than 20 features):
  - Inflation expectations: Household (BOJ opinion survey: Raw average, Imputed average), Corporate (Tankan), Consensus forecasts.
  - Price passthrough: USD-yen exchange rate, PPI, Chained PPI, PPI for services, Import price index, Energy import price.
  - Macroeconomics: GDP growth, Unemployment rate, BOJ output gap, HH disposable income, Consumer activity index, Earnings growth, Shadow interest rate, Credit growth, Gov. consumption, Gov. investment.
  - Other: TOPIX stock index, Tourist arrivals.
- Transformations:
  - All variables transformed to percentages; level variables converted to year-over-year percentage changes.
  - Three-month lags considered.
  - Core inflation adjusted for VAT increases and other one-off factors (e.g., go-to travel subsidies, policy-driven drop in mobile phone charges).

### Model performance and key quantitative results

### Back-testing results (July – December 2023)
- LASSO achieves the lowest RMSE among models for all three forecasting horizons.
- RMSE values (reported approximately):
  - 1-month ahead: about 0.1 percentage point
  - 3-month ahead: about 0.2 percentage point
  - 6-month ahead: about 0.3 percentage point
- Elastic Net: slightly higher RMSE than LASSO but still outperforms benchmarks (AR and RW).
- Random Forest and XGBoost: higher RMSE than benchmark models in this study; produced forecasts consistently below actual inflation with large deviations.
- RMSE increases with forecast horizon across all models.
- Formal statistical tests:
  - Giacomini and Rossi’s fluctuation test indicates LASSO performs significantly better than benchmark models in most of 2022 and 2023.
  - LASSO out-performance more pronounced in the first half of 2023 and for longer horizons (6-month vs. 3-month or 1-month).

### Interpretability and key predictors identified by LASSO

### Features consistently selected with predictive power (beyond lagged inflation)
- Six features were consistently selected across training periods and horizons:
  - Lagged core inflation
  - Last quarter’s household inflation expectations (both simple average and imputed average)
  - Year-over-year growth of tourist arrivals
  - Year-over-year exchange rate depreciation
  - Output gap
- Directional effects:
  - An increase in any of those variables predicts higher core inflation in the near term (1 to 6 months ahead).

### Economic rationale
- Household inflation expectations (average level and distribution) help predict near-term inflation.
- Sharp yen exchange rate movements have predictive power via passthrough into PPI and CPI.
- Output gap captures aggregate demand slackness; unemployment was smoother due to government support.
- Recovery in inbound tourist arrivals since border reopening in October 2022 put pressure on services prices.

### Explanations for poor performance of tree-based models in this study
- Two main reasons:
  1. Bias-variance tradeoff: flexible nonparametric models reduce bias but increase variance, leading to overfitting when trained on a stable inflation history; higher variance limits generalization to new inflationary periods.
  2. Lack of clear nonlinear relationships in Japan’s core inflation: core inflation rose linearly since 2022 before flattening; no evidence of strong nonlinear predictor effects in this context.

### Conclusions, implications, and practical recommendations

### Main conclusions
- LASSO, a linear ML model with regularization and variable selection, accurately forecasts near-term core inflation in Japan (1 to 6 months ahead) and outperforms other ML and standard benchmark models in the study period.
- Reported average RMSE for LASSO: about 0.1, 0.2 and 0.3 percentage point for 1-month, 3-month, and 6-month ahead forecasts, respectively.
- LASSO’s interpretability identifies household inflation expectations, output gap, dollar-yen exchange rate, and tourist arrivals as leading short-term predictors.

### Practical recommendations
- Retrain models with each new data point to capture evolving inflation dynamics.
- Combine ML flexibility with domain knowledge—include as many relevant features as possible, including nonstandard predictors like tourist arrivals where justified.
- Reassess ML model performance for inflation forecasting across other advanced economies given post-2020 shifts in inflation dynamics.

*IMF Working Paper — Mending the Crystal Ball: Enhanced Inflation Forecasts with Machine Learning*

### Introduction ...........................................................................................................

### wpiea2024206-print-pdf - Introduction ...........................................................................................................

### Introduction
- Japan experienced inflation above the two-percent target for more than two years since April 2022, the first time in three decades.
- Inflation drivers:
  - Initially dominated by cost-push factors.
  - Increasingly driven by demand-side factors, with services inflation above 2 percent.
- Forecasting challenges:
  - The Bank of Japan (BOJ) repeatedly revised upward its quarterly economic outlook; the level and persistence of core inflation surprised the BOJ.
  - Many central banks and the World Economic Outlook under-projected inflation in 2021–22 (see Koch and Noureldin, 2023).
  - Structural changes from the pandemic (e.g., persistent supply chain constraints, steepening Phillips curves) were not captured by prevailing models.
  - Three decades of disinflation in Japan complicate forecasters’ tasks.
- Rationale for machine learning (ML):
  - Traditional forecasting models assume a pre-specified stochastic relationship and can only be as good as their specification (Breiman, 2001).
  - ML models find a function that best represents the relationship between input and output data using rigorous training and testing algorithms.
  - Advantages of ML over economic forecasting models:
    - Can incorporate a wider range of variables via regularization and handle non-linear relationships, helping capture new pandemic-era drivers (Kohlscheen, 2021; Medeiros, 2021).
    - Emphasize prediction with rigorous selection of best-performing models; focus on out-of-sample forecasting performance (Mullainathan and Spiess, 2017).
    - Can be easily re-trained with latest data and adapt to changing data patterns.
- Scope of the paper:
  - First application of ML to inflation forecasting in Japan, targeting core inflation excluding fresh food and energy (VAT rate hikes excluded as one-off policy effects).
  - Forecasting horizons chosen to maximize training samples with inflationary pressure: one-month, three-month and six-month ahead.
  - Models considered: linear ML models (LASSO, Elastic Net) and tree-based ML models (Xgboost and Random Forest).
  - Benchmarks: simple univariate forecasting models (random walk and autoregressive), which are difficult to systematically outperform (Faust and Wright, 2013).
- Key empirical result (back testing, post 2022):
  - Among six models, LASSO performs best for core inflation forecasts.
  - Average root mean squared error (RMSE) of LASSO forecasts: about 0.1, 0.2 and 0.3 percentage point for 1-month, 3-month, and 6-month ahead inflation forecasts, respectively.
  - Formal tests indicate LASSO performs significantly better than the benchmark models.
  - Forecasting errors of LASSO appear smaller than professional forecasters’ errors in 2022–23:
    - Consensus forecast bias: averaging -0.6 percentage points for 3-month ahead inflation and -1.2 percentage points for 6-month ahead inflation.
  - Semi-structural BOJ model and forecasters calibrated to pre-pandemic data missed inflation in 2022 and 2023.
- LASSO-selected predictors with high predictive power (other than lagged inflation):
  - Rising household inflation expectations (measured by simple average and imputed qualitative responses in the BOJ opinion survey).
  - Inbound tourism surge since border reopening in October 2022.
  - Sharp yen depreciation since early 2022 (gradually passed on to consumers).
  - Closing output gap as the economy recovers from the pandemic.

### Methodology
- Forecasting framework:
  - Model: π_{t+h} = F_h(x_t) + u_{t+h}, h = 1,...,H, t = 1,...,T.
    - π_{t+h} is core inflation (excluding fresh food and energy) in month t+h.
    - x_t = (x_{1t},...,x_{nt})′ is an n-vector of covariates possibly containing lags of π_t and a large set of potential predictors.
    - F_h(.) is the mapping between covariates and future inflation; u_{t+h} is zero-mean random error.
    - Different mapping for each forecasting horizon h (h = 1, 3, or 6).
    - More than twenty numerical features considered, including economic activity indicators, exchange rates, import prices and tourist arrivals.
  - Forecast equation: π̂_{t+h|t} = F̂_{h,t−R_h+1:t}(x_t), where F̂ is estimated using data from time t−R_h+1 to t and R_h is the window size (varies by horizon and number of lags).
  - Direct forecasts are used (covariates are not predicted).
- Training, validation, and selection:
  - Cross validation on monthly data since 2012 using time-split K-fold cross-validation to preserve temporal dependency and avoid leakage.
    - The k-th split treats the first k folds as training set and the (k+1)-th fold as test set; successive training sets are supersets of previous ones.
    - Hyperparameters chosen to minimize average RMSE across test folds.
  - Back-testing using observations in July-December 2023:
    - Forecast core inflation 1, 3, or 6 months ahead using each model.
    - Calculate RMSE over back-testing period; select model with lowest RMSE to forecast inflation.
  - Rolling-window framework (Giacomini and White, 2006):
    - With each new data point, cross-validation sample expands by one period; back-testing sample shifts forward by one period.
    - Framework allows adaptation to recent data and attenuates effects of structural breaks.
- Models considered (details in Annex I):
  1. LASSO (Least Absolute Shrinkage and Selection Operator): linear regression with L1 regularization for sparsity and feature selection (Tibshirani, 1996).
  2. Elastic net: combines LASSO and Ridge with both L1 and L2 penalties; better handles highly correlated variables and reduces overfitting relative to LASSO (Zou and Hastie, 2005).
  3. Random Forest (RF): bagging method for regression trees (Breiman, 2001).
     - Multiple bootstrap samples; a regression tree built per bootstrap sample.
     - Each tree uses a random subset of features to determine best splits.
     - Produces feature importance via average decrease in impurity (variance) when a feature is used.
     - RF can describe nonlinear mappings nonparametrically and has produced superior inflation forecasts in some cases (Medeiros et al, 2019; Araujo and Gaglianone, 2023).
  4. Xgboost (tree-based ensemble) — mentioned in model list but detailed in Annex I.

### Key procedural details and data notes
- Forecasting horizons: h = 1, 3, or 6 months ahead.
- Monthly data used since 2012 for training and hyper-parameter tuning.
- Back-testing period: July-December 2023.
- Target series: core inflation excluding fresh food and energy; VAT rate hike effects excluded as one-off policy-driven effects.
- Model selection criterion: lowest RMSE in cross-validation and lowest RMSE in back-testing.

_Italic: Content derived from Introduction and Methodology sections of the IMF working paper chapter "Mending the Crystal Ball: Enhanced Inflation Forecasts with Machine Learning."_

### 4. Gradient boosting is a type of boosting method used with regression trees (see Friedman, 2001).

### 4. Gradient boosting is a type of boosting method used with regression trees (see Friedman, 2001).

### Machine learning methods and implementation
- Gradient boosting: a boosting method that trains a sequence of models adaptively so each model compensates for errors of preceding ones; subsequent models focus on instances the ensemble finds difficult to classify.
- XGBoost: used in this paper, designed and developed by Chen and Guestrin (2016).
- Other ML methods used:
  - LASSO (linear_model.LassoCV from sklearn)
  - Elastic Net (linear_model.ElasticNetCV from sklearn)
  - Random Forest (RandomForestRegressor from sklearn.ensemble)
  - XGBoost (xgboost.XGBRegressor)
- Hyperparameter tuning:
  - Bayesian Search algorithm (Wu et al. 2019) / BayesSearchCV from skopt for tree-based models.
  - TimeSeriesSplit from sklearn.model_selection with 10 folds (n_splits=10) for cross validation.
- Implementation environment: Python with standard Python ML packages (Pedregosa et al. (2011) and Chen and Guestrin (2016)).
- Models retrained under a rolling-window framework (examples: one-month ahead forecast in July 2023 uses data until June 2023; three-month ahead in July 2023 uses data until April 2023).

### Benchmarks and forecast setup
- Benchmarks:
  - Random walk (RW): forecasts 휋̂_{t+h|t} = 휋_t.
  - Autoregressive (AR) model of order p: 휋̂_{t+h|t} = 휙̂_{0,h} + 휙̂_{1,h}·휋_t + … + 휙̂_{p,h}·휋_{t-p+1} (different model for each horizon h).
- Forecast evaluation metric:
  - Root mean squared errors (RMSE) defined as:
    RMSE_{m,h} = √(1/(T−T0+1) ∑_{t=T0}^{T} ê_{t,m,h}^2 ), where ê_{t,m,h} = π_t − π̂_{t,m,h}.
  - RMSE calculated for each forecasting horizon during back testing period (July – December 2023).

### Potential predictors (features) and data transformations
- Feature set (more than 20 features); categorized in Table 1:
  - Inflation expectations:
    - Household (BOJ opinion survey): Raw average, Imputed average from qualitative responses
    - Corporate (Tankan)
    - Consensus forecasts
  - Price passthrough:
    - USD-yen exch. rate
    - PPI
    - Chained PPI
    - PPI for services
    - Import price index
    - Energy import price
  - Macroeconomics:
    - GDP growth
    - Unemployment rate
    - BOJ output gap
    - HH disposable income
    - Consumer activity index
    - Earnings growth
    - Shadow interest rate
    - Credit growth
    - Gov. consumption
    - Gov. investment
  - Other:
    - TOPIX stock index
    - Tourist arrivals
- Data transformations applied:
  - All variables transformed to percentages; for variables in levels, year-over-year percentage changes computed.
  - Three-month lags of variables considered.
  - Core inflation adjusted for VAT increases and other one-off factors (e.g., go-to travel subsidies, policy-driven drop in mobile phone charges).

### Model performance and key quantitative results
- Models evaluated: four ML models (LASSO, Elastic Net, Random Forest, XGBoost) and two benchmarks (AR, RW).
- Back-testing results (July – December 2023):
  - LASSO achieves the lowest RMSE among models for all three forecasting horizons.
  - RMSE values (approximately):
    - 1-month ahead: about 0.1 percentage point
    - 3-month ahead: about 0.2 percentage point
    - 6-month ahead: about 0.3 percentage point
  - Elastic Net: slightly higher RMSE than LASSO but still outperforms benchmarks (AR and RW).
  - Random Forest and XGBoost: higher RMSE than benchmark models in this study; produced forecasts consistently below actual inflation with large deviations.
  - RMSE increases with forecasting horizon across all models.
- Formal tests:
  - Giacomini and Rossi’s fluctuation test (rolling-window Diebold-Mariano style) indicates LASSO performs significantly better than benchmark models in most of 2022 and 2023.
  - The out-performance of LASSO is more pronounced in the first half of 2023 and for longer horizons (6-month vs. 3-month or 1-month).

### Interpretability and key predictors identified by LASSO
- LASSO provides interpretability and automatic variable selection; six features were consistently selected across training periods and horizons:
  - Lagged core inflation
  - Last quarter’s household inflation expectations (both simple average and imputed average)
  - Year-over-year growth of tourist arrivals
  - Year-over-year exchange rate depreciation
  - Output gap
- Directional effect:
  - An increase in any of those variables predicts higher core inflation in the near term (1 to 6 months ahead).
- Economic rationale and alignment with literature:
  - Household inflation expectations (average level and distribution) help predict near-term inflation.
  - Sharp yen exchange rate movements have predictive power via passthrough into PPI and CPI.
  - Output gap captures aggregate demand slackness; unemployment rate was smoother due to government support.
  - Recovery in inbound tourist arrivals since border reopening in October 2022 put pressure on services prices.

### Explanations for poor performance of tree-based models in this study
- Two main reasons why Random Forest and XGBoost underperformed here:
  1. Bias-variance tradeoff: flexible nonparametric models reduce bias but increase variance, leading to overfitting when trained on a stable inflation history; higher variance limits generalization to new inflationary periods.
  2. Lack of clear nonlinear relationships in Japan’s core inflation: core inflation rose linearly since 2022 before flattening; no evidence of strong nonlinear predictor effects in this context.

### Conclusions and implications for forecasting practice
- Main findings:
  - LASSO, a linear ML model with variable selection and regularization, accurately forecasts near-term core inflation in Japan (1 to 6 months ahead) and outperforms other ML and standard benchmark models in the study period.
  - Average RMSE is about 0.1, 0.2 and 0.3 percentage point for 1-month, 3-month, and 6-month ahead forecasts, respectively.
  - LASSO’s interpretability identifies household inflation expectations, output gap, dollar-yen exchange rate, and tourist arrivals as current leading predictors.
- Practical recommendations:
  - Retrain models with each new data point to capture evolving inflation dynamics.
  - Combine ML flexibility with domain knowledge—include as many relevant features as possible, including nonstandard predictors like tourist arrivals where justified by domain knowledge.
  - Reassess ML model performance for inflation forecasting across other advanced economies given post-2020 shifts in inflation dynamics.
- Broader implication:
  - ML models, especially interpretable linear regularized models like LASSO, can offer improved short-term inflation forecasts and insights into drivers of inflation compared with traditional econometric approaches.

### Annex I — Descriptions of linear ML methods used
- LASSO (Least Absolute Shrinkage and Selection Operator):
  - L1 regularization that adds a penalty term 휆_1‖β‖_1 to the residual sum of squares.
  - Promotes sparsity and automatic feature selection; optimal 휆 determined using time-split K-fold cross-validation.
  - Handles some multicollinearity but may arbitrarily drop one of highly correlated features.
- Elastic Net:
  - Combines L1 and L2 penalties (휆_1‖β‖_1 + 휆_2‖β‖_2).
  - Addresses multicollinearity while enabling feature selection.
  - Potentially superior to LASSO when: 1) number of regressors exceeds number of observations (“fat data”); 2) groups of variables show high pairwise correlation; 3) number of observations significantly exceeds number of regressors (“tall data”).

*IMF Working Paper — Mending the Crystal Ball: Enhanced Inflation Forecasts with Machine Learning*

### 3. Random Forest.

### 3. Random Forest.

### Overview of the method
- Random forest is a commonly-used machine learning algorithm that combines the output of multiple decision trees to reach a single result.
- Each tree in the ensemble is comprised of a data sample drawn from a training set with replacement, called the bootstrap sample.
- The algorithm has several main hyperparameters that need to be set before training, including tree depth, node size, the number of trees, and the number of features sampled.
- From there, the random forest classifier can be used to solve for regression or classification problems.
- Random forest can reduce the risk of overfitting (compared to decision trees) and can evaluate variable importance to the model.
- The method tends to work better with large datasets and is more complex and harder to interpret.

### Key properties and comparisons with boosting (XGBoost / GBDT)
- Both random forest and GBDT build a model consisting of multiple decision trees. The difference is in how the trees are built and combined.
  - Random forest uses a technique called bagging to build full decision trees in parallel from random bootstrap samples of the data set.
  - GBDTs iteratively train an ensemble of shallow decision trees, with each iteration using the error residuals of the previous model to fit the next model.
- Compared to random forest, gradient boosting can be more accurate and powerful since it uses gradient descent and residuals to optimize the ensemble and reduce the bias.
- Gradient boosting can also be slower and harder to train, more prone to overfitting, and more sensitive to outliers and noise.
- XGBoost specifics:
  - XGBoost is a scalable and highly accurate implementation of gradient boosting that pushes the limits of computing power for boosted tree algorithms.
  - With XGBoost, trees are built in parallel, instead of sequentially like Gradient Boosting Decision Trees (GBDT).
  - It follows a level-wise strategy, scanning across gradient values and using these partial sums to evaluate the quality of splits at every possible split in the training set.

### Implementation details (Python)
- Cross-validation for all four ML methods:
  - TimeSeriesSplit from sklean.model_selection with 10 folds (n_splits=10), a cross-validation technique tailored for time-series data.
- LASSO:
  - Use linear_model.LassoCV from sklearn.
  - LassoCV selects the best value of the hyperparameter α (the regularization strength) via built-in cross-validation.
  - In this implementation, the range of α values is not explicitly specified; LassoCV infers a range of values based on the data.
  - The α that results in the lowest mean squared error during cross-validation is selected as the final model.
- Elastic Net:
  - Use linear_model.ElasticNetCV from sklearn.
  - Two parameters to specify: α and l1_ratio (mix ratio between L1 and L2 regularization).
  - The range of l1_ratio chosen is [.1, .5, .7, .9, .95, .99].
  - For each combination of α and l1_ratio, ElasticNetCV trains and validates models; the combination with the lowest mean squared error during cross-validation is selected.
  - Both LassoCV and ElasticNetCV use coordinate descent algorithm to fit models along the regularization path.

### Random Forest implementation and hyperparameter tuning
- RandomForestRegressor from sklearn.ensemble is used in combination with BayesSearchCV from skopt.
- RandomForestRegressor’s splitting criterion is defined as "squared_error", which indicates the algorithm will choose splits that minimize the mean squared error within each node.
- BayesSearchCV details:
  - Bayesian optimization implementation that looks for the best hyperparameters that maximizes the RMSE score.
  - Unlike grid search or random search, BayesSearchCV makes decisions sequentially using results of previous evaluations.
- Search space specified for Random Forest:
  - max_depth, ranging from 2 to 20.
  - min_samples_leaf, ranging from 1 to 10.
  - Min_samples_leaf indicates every single tree in the random forest model is grown up until there are 1 to 10 observations in every leaf.
  - Max_depth indicates that the maximum depth of the trees is from 2 to 20.

### XGBoost implementation and hyperparameter tuning
- XGBoost is fitted from the library xgboost.XGBRegressor.
- Hyperparameters are tuned using BayesSearchCV in combination with TimeSeriesSplit tailored for time series data.
- The scoring method in cross-validation is the RMSE.
- Search space for XGBoost regressor includes:
  - max_depth: Maximum depth of the individual trees.
  - learning_rate: Boosting learning rate.
  - subsample: Proportion of training data to sample in each boosting round.
  - reg_lambda and reg_alpha: L2 and L1 regularization terms respectively.
  - gamma: Minimum loss reduction to make a split.
  - colsample_bytree and colsample_bylevel: Fraction of features to be randomly sampled for building trees and for each level respectively.
  - n_estimators: Number of boosting rounds (trees to be constructed).
- For each optimization in BayesSearchCV, we run 30 iterations (n_iter=30).

*Source: IMF Working Paper chapter "3. Random Forest."*

---


_Source: https://www.imf.org/-/media/files/publications/wp/2024/english/wpiea2024206-print-pdf.pdf_