## wpiea2021263-print-pdf

## Source details

**Canonical URL:** [wpiea2021263-print-pdf](https://www.imf.org/-/media/files/publications/wp/2021/english/wpiea2021263-print-pdf.pdf)

## Other formats

- [Markdown version](/-/media/files/publications/wp/2021/english/wpiea2021263-print-pdf.pdf.md)
- [Structured JSON version](/-/media/files/publications/wp/2021/english/wpiea2021263-print-pdf.pdf.json)

---

### Key findings and forecast performance
- Preferred forecasting model: machine learning Random Forest predicting unrest one year ahead with a balanced accuracy level of 66 percent (correct in predicting unrest approximately two-thirds of the time).
- Model scope:
  - Draws on over 340 indicators.
  - Covers 125 countries.
- Most important predictors (via Shapley values):
  - high current and prior levels of unrest,
  - food price inflation,
  - mobile phone penetration,
  - unrest in neighboring states.
- Empirical relationships with macro outcomes and political change:
  - Unconditional relationship: a 0.5 percentage point decline in per capita GDP around unrest events in the sample.
  - Instrumented estimate (Hlatshwayo and Redl (2020)): these events are associated with a decline in GDP of around 1 percent over a three year period.
  - Hadzi-Vaskov et al. (2021): GDP declines by 0.2 percentage points following an unrest shock.

### Data sources, event definition, and classification
- Unrest events measured using the newspaper-based Reported Social Unrest Index (RSUI) of Barrett et al. (2020), drawn from the Dow Jones Factiva news aggregator.
- RSUI construction notes:
  - RSUI is a monthly count of articles that include the country’s name where the words “protest” or “riot” or “revolution” within 10 words of “unrest” scaled by the total number of articles in the period.
  - Articles must exclude false-positive terms (examples listed), be tagged by Dow Jones as Domestic Politics or Civil Unrest, have more than 100 words, and the location tag must match the country name.
  - An unrest event is defined as a local monthly peak where more than 10 percent of articles are on the topic of unrest and meeting top-2-percent/4-standard-deviation thresholds for that country.
- Event screening and quality:
  - Of 679 candidate events, 15 percent (99 of 679) failed hand-check screening.
  - 69 percent of those failed events were corrected by systematic changes to search terms, yielding a final mis-identification rate of 4.6 percent.
- Manual classification into seven non-exclusive event categories; shares of events (percent):
  - Government: 17.5
  - Democratic-reform related: 9.4
  - Global issues: 3.6
  - Religious: 0.6
  - Elections: 14.0
  - Basic needs: 8.3
  - Coup/Sudden End to Tenure: 5.7
  - Violence: 4.9
  - Unknown: 36.0

### Predictors and variable construction
- Predictor coverage (selected categories):
  - Macro-financial: GDP level and growth, exchange rates, terms of trade, foreign reserves, fiscal balance, aid receipts, remittances, population, volatility measures, lags and differences.
  - Political/institutional: International Country Risk Guide, CNTSD indicators, Polity IV scores, executive selection, parliamentary responsibility, head of state type, military political activity, assassinations, political violence, mass strikes, prison population measures.
  - Socioeconomic and development: poverty (poverty gap at $5.5/day), prevalence of food insecurity, multidimensional poverty, access to electricity/sanitation/healthcare, Gini indices, income share of highest 10 percent, financial access of poorest 40 percent, gender inequality in schooling and literacy, urbanization and population density, share of international migrants, unemployment and youth unemployment by gender.
  - Information/coordination proxies: internet penetration, mobile phone penetration, televisions per capita, newspaper circulation.
  - Price measures: annual maximum of year-on-year monthly CPI inflation figures for various categories from the IMF CPI database (including food inflation).
  - Natural disasters: EM-DAT disaster counts and people affected (drought, earthquake, epidemic, extreme temperatures, floods, industrial accidents, storms).
  - Policy uncertainty: Economist Intelligence Unit–based text uncertainty measure (Ahir et al. (2018)) covering more than 140 countries from 1996 onwards.
  - Structural reform narrative indicators: Duval et al. (2018) indicators for product market and labor market reforms.
- Transformations used: growth rates, first differences, lags, volatility measures, annual maxima for inflation categories.
- Appendix II sample variable indices and names (selected):
  - 0 AE Dummy — Dummy
  - 48 Oil Price Inflation — Oil Inflation
  - 51 Inflation — Inflation
  - 52 Per Capita GDP Growth — GDP Growth
  - 59 Prim. Bal./GDP — Primary Balance
  - 118 GDP, USD, log — Size of Economy
  - 142 Reported Social Unrest Index — Reported Social Unrest Index
  - 148 Country Size in Square Kilometers — Size of Economy
  - 171 Registered Voters/Population (Scaling: 0.001) — Political
  - 198 Prison Population, Total — Prisons
  - 201 Current Account Risk (% GDP) — GDP Growth
  - 240 Prevalence of moderate or severe food insecurity — Poverty
  - 273 Drought — Natural Disasters
  - 281 World Uncertainty Index — Uncertainty
  - 287 Alcoholic Beverages, Tobacco, and Narcotics_MAX — Inflation
  - 300 Domestic finance - Overall — Structural Reforms
  - Variable list continues through indices up to 343.

### Model approach and interpretation
- Forecast target: binary indicator of whether an unrest event occurs in the following year (annual frequency).
- Modeling strategy: flexible horse-race across algorithms including regularized logistic regression, neural networks, support vector machines (SVM), and tree-based models (AdaBoost, Gradient Boosted Trees, Random Forest).
- Rationale for annual target: reduces high-frequency noise and mitigates monthly timing errors.
- Feature importance:
  - Shapley value analysis indicates current-year unrest levels dominate predictive power; additional meaningful contributions from food price inflation, mobile phone penetration, and neighboring-state unrest.

### Machine learning models considered (overview and key settings)
- Model groups:
  - Linear regularized logistic models: Ridge (ρ = 0) and Lasso (ρ = 1) logistic regression; assumed C = 1.
  - Non-linear: Neural Network and SVM.
    - Neural Network: loss L(y, ŷ, W) = −y ln ŷ − (1−y) ln(1−ŷ) + α ||W||^2_2 with α = 0.0001; architecture Am = 3 layer network with grid search over node counts; optimal parameters: 5,10 and 10 for the three respective layers.
    - SVM: kernel K(x,x') = exp( −γ ||x − x'||^2 ); grid searched C = {0.1,1,10,100,1000,10000}, γ = {0.001,0.01,0.1,1,10,100}; optimal parameters: {C, γ} = {1, 0.001}.
  - Tree-based and ensemble:
    - Random Forest: grid search max depth ∈ {1,3,5}, max features ∈ {50,100,200}; optimal parameters reported: max tree depth of 3 and max features of 200.
    - AdaBoost: considered number of estimators ranging from 50 to 300 in increments of 25; learning rate considered from 0.1 to 0.6 in increments of 0.1; reported optimal number of estimators = 50 for one specification.
    - Gradient Boosted Trees: hyperparameter grids evaluated included Number of estimators: {30,50,100,200}; Learning rate: {0.1,0.2,0.5}; Subsample: {0.8,1}; Max depth of trees: {1,3}. One statement: "We found the optimal performance when the number of estimators is 50 and learning rate 0.1." Later reported optimal combination: {30,0.1,0.8,1} (interpreted as number of estimators = 30; learning rate = 0.1; subsample = 0.8; max depth = 1).

### Model evaluation and performance (test set: 23 out-of-sample years, 1996-2019)
- Model AUC comparisons:
  - Lasso logistic model: AUC = 0.55.
  - Ridge logistic model: AUC = 0.55.
  - Neural net model: AUC = 0.54.
  - SVM model (radial basis function kernel with favored parameterization more linear): performs significantly better than the linear and neural net models.
  - Tree-based models outperform other models; Random Forest possesses the highest mean θ̂ and is the preferred forecasting model.
- Preferred Random Forest model performance (threshold chosen to minimize average of missed unrest events and false alarms for each out-of-sample prediction):
  - Average probability of an unrest event in a country in a year = 16.1%.
  - Average threshold selected by the procedure = 16.8%.
  - Balanced accuracy = 65.9%.
  - Recall (TP / (TP + FN)) = 70.9%.
  - Precision (TP / (TP + FP)) = 30.8%.
  - ROC trade-off: to achieve true positive rates in the 60-80% region requires accepting a relatively high false positive rate (40-60%).
- Temporal and event-type variation:
  - Model performance highest in the early 2000s and late 2010s; worst performance around 2009-10 (structural break due to the Arab Spring).
  - Performance by unrest type: similar across most categories except weaker for violent unrest and coups; violent unrest is rarer and less predictable.

### Feature importance: Shapley values and time variation
- Shapley methodology:
  - Implemented via Lundberg and Lee (2017).
  - Model fitted on initial sample 1990-1995; Shapley values calculated for 1996; rolled forward year-by-year to 2019.
  - Absolute values of Shapley contributions averaged over test sets; φ_S_0 (average prediction) = 16 percent for all years and countries.
- Key average contributions:
  - Reported Social Unrest Index (RSUI, including lags) adds on average around 6.5 percentage points to the average prediction.
  - Next most important groups relate to inflation, with food and oil inflation broken out separately.
  - Contagion (unrest in neighboring states) contributes to predictions.
  - Digital and Media (internet, television, newspaper, telephone, mobile phone usage) contribute to predictions.
- Time variation examples:
  - Large role for food price inflation during the sharp increase in unrest following the Arab Spring of 2011.
  - Contagion from neighbors became more important around the Arab Spring period.
  - In the 2010s: overall inflation remained important; role of food prices declined while oil prices' importance increased.

### Social Unrest Risk Index: patterns and country illustrations
- Risk index properties:
  - The model produces a probability of unrest for each country-year; treated as a risk index.
  - Average risk for the sample = 16 percent.
  - Lower risk in first half of sample (~12 percent), rising after the Arab Spring and maintaining elevated RSUI levels in the 2010s closer to 20 percent.
  - Distribution shifts:
    - 1990s: majority of countries low risk with a long right tail.
    - By 2010: risks rose in the tail and a bi-modal profile emerged.
    - After 2015: bi-modal distribution persists with the mode to the right of the mean and a larger incidence of high-risk countries.
  - Overall narrative: unrest moved from a low probability tail risk in the 1990s (around 10 percent on average) to a relatively likely outcome by the end of the 2010s (around 25 percent).
- Country case illustrations (risk index aligned one year before unrest events):
  - United Kingdom: low risk prior to last decade, then rising; elevated risk from 2012 foreshadowing frequent protests thereafter. Model did not warn global Occupy events but indicated elevated risk from 2012.
  - United States: risk index rose prior to labeled events (election of President Donald Trump and protests against police violence) in 2015 and remained high thereafter.
  - Egypt: model did not pick up the risk of the Tahrir events/Arab Spring structural break; model generally performs poorly around that break.
  - Thailand: model performed better around events leading up to the 2013 coup d’etat with a high and rising risk index (excluding 2012).

### Policy relevance and implications
- Risks from social unrest:
  - Social unrest poses risks to financial, economic, and political stability.
  - Equity valuations decline around unrest events (high-frequency evidence); firms connected to administrations in power can experience valuation losses.
  - Real economic impacts are non-trivial (see GDP decline estimates above).
- Predictive monitoring:
  - A 1-year-ahead probabilistic unrest risk index based on this model can inform surveillance and policy preparedness given the model’s balanced accuracy of 66 percent.
- Drivers highlighted for policy attention:
  - Monitoring food price inflation and food security metrics.
  - Tracking digital connectivity metrics (mobile phone and internet penetration) for their role in coordination and information diffusion.
  - Paying attention to recent and neighboring-country unrest as strong predictors of future unrest.
  - Considering socioeconomic stressors (poverty, unemployment, access to services) and natural disaster exposure as potential catalysts.
- Policy-relevant suggestions and future work:
  - Incorporate social media data and higher-frequency inflation data to build higher-frequency predictions.
  - Use the risk index as a monitoring tool given the rise in average unrest risk from ~10 percent in the early 1990s to ~25 percent by the late 2010s.

*Source: wpiea2021263-print-pdf*

### 2020. The risk of social unrest is based on the probability of unrest in the following

### 2020. The risk of social unrest is based on the probability of unrest in the following year derived from a machine learning model drawing on over 340 indicators covering a wide range of macro-financial, socioeconomic, development and political variables.

### Key findings and forecast performance
- The preferred machine learning forecasting model predicts unrest one year ahead with a balanced accuracy level of 66 percent (correct in predicting unrest approximately two-thirds of the time).
- The model draws on over 340 indicators and covers a broad set of countries (125).
- The most important predictors (via Shapley values) include:
  - high current and prior levels of unrest,
  - food price inflation,
  - mobile phone penetration,
  - unrest in neighboring states.
- Empirical relationships with macro outcomes and political change:
  - An unconditional relationship shows a 0.5 percentage point decline in per capita GDP around unrest events in the sample.
  - Using neighboring-state variation as an instrument, Hlatshwayo and Redl (2020) find these events are associated with a decline in GDP of around 1 percent over a three year period.
  - Hadzi-Vaskov et al. (2021) find GDP declines by 0.2 percentage points following an unrest shock.

### Data sources, event definition, and classification
- Unrest events are measured using the newspaper-based Reported Social Unrest Index (RSUI) of Barrett et al. (2020), drawn from the Dow Jones Factiva news aggregator.
- RSUI construction notes:
  - RSUI is a monthly count of articles that include the country’s name where the words “protest” or “riot” or “revolution” within 10 words of “unrest” scaled by the total number of articles in the period.
  - Articles must exclude false-positive terms (examples listed), be tagged by Dow Jones as Domestic Politics or Civil Unrest, have more than 100 words, and the location tag must match the country name.
  - An unrest event is defined as a local monthly peak where more than 10 percent of articles are on the topic of unrest and meeting top-2-percent/4-standard-deviation thresholds for that country.
- Event screening and quality:
  - Of 679 candidate events, 15 percent (99 of 679) failed hand-check screening.
  - 69 percent of those failed events were corrected by systematic changes to search terms, yielding a final mis-identification rate of 4.6 percent.
- Manual classification into seven non-exclusive event categories based on key words; shares of events (percent):
  - Government: 17.5
  - Democratic-reform related: 9.4
  - Global issues: 3.6
  - Religious: 0.6
  - Elections: 14.0
  - Basic needs: 8.3
  - Coup/Sudden End to Tenure: 5.7
  - Violence: 4.9
  - Unknown: 36.0

### Predictors and variable construction
- Predictor coverage:
  - Macro-financial: GDP level and growth, exchange rates, terms of trade, foreign reserves, fiscal balance, aid receipts, remittances, population, volatility measures, lags and differences.
  - Political/institutional: International Country Risk Guide, CNTSD indicators (legislative/executive effectiveness, government cohesion, legitimacy, bureaucratic quality), Polity IV scores, executive selection, parliamentary responsibility, head of state type, military political activity, assassinations, political violence, mass strikes, prison population measures.
  - Socioeconomic and development: poverty (poverty gap at $5.5/day), prevalence of food insecurity, multidimensional poverty, access to electricity/sanitation/healthcare, Gini indices, income share of highest 10 percent, financial access of poorest 40 percent, gender inequality in schooling and literacy, urbanization and population density, share of international migrants, unemployment and youth unemployment by gender.
  - Information/coordination proxies: internet penetration, mobile phone penetration, televisions per capita, newspaper circulation.
  - Price measures: annual maximum of year-on-year monthly CPI inflation figures for various categories from the IMF CPI database (including food inflation).
  - Natural disasters: EM-DAT disaster counts and people affected (drought, earthquake, epidemic, extreme temperatures, floods, industrial accidents, storms).
  - Policy uncertainty: Economist Intelligence Unit–based text uncertainty measure (Ahir et al. (2018)) covering more than 140 countries from 1996 onwards.
  - Structural reform narrative indicators: Duval et al. (2018) indicators for product market and labor market reforms.
- Transformations used: growth rates, first differences, lags, volatility measures, annual maxima for inflation categories.

### Model approach and interpretation
- Modeling strategy:
  - A flexible machine learning horse-race across a variety of algorithms (see appendix for models considered, including regularized logistic regression, neural networks, support vector machines, and tree-based models like AdaBoost and Gradient Boosted Trees).
  - Forecast target is a binary indicator of whether an unrest event occurs in the following year (annual frequency), rather than forecasting the continuous monthly RSUI.
- Feature importance:
  - Shapley value analysis indicates the dominant predictive power of current-year unrest levels, with additional meaningful contributions from food price inflation, mobile phone penetration, and neighboring-state unrest.
- Rationale for annual target:
  - Annual aggregation reduces high-frequency noise and mitigates monthly timing errors (e.g., article discussion of past events still yields an annual unrest flag).

### Policy relevance and implications
- Social unrest poses risks to financial, economic, and political stability:
  - Equity valuations decline around unrest events (high-frequency evidence), and firms connected to administrations in power can experience valuation losses.
  - Real economic impacts are non-trivial (examples above of GDP declines).
- Predictive monitoring:
  - A 1-year-ahead probabilistic unrest risk index based on this model can inform surveillance and policy preparedness given the model’s balanced accuracy of 66 percent.
- Drivers highlighted for policy attention:
  - Monitoring food price inflation and food security metrics.
  - Tracking digital connectivity metrics (mobile phone and internet penetration) for their role in coordination and information diffusion.
  - Paying attention to recent and neighboring-country unrest as strong predictors of future unrest.
  - Considering socioeconomic stressors (poverty, unemployment, access to services) and natural disaster exposure as potential catalysts.

*JEL Classification Numbers: C45, C53, P16 — Keywords: Social unrest, machine learning. Author’s E-Mail Address: CRedl@imf.org;SHlatshwayo@imf.org*

### 3.1  Machine Learning Models Considered

### 3.1  Machine Learning Models Considered

### Overview
- Forecasting task: predict unrest events.
- Models fall into three groups: linear (with logistic transformation and regularization), non-linear (neural net and support vector machine), and tree-based (decision trees, boosting, bagging, Random Forest).
- Formal model descriptions are provided in the appendix; this section provides intuition.

### Linear Regularized Logistic Models
- Use a logistic function to map linear model outputs to the range [0, 1].
- Regularization used to allow many predictors without over-fitting:
  - Ridge adds a quadratic penalty term (L2 norm).
  - Lasso adds an absolute value penalty (L1 norm).
- Modeling assumptions:
  - Ridge assumes the true model is dense (all predictors matter).
  - Lasso assumes the true model is sparse (only a few predictors are important).

### Non-linear Models: SVM and Neural Nets
- Support Vector Machine (SVM):
  - Classifies events using a non-linear decision boundary that aims to maximize the gap between groups of observations.
  - In the simplest case of linearly separable data, the SVM chooses parameters to maximize the area around the regression line where no target variables overlap.
  - Non-linearities can be captured via transformations of the original data (basis functions).
- Neural Networks:
  - Closely related to logistic regression but use outputs of logistic units as inputs to other logistic units (layering).
  - Layering enables capturing highly non-linear functions efficiently.
- Both model types have performed well on non-traditional data (text, image recognition).

### Tree-Based Models and Ensemble Methods
- Decision tree: partitions predictor space so membership in a region predicts the target (example: food price inflation cut-off such as 10%).
- Over-fitting issues addressed via ensemble methods:
  - Boosting (AdaBoost, Gradient Boost):
    - Build strong predictors from many weak (simple) models.
    - AdaBoost (Freund and Schapire (1997)) sequentially evaluates simple models, placing higher weight on models that perform well and on mis-classified data; final prediction is a weighted average.
    - Gradient boosting sequentially models the slope of the loss function to improve models (see Chen and Guestrin (2016)).
  - Bagging/Bootstrapping:
    - Fit many models on bootstrapped samples and average results to reduce variance and over-fitting.
  - Random Forest (Breiman (2001)):
    - Averages many de-correlated trees; each tree grown using a random subsample of available predictors.
    - Algorithm 1 (Random Forest Classification) begins by drawing a bootstrap sample Z from the N predictors in the training sample, X′ = (X1, X2, ..., XN).
- Empirical finding in this context:
  - Tree-based models perform best, with Random Forest providing the best overall model.
  - Random Forest performance is comparable to boosting methods.

### Model Evaluation and Performance (Section 3.2 content)
- Training/testing split: expanding window approach.
  - Initial training set: annual data from 1990 to 1995 to predict the test set of 1996.
  - Rolling the sample forward generates predictions up to the year 2019.
- Hyperparameter selection:
  - Chosen using a coarse grid based on average model performance over the full rolling window sample (up to 2019); see appendix I.
- Cross-validation considerations:
  - K-fold cross validation not pursued due to time dependence in the data and significant persistence in the RSUI measure.
  - Shao (1993) demonstrates asymptotic inconsistency for K-fold cross-validation with time dependence.
  - Alternatives (e.g., gap between testing and training set as in Racine (2000)) would consume too many degrees of freedom given the relatively short annual dataset.
- Toolkit note:
  - Uses Pedregosa et al. (2011) toolkit which, for decision trees with a discrete outcome, averages the probability of an event from each tree (rather than majority vote).

*Source: wpiea2021263-print-pdf - 3.1  Machine Learning Models Considered*

### 2. Grow a random-forest treeT

### 2. Grow a random-forest treeT

### Method: growing a random-forest tree
- Procedure (recursive until minimum node size n_min is reached):
  - Select m variables at random from the p variables available.
  - Pick the best variable and split point from among them using the CART algorithm (Breiman et al. (1984)):
    - Let the data at node m be partitioned into Q_left_m(θ) and Q_right_m(θ) where θ = (j,s) is a candidate split threshold(s) for feature (j). The split is computed from the loss function:
      - θ* = arg min_θ [∑_{k=left,right} n_k_m p_k_m (1 − p_k_m)], where n_k_m is the share of observations in region k, p_k_m is the share of unrest events in the region Q_k_m(θ) and k = left, right.
    - Repeat for subsets Q_left_m(θ*) and Q_right_m(θ*) until a single observation remains at node m.
  - Split the node into two daughter nodes.
- Output: a collection of B trees, {T_b}_{1..B}. The probability for a new data point x resulting in an unrest event is:
  - f̂_B_rf(x) = (1/B) ∑_{b=1}^B f̂_{T_b}(x), where f̂_{T_b}(x) is the share of unrest events for the terminal node associated with x for the b-th tree.

### Evaluation metric: AUC and DeLong test
- Test statistic: area under the Receiver Operator Characteristic (ROC) curve (AUC).
  - ROC plots true positive rate vs. false positive rate as classification threshold varies.
  - Benchmark: random guess (coin flip) reflected by AUC’s 45 degree line (AUC = 0.5).
- Empirical AUC (DeLong et al. (1988)):
  - θ̂ = (1/(N M)) ∑_{j=1}^M ∑_{i=1}^N H(X_i, Y_j), with H(X,Y) = 1 if Y < X; 0.5 if Y = X; 0 if Y > X.
  - Interpretation: equivalent to Mann-Whitney U test; estimates probability that a randomly selected event with no unrest has a risk score less than or equal to a randomly selected event with unrest.
- Model comparison uses DeLong test z-scores:
  - z = (θ̂_A − θ̂_B) / sqrt(Var[θ̂_A − θ̂_B]).
  - To test against random guess, use z = (θ̂_i − 0.5)/σ_i where σ_i is the standard deviation of θ̂ over the test set.

### Model performance (test set: 23 out-of-sample years, 1996-2019)
- Summary comparisons:
  - Lasso logistic model: AUC = 0.55.
  - Ridge logistic model: AUC = 0.55.
  - Neural net model: AUC = 0.54.
  - SVM model (radial basis function kernel with favored parameterization more linear): performs significantly better than the linear and neural net models.
  - Tree-based models outperform other models; Random Forest possesses the highest mean θ̂ and is the preferred forecasting model.
- Preferred Random Forest model performance (threshold chosen to minimize average of missed unrest events and false alarms for each out-of-sample prediction):
  - Average probability of an unrest event in a country in a year = 16.1%.
  - Average threshold selected by the procedure = 16.8%.
  - Balanced accuracy = 65.9%.
  - Recall (TP / (TP + FN)) = 70.9%.
  - Precision (TP / (TP + FP)) = 30.8%.
  - Trade-off illustrated by ROC: to achieve true positive rates in the 60-80% region requires accepting a relatively high false positive rate (40-60%).
- Temporal and event-type variation:
  - Model performance highest in the early 2000s and late 2010s; worst performance around 2009-10 (structural break due to the Arab Spring).
  - Performance by unrest type: similar across most categories except weaker for violent unrest and coups; violent unrest is rarer and less predictable.

### Feature importance: Shapley values
- Method:
  - Shapley decomposition applied via Lundberg and Lee (2017) implementation.
  - Model fitted on initial sample 1990-1995 and Shapley values calculated for 1996; rolling the sample forward year-by-year to 2019 to produce out-of-sample Shapley values.
  - Features aggregated into categories (mapping in appendix).
  - Absolute values of Shapley contributions averaged over test sets; φ_S_0 (average prediction) = 16 percent for all years and countries.
- Key findings (average contributions):
  - Reported Social Unrest Index (RSUI, including lags) is by far the most important category, adding on average around 6.5 percentage points to the average prediction.
  - Next most important groups relate to inflation, with food and oil inflation broken out separately.
  - Contagion (unrest in neighboring states) contributes to predictions.
  - Digital and Media (internet, television, newspaper, telephone, mobile phone usage) contribute to predictions.
- Time variation:
  - Influence of features can change abruptly (e.g., large role for food price inflation during the sharp increase in unrest following the Arab Spring of 2011).
  - Contagion from neighbors became more important around the Arab Spring period.
  - In the 2010s: overall inflation remained important; role of food prices declined while oil prices' importance increased.

### Social Unrest Risk Index
- The model produces a probability of unrest for each country-year; treated as a risk index.
- Aggregate patterns:
  - Average risk for the sample = 16 percent.
  - Lower risk in first half of sample (~12 percent), rising after the Arab Spring and maintaining elevated RSUI levels in the 2010s closer to 20 percent.
  - Distribution shifts:
    - 1990s: majority of countries low risk with a long right tail.
    - By 2010: risks rose in the tail and a bi-modal profile emerged.
    - After 2015: bi-modal distribution persists with the mode to the right of the mean and a larger incidence of high-risk countries.
  - Overall: unrest moved from a low probability tail risk in the 1990s (around 10 percent on average) to a relatively likely outcome by the end of the 2010s (around 25 percent).

- Country case illustrations (risk index aligned one year before unrest events):
  - United Kingdom: low risk prior to last decade, then rising; elevated risk from 2012 foreshadowing frequent protests thereafter. Global events include Occupy and Brexit; democratic event is Scottish independence referendum; basic needs unrest relates to anti-austerity protests. Model did not warn global Occupy events but indicated elevated risk from 2012.
  - United States: risk index rose prior to labeled events (election of President Donald Trump and protests against police violence) in 2015 and remained high thereafter.
  - Egypt: model did not pick up the risk of the Tahrir events/Arab Spring structural break; model generally performs poorly around that break.
  - Thailand: model performed better around events leading up to the 2013 coup d’etat with a high and rising risk index (excluding 2012).

### Conclusions and implications
- Social unrest increases near-term financial, economic and political risks; early warning systems can be useful for policymakers.
- Drivers of unrest are disparate and interact non-linearly across socioeconomic, environmental, political and macroeconomic conditions.
- Empirical model conclusions:
  - Linear models perform poorly in this application; highly complex models (neural nets) also underperform due to relatively small dataset.
  - Tree-based models, particularly Random Forest, offer an appropriate balance of flexibility and simplicity.
  - Preferred Random Forest: balanced accuracy ~66% and AUC = 65% (significantly better than chance per DeLong et al. (1988)).
- Feature findings align with prior literature: emphasis on food price inflation, media usage, contagion from neighboring states, and a strong auto-regressive component (past unrest predicts future unrest).
- Policy-relevant suggestions and future work:
  - Incorporate social media data and higher-frequency inflation data to build higher-frequency predictions.
  - Use the risk index as a monitoring tool given the rise in average unrest risk from ~10 percent in the early 1990s to ~25 percent by the late 2010s.

*Source: wpiea2021263-print-pdf - 2. Grow a random-forest treeT*

### References

### References

### Bibliographic citations
- Acemoglu, D., Fergusson, L., and Johnson, S. (2019). Population and Conflict. The Review of Economic Studies, 87(4):1565–1604.
- Acemoglu, D., Hassan, T. A., and Tahoun, A. (2017). The Power of the Street: Evidence from Egypts Arab Spring. The Review of Financial Studies, 31(1):1–42.
- Acemoglu, D. and Robinson, J. A. (2000). Why Did the West Extend the Franchise? Democracy, Inequality, and Growth in Historical Perspective. The Quarterly Journal of Economics, 115(4):1167–1199.
- Ackermann, K. (2017). Individual differences and political contexts the role of personality traits and direct democracy in explaining political protest. Swiss Political Science Review, 23:21–49.
- Ahir, H., Bloom, N., and Furceri, D. (2018). World uncertainty index. mimeo.
- Aidt, T. S. and Franck, R. (2015). Democratization under the threat of revolution: Evidence from the great reform act of 1832. Econometrica, 83(2):505–547.
- Aidt, T. S. and Leon, G. (2016). The democratic window of opportunity: Evidence from riots in sub-saharan africa. Journal of Conflict Resolution, 60(4):694–717.
- Arezki, R., Adessé Dama, A., Djankov, S., and Nguyen, H. (2020). Contagious protests. World Bank Policy Research Working Paper 9321.
- Banks, A. and Wilson, K. (2020). Cross-national time-series data archive.
- Barrett, P., Appendino, M., Nguyen, K., and de Leon Miranda, J. (2020). Measuring social unrest using media reports. IMF Working Paper No. 20/129.
- Barrett, P., Bondar, M., Chen, S., Chivakul, M., and Igan, D. (2021). Social unrest and financial markets. forthcoming IMF Working Paper.
- Bellemare, M. F. (2015). Rising food prices, food price volatility, and social unrest. American Journal of Agricultural Economics, 97(1):1–21.
- Bloom, N. (2014). Fluctuations in Uncertainty. Journal of Economic Perspectives, 28(2):153–176.
- Breiman, L. (2001). Random forests. Machine Learning.
- Breiman, L., Friedman, J., Olshen, R., and Stone, C. (1984). Classification And Regression Trees. CRC Press.
- Cadena, J., Korkmaz, G., Kuhlman, C. J., Marathe, A., Ramakrishnan, N., and Vullikanti, A. (2015). Forecasting social unrest using activity cascades. PLOS ONE, 10(6):1–27.
- Chen, T. and Guestrin, C. (2016). Xgboost. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.
- DeLong, E. R., DeLong, D. M., and Clarke-Pearson, D. L. (1988). Comparing the areas under two or more correlated receiver operating characteristic curves: A nonparametric approach. Biometrics, 44(3):837–845.
- Duval, R. A., Furceri, D., Hu, B., Jalles, J. T., and Nguyen, H. (2018). A Narrative Database of Major Labor and Product Market Reforms in Advanced Economies. IMF Working Papers 2018/019, International Monetary Fund.
- Enikolopov, R., Makarin, A., and Petrova, M. (2020). Social media and protest participation: Evidence from russia. Econometrica, 88(4):1479–1514.
- Freund, Y. and Schapire, R. E. (1997). A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences, 55(1):119 – 139.
- Gentzkow, M., Kelly, B., and Taddy, M. (2019). Text as data. Journal of Economic Literature, 57(3):535–74.
- Goodfellow, I., Bengio, Y., and Courville, A. (2016). Deep Learning. MIT Press.
- Hadzi-Vaskov, M., Pienknaguraand, S., and Ricci, L. A. (2021). The macroeconomic impact of social unrest. IMF Working paper WP/21/135.
- Hastie, T., Tibshirani, R., and Friedman, J. (2001). The Elements of Statistical Learning. Springer Series in Statistics. Springer New York Inc., New York, NY, USA.
- Hellwig, K.-P. (2020). Predicting fiscal crises: A machine learning approach. mimeo.
- Hlatshwayo, S. and Redl, C. (2020). The macroeconomic relevance of social unrest. forthcoming.
- Joseph, A. (2019). Parametric inference with universal function approximators. Bank of England working papers 784, Bank of England.
- Kaminsky, G. L. and Reinhart, C. M. (1999). The twin crises: The causes of banking and balance-of-payments problems. American Economic Review, 89(3):473–500.
- Kingma, D. P. and Ba, J. (2017). Adam: A method for stochastic optimization.
- Lundberg, S. M. and Lee, S.-I. (2017). A unified approach to interpreting model predictions. In Guyon, I., Luxburg, U. V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., and Garnett, R., editors, Advances in Neural Information Processing Systems 30, pages 4765–4774. Curran Associates, Inc.
- Madestam, A., Shoag, D., Veuger, S., and Yanagizawa-Drott, D. (2013). Do Political Protests Matter? Evidence from the Tea Party Movement*. The Quarterly Journal of Economics, 128(4):1633–1685.
- Manacorda, M. and Tesei, A. (2020). Liberation technology: Mobile phones and political mobilization in africa. Econometrica, 88(2):533–567.
- Moseley, M. W. (2015). Contentious engagement understanding protest participation in latin american democracies. Journal of Politics in Latin America, 7(3):3–48.
- Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., and Duchesnay, E. (2011). Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825–2830.
- Racine, J. (2000). Consistent cross-validatory model-selection for dependent data: hv-block cross-validation. Journal of Econometrics, 99(1):39 – 61.
- Redmon, J. and Farhadi, A. (2017). Yolo9000: Better, faster, stronger. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 6517–6525.
- Shao, J. (1993). Linear model selection by cross-validation. Journal of the American Statistical Association, 88(422):486–494.
- Strumbelj, E. and Kononenko, I. (2010). An efficient explanation of individual classifications using game theory. Journal of Machine Learning Research, 11.
- Turchin, P. and Korotayev, A. (2020). The 2010 structural-demographic forecast for the 2010 to 2020 decade: A retrospective assessment. PLoS ONE 15(8).

*Italic: Source: wpiea2021263-print-pdf - References*

### Appendix I: Machine learning models

### Overview
- A range of ML tools were compared for prediction performance using the expanding window approach based on the AUC.
- The Python library of Pedregosa et al. (2011), scikit-learn.org, was used to implement algorithms.
- This section draws heavily on Pedregosa et al. (2011) and Hastie et al. (2001).

### 7.1 Linear models: regularized logistic regression
- Objective solved:
  min_{β,α} { (1−ρ)/2 ||β||^2_2 + ρ||β||_1 + C/n ∑_{i=1}^n [ log( exp( −y_i (X_i^T β + α) ) ) + 1 ] }
- If ρ = 1 then Lasso estimator for logistic regression.
- If ρ = 0 then Ridge logistic regression.
- Assumed C = 1.
- No hyperparameters required tuning given this formulation; expanding window test run for the two extreme values of ρ.

### 7.2 Neural Network
- Loss function minimized:
  L(y, ŷ, W) = −y ln ŷ − (1−y) ln(1−ŷ) + α ||W||^2_2
- α was set to 0.0001.
- Weights solved using the gradient descent algorithm of Kingma and Ba (2017).
- Architecture: Am = 3 layer network with grid search over node counts.
  - layer 1: 5,10,20,30
  - layer 2: 10, 8, 5, 1
  - layer 3: 10, 5, 1
- This leads to 48 models whose forecasts are tested.
- Optimal parameters: 5,10 and 10 for the three respective layers.

### 7.3 Support vector machine
- Objective:
  min_{w,b,ζ} 1/2 w^T w + C/n ∑_{i=1}^n ζ_i
  s.t. y_i ( w^T φ(x) + b ) ≥ 1 − ζ_i, ζ_i ≥ 0, i = 1,...,n
- Kernel: K(x,x') = φ(x)^T φ(x) = exp( −γ ||x − x'||^2 ).
- Predictors mean-variance normalized prior to applying algorithm.
- Grid searched:
  - C = {0.1,1,10,100,1000,10000}
  - γ = {0.001,0.01,0.1,1,10,100}
- Optimal parameters: {C, γ} = {1, 0.001}.
- Interpretation: C controls penalty strength on boundary; γ affects influence of single observation versus cluster.

### 7.4 Tree based models
- Random Forests: grid search over max depth and max number of features.
  - max depth ∈ {1,3,5}
  - max features ∈ {50,100,200}
- Optimal parameters: max tree depth of 3 and max features of 200.

### 7.4.1 AdaBoost
- Algorithm summary:
  1. Initialize weights for observations w_i = 1/N, i = 1,...,N
  2. For m = 1 to M:
     - (a) Fit a decision tree T_m to the training data using weights.
     - (b) Compute err_m = ∑_{i=1}^N w_i I(y_i ≠ T_m(x_i)) / ∑_{i=1}^N w_i
     - (c) Compute α_m = ln((1 − err_m)/err_m)
     - (d) Set w_i = w_i exp(α_m I(y_i ≠ T_m(x_i))) for i = 1,...,N
  3. Update model output: G(x) = sign{ ∑_{m=1}^M α_m T_m(x) }
  4. Update model using the learning rate: T_t(x) = T_{t−1}(x) + ν G(x)
- Considered number of estimators ranging from 50 to 300 in increments of 25.
- Considered learning rate from 0.1 to [end of provided content].

### 0.6 in increments of 0.1. We found the optimal performance when the number of estimators is 50 and learning rate

### Gradient Boosted Trees and Appendix II: Input Data and Aggregation scheme

### Gradient Boosted Trees — method and formulation
- Model formulation as presented:
  - ˆy_i = F_M(x_i) = M ∑ i=1 ν h_m(x_i)
  - Recursive formulation: F_m(x) = F_{m−1}(x) + ν h_m(x)
  - Optimization objective: min_{h_m} ∑_{i=1}^M L(y_i, F_{m−1}(x) + ν h_m(x))
  - Initialization: F_0 is a constant (for square loss, the mean of y)
  - First-order Taylor approximation of the loss:
    - L(y_i, F_{m−1}(x_i) + h_m(x_i)) ≈ L(y_i, F_{m−1}(x_i)) + ν h_m(x_i) [∂L(y_i, F(x_i)) / ∂F(x_i)]_{F=F_{m−1}}
  - Denote g_i for the derivative in square brackets. The optimal h_m will be chosen proportional to −ν g_i.
  - For classification, map F(x) into a probability (for example, with a sigmoid function).
  - The algorithm can be combined with bootstrap sampling of the features at each iteration (sub sampling as in Pedregosa et al. (2011)).

### Hyperparameter grids evaluated
- Number of estimators: {30,50,100,200}
- Learning rate: {0.1,0.2,0.5}
- Subsample: {0.8,1}
- Max depth of trees: {1,3}

### Reported performance and optimal settings (as reported in the source)
- One statement: "We found the optimal performance when the number of estimators is 50 and learning rate 0.1."
- Later reported optimal combination: {30,0.1,0.8,1} (interpreted as number of estimators = 30; learning rate = 0.1; subsample = 0.8; max depth = 1).

### Appendix II — Input data and aggregation scheme (selected points)
- Variables are enumerated with index and name (examples from the list):
  - 0 AE Dummy — Dummy — Hellwig (2020)
  - 48 Oil Price Inflation — Oil Inflation — Hellwig (2020)
  - 51 Inflation — Inflation — Hellwig (2020)
  - 52 Per Capita GDP Growth — GDP Growth — Hellwig (2020)
  - 59 Prim. Bal./GDP — Primary Balance — Hellwig (2020)
  - 118 GDP, USD, log — Size of Economy — Hellwig (2020)
  - 142 Reported Social Unrest Index — Reported Social Unrest Index — Barrett et al (2020)
  - 148 Country Size in Square Kilometers — Size of Economy — Cross-National Time Series (CNTS) Data Archive (2019 Edition)
  - 171 Registered Voters/Population (Scaling: 0.001) — Political — Cross-National Time Series (CNTS) Data Archive (2019 Edition)
  - 198 Prison Population, Total — Prisons — World Prison Brief
  - 201 Current Account Risk (% GDP) — GDP Growth — International Country Risk Guide
  - 240 Prevalence of moderate or severe food insecurity — Poverty — World Bank Development Indicators
  - 273 Drought — Nautral Disasters — EM-DAT
  - 281 World Uncertainty Index — Uncertainty — Ahir et al (2018)
  - 287 Alcoholic Beverages, Tobacco, and Narcotics_MAX — Inflation — IMF CPI Database
  - 300 Domestic finance - Overall — Structural Reforms — Duval et al (2018)
- The variable list continues through indices up to 343 with detailed variable names, transformations, aggregate categories, and sources (examples above reflect the structure: Variable Name, transformation, Aggregate Category, Source).

*Source: wpiea2021263-print-pdf*

---


_Source: https://www.imf.org/-/media/files/publications/wp/2021/english/wpiea2021263-print-pdf.pdf_