## _wp1465 - References ## Source details **Canonical URL:** [_wp1465 - References](https://www.imf.org/-/media/websites/imf/imported-full-text-pdf/external/pubs/ft/wp/2014/_wp1465.pdf) ## Other formats - [Markdown version](/-/media/websites/imf/imported-full-text-pdf/external/pubs/ft/wp/2014/_wp1465.pdf.md) - [Structured JSON version](/-/media/websites/imf/imported-full-text-pdf/external/pubs/ft/wp/2014/_wp1465.pdf.json) --- ### I. Introduction and objective - Goal: compare performance of two parametric limited dependent variable early warning systems (EWS) — fixed effects logit and random effects probit — in predicting in-sample and out-of-sample currency crises in emerging market economies (EMs). - Dataset: monthly data for 29 EMs between January 1995 and December 2012. - Crisis definitions used: exchange rate pressure index (ERPI) exceeding country-specific mean by η standard deviations, with η = 2 and η = 3 compared. - Forecast horizon for crisis incidence: 24 months (forward-looking crisis variable Yit equals 1 if a crisis occurs within next 24 months). - Cut-off selection for estimated crisis probability: optimal cut-off chosen to minimize total misclassification error (TME = Type 1 error + Type 2 error), with Type 1 = missed crises / total crises and Type 2 = false alarms / total non-crisis episodes. ### II. Key methodology - ERPI construction: weighted average of one-month changes in nominal exchange rate and foreign exchange reserves, normalized by country-specific standard deviations. - Two crisis thresholds: - η = 2 → ERPI identifies 191 crisis episodes in the panel (Jan 1995–Dec 2012). - η = 3 → ERPI identifies 77 crisis episodes in the panel (Jan 1995–Dec 2012). - Forward-looking crisis variable Yit: - Yit = 1 if ∃ k ∈ {1,...,24} s.t. CCit+k = 1; otherwise 0. - Econometric models: - Fixed effects logit (conditional logit) and probit specifications estimated with the crisis binary as dependent variable. - Explanatory variables (external vulnerability indicators): FXR/STED, CAB/Y, ΔY, REERM, NFA/Y, PRCR/Y. - Model selection: - For each estimation technique and crisis definition, seven specifications estimated. - Specification with largest ROC statistic (specification (7) in the paper) selected for in-sample and out-of-sample performance comparison. - Performance evaluation: - ROC analysis used to evaluate discrimination ability; optimal probability cut-off chosen to minimize TME. ### III. Main empirical findings (coefficients and significance) - Robust predictors across specifications: - Stronger real GDP growth rates (ΔY) significantly reduce probability of a currency crisis. - Higher net foreign assets (NFA/Y) significantly reduce crisis probability in many specifications. - Higher credit to the private sector (PRCR/Y) is significantly associated with higher crisis incidence. - Reserve-related measure FXR/STED: - Correct sign (negative) but statistically significant mainly when η = 2; loses significance when η = 3. - CAB/Y and REERM: - Correct signs generally (higher CAB/Y → lower crisis probability; REERM overvaluation → higher crisis probability) but not always statistically significant across specifications. - ROC statistics (selected values from tables): - Logit Panel A (η=2): ROC range across columns: 0.624 to 0.691 (specification (7) shows 0.691). - Logit Panel B (η=3): ROC range across columns: 0.596 to 0.753 (specification (7) shows 0.753). - Probit Panel A (η=2): ROC range: 0.6346 to 0.6844 (specification (7) shows 0.684). - Probit Panel B (η=3): ROC range: 0.6936 to 0.7478 (specification (7) shows 0.7478). ### IV. In-sample versus out-of-sample performance (selected metrics and ranges) - General pattern: out-of-sample performance deteriorates relative to in-sample; sensitivity to inclusion of year 2008 in estimation sample. - Logit EWS (selected specification (7)): - When η=2: - Out-of-sample TME scores vary between 68 and 111. - Implied correct classification between 44% and 66% of total out-of-sample observations. - When η=3: - Out-of-sample TME scores vary between 83 and 116. - Implied correct classification between 42% and 58% of total out-of-sample observations. - Probit EWS (selected specification (7)): - When η=2: - Out-of-sample TME scores vary between 72 and 97. - Implied correct classification between 51% and 64% of total out-of-sample observations. - When η=3: - Out-of-sample TME scores vary between 87 and 117. - Implied correct classification between 41% and 53% of total out-of-sample observations. - Sample-window specifics (selected years and metrics): - Logit (η=2) out-of-sample: - 2007: Prob. cut-off value 0.06; Crisis Episodes Correctly Called 44.8%; Non-Crisis Episodes Correctly Called 86.8%; Missed Crisis Episodes 55.2%; False Alarms 13.2%; Crisis Prob. Given Alarm 0.74; Crisis Prob. Given No Alarm 0.35; TME 68. - 2009: Prob. cut-off value 0.07; Crisis Episodes Correctly Called 30.0%; Non-Crisis Episodes Correctly Called 58.9%; Missed Crisis Episodes 70.0%; False Alarms 41.1%; Crisis Prob. Given Alarm 0.04; Crisis Prob. Given No Alarm 0.07; TME 111. - Probit (η=2) out-of-sample: - 2007: Prob. cut-off value 0.51; Crisis Episodes Correctly Called 31.5%; Non-Crisis Episodes Correctly Called 81.8%; Missed Crisis Episodes 68.5%; False Alarms 18.2%; Crisis Prob. Given Alarm 0.83; Crisis Prob. Given No Alarm 0.70; TME 87. - 2010: Prob. cut-off value 0.52; Crisis Episodes Correctly Called 45.8%; Non-Crisis Episodes Correctly Called 56.5%; Missed Crisis Episodes 54.2%; False Alarms 43.5%; Crisis Prob. Given Alarm 0.37; Crisis Prob. Given No Alarm 0.35; TME 98. - Effect of crisis-definition choice: - Both EWS perform better when η=2 (more crisis episodes) than when η=3 (fewer crisis episodes). TME scores systematically lower for η=2. ### V. Policy-relevant conclusions and recommendations - EWS sensitivity to estimation sample: - Out-of-sample performance (TME and the conditional probability of crisis given an alarm) can vary considerably with inclusion of particular years with many outlying observations (notably 2008). - Recommendation: similar early warning exercises should be run at least once every year to incorporate new economic and financial data. - Crisis-definition choice for policy applications: - Defining crises with η=2 (identifying more crisis episodes) reduces TME relative to η=3. - Policy implication: for macroeconomic policy purposes, prefer a crisis definition that identifies more rather than fewer crisis episodes, accepting a higher risk of false alarms to reduce missed crises. - Limitations and suggested extensions: - Current EWS rely mainly on standard macroeconomic and external vulnerability indicators; potential improvement by including indicators of cross-country contagion, spillovers, or cross-border financial linkages. - Suggested extensions: modeling sudden stops in capital inflows and assessing out-of-sample performance; exploration of the optimal value of η that minimizes TME. ### Annex — Country list (29 EMs in panel) - Argentina - Brazil - Bulgaria - Chile - China - Colombia - Croatia - Czech Republic - Egypt - Hungary - India - Indonesia - Kazakhstan - Korea - Malaysia - Mexico - Pakistan - Peru - Philippines - Poland - Romania - Russia - South Africa - Taiwan - Thailand - Turkey - Ukraine - Uruguay - Vietnam ### Annex — Description of the variables - (1) FXR/STED: Ratio between foreign exchange reserves and short term external debt. Quarterly data interpolated to monthly. Source: Joint External Debt Hub (www.jedh.org). - (2) CAB/Y: Current account balance as a percentage of GDP. Annual data interpolated to monthly. Source: World Economic Outlook Database, International Monetary Fund. - (3) ΔY: Real GDP growth, annual percentage change. Annual data interpolated to monthly. Source: World Economic Outlook Database, International Monetary Fund. - (4) REERM: Real effective exchange rate misalignment (deviation from three-year moving average). Monthly data. Source: International Financial Statistics, International Monetary Fund. - (5) NFA/Y: Ratio between the stock of net foreign assets and nominal GDP. Source: International Financial Statistics, International Monetary Fund. - (6) PRCR/Y: Ratio between private credit and nominal GDP. Available for most emerging economies only from January 2001 onwards. Source: International Financial Statistics, International Monetary Fund. ### Figures — Performance metrics (as labeled in source) - Figure 2. Logit In-sample and Out-of-sample Performances - Panels for η=2 and η=3 - Out-of-sample years: 2007, 2008, 2009, 2010, 2011 - Performance metrics labeled: % cr. corr called; % non-cr. corr called; % missed crises; % false alarms; TME - Figure 3. Probit In-sample and Out-of-sample Performances - Panels for η=2 and η=3 - Out-of-sample years: 2007, 2008, 2009, 2010, 2011 - Performance metrics labeled: % cr. corr called; % non-cr. corr called; % missed crises; % false alarms; TME *Source: IMF staff analysis contained in the provided PDF content (January 1995–December 2012 dataset and tables/figures as presented).* ### References ............................................................................................................. ### _wp1465 - References ### I. Introduction and objective - Goal: compare performance of two parametric limited dependent variable early warning systems (EWS) — fixed effects logit and random effects probit — in predicting in-sample and out-of-sample currency crises in emerging market economies (EMs). - Dataset: monthly data for 29 EMs between January 1995 and December 2012. - Crisis definitions used: exchange rate pressure index (ERPI) exceeding country-specific mean by η standard deviations, with η = 2 and η = 3 compared. - Forecast horizon for crisis incidence: 24 months (forward-looking crisis variable Yit equals 1 if a crisis occurs within next 24 months). - Cut-off selection for estimated crisis probability: optimal cut-off chosen to minimize total misclassification error (TME = Type 1 error + Type 2 error), with Type 1 = missed crises / total crises and Type 2 = false alarms / total non-crisis episodes. ### II. Key methodology - ERPI construction: weighted average of one-month changes in nominal exchange rate and foreign exchange reserves, normalized by country-specific standard deviations. - Two crisis thresholds: - η = 2 → ERPI identifies 191 crisis episodes in the panel (Jan 1995–Dec 2012). - η = 3 → ERPI identifies 77 crisis episodes in the panel (Jan 1995–Dec 2012). - Forward-looking crisis variable Yit: - Yit = 1 if ∃ k ∈ {1,...,24} s.t. CCit+k = 1; otherwise 0. - Econometric models: - Fixed effects logit (conditional logit) and probit specifications estimated with the crisis binary as dependent variable. - Explanatory variables (external vulnerability indicators): FXR/STED (foreign exchange reserves / short-term external debt), CAB/Y (current account balance as % of nominal GDP), ΔY (real GDP growth rate), REERM (real effective exchange rate misalignment), NFA/Y (net foreign assets as % of nominal GDP), PRCR/Y (credit to the private sector as % of nominal GDP). - Model selection: for each estimation technique and crisis definition, seven specifications estimated; the specification with largest ROC statistic (specification (7) in the paper) is selected for in-sample and out-of-sample performance comparison. - ROC analysis used to evaluate discrimination ability; optimal probability cut-off chosen to minimize TME. ### III. Main empirical findings (coefficients and significance) - Robust predictors across specifications: - Stronger real GDP growth rates (ΔY) significantly reduce probability of a currency crisis (logit and probit; significance levels reported across specifications). - Higher net foreign assets (NFA/Y) significantly reduce crisis probability in many specifications. - Higher credit to the private sector (PRCR/Y) is significantly associated with higher crisis incidence. - Reserve-related measure FXR/STED: - Correct sign (negative) but statistically significant mainly when η = 2 (191 crises); loses significance when η = 3 (77 crises). - CAB/Y and REERM: - Correct signs generally (higher CAB/Y → lower crisis probability; REERM overvaluation → higher crisis probability) but not always statistically significant across specifications. - ROC statistics (selected examples from tables): - Logit Panel A (η=2), ROC range across columns: 0.624 to 0.691 (selected specification (7) shows 0.691). - Logit Panel B (η=3), ROC range across columns: 0.596 to 0.753 (specification (7) shows 0.753). - Probit Panel A (η=2), ROC range: 0.6346 to 0.6844 (specification (7) shows 0.684). - Probit Panel B (η=3), ROC range: 0.6936 to 0.7478 (specification (7) shows 0.7478). ### IV. In-sample versus out-of-sample performance (selected metrics and ranges) - General pattern: out-of-sample performance deteriorates relative to in-sample; sensitivity to inclusion of year 2008 in estimation sample. - Logit EWS (selected specification (7)): - When η=2: - Out-of-sample TME scores vary between 68 and 111. - Implied correct classification between 44% and 66% of total out-of-sample observations. - When η=3: - Out-of-sample TME scores vary between 83 and 116. - Implied correct classification between 42% and 58% of total out-of-sample observations. - Probit EWS (selected specification (7)): - When η=2: - Out-of-sample TME scores vary between 72 and 97. - Implied correct classification between 51% and 64% of total out-of-sample observations. - When η=3: - Out-of-sample TME scores vary between 87 and 117. - Implied correct classification between 41% and 53% of total out-of-sample observations. - Sample-window specifics (examples from Tables 3 and 4 for selected years): - Logit (η=2) out-of-sample: - 2007: Prob. cut-off value 0.06; Crisis Episodes Correctly Called 44.8%; Non-Crisis Episodes Correctly Called 86.8%; Missed Crisis Episodes 55.2%; False Alarms 13.2%; Crisis Prob. Given Alarm 0.74; Crisis Prob. Given No Alarm 0.35; TME 68. - 2009: Prob. cut-off value 0.07; Crisis Episodes Correctly Called 30.0%; Non-Crisis Episodes Correctly Called 58.9%; Missed Crisis Episodes 70.0%; False Alarms 41.1%; Crisis Prob. Given Alarm 0.04; Crisis Prob. Given No Alarm 0.07; TME 111. - Probit (η=2) out-of-sample: - 2007: Prob. cut-off value 0.51; Crisis Episodes Correctly Called 31.5%; Non-Crisis Episodes Correctly Called 81.8%; Missed Crisis Episodes 68.5%; False Alarms 18.2%; Crisis Prob. Given Alarm 0.83; Crisis Prob. Given No Alarm 0.70; TME 87. - 2010: Prob. cut-off value 0.52; Crisis Episodes Correctly Called 45.8%; Non-Crisis Episodes Correctly Called 56.5%; Missed Crisis Episodes 54.2%; False Alarms 43.5%; Crisis Prob. Given Alarm 0.37; Crisis Prob. Given No Alarm 0.35; TME 98. - Effect of crisis-definition choice: - Both EWS perform better when η=2 (more crisis episodes) than when η=3 (fewer crisis episodes). TME scores systematically lower for η=2. ### V. Policy-relevant conclusions and recommendations - EWS sensitivity to estimation sample: - EWS out-of-sample performance (TME and the conditional probability of crisis given an alarm) can vary considerably with the inclusion of particular years with many outlying observations (notably 2008). Implication: similar early warning exercises should be run at least once every year to incorporate new economic and financial data. - Crisis-definition choice for policy applications: - Defining crises with η=2 (identifying more crisis episodes) reduces TME relative to η=3. Policy implication: for macroeconomic policy purposes, prefer a crisis definition that identifies more rather than fewer crisis episodes, accepting a higher risk of false alarms to reduce missed crises. - Limitations and suggested extensions: - Current EWS rely mainly on standard macroeconomic and external vulnerability indicators; potential improvement by including indicators of cross-country contagion, spillovers, or cross-border financial linkages. - Extension to modeling sudden stops in capital inflows and assessing out-of-sample performance suggested. - Exploration of the optimal value of η (the multiplier on ERPI standard deviation) that minimizes TME is an open question. *Source: IMF staff analysis contained in the provided PDF content (January 1995–December 2012 dataset and tables/figures as presented).* ### REFERENCES ### _wp1465 - REFERENCES ### References - Abiad, Abdul, 2003, ”Early Warning Systems: A Survey and a Regime-Switching Approach”, IMF Working Paper, No. WP/03/32, (Washington: International Monetary Fund). - Beckmann, Daniela, Lukas Menkhoff, and Katja Sawischlewski, 2006, ”Robust Lessons About Practical Early Warning Systems”, Journal of Policy Modeling, Vol. 28, pp. 163–193. - Berg, Andrew, and Catherine Pattillo, 1999, ”Predicting Currency Crises: The Indicators Approach and an Alternative”, Journal of International Money and Finance, Vol. 18, pp. 561–586. - Berkmen, S. Pelin, Gaston Gelos, Robert Rennhack, and James Walsh, 2012, ”The global financial crisis: Explaining cross-country differences in the output impact”, Journal of International Money and Finance, Vol. 31, pp. 42–59. - Blanchard, Olivier, Mitali Das, and Hamid Faruqee, 2010, ”The Initial Impact of the Crisis on Emerging Market Countries”, Brooking Papers on Economic Activity, Spring 2010, pp. 263–323. - Borio, Claudio, and Mathias Drehmann, 2008, ”Assessing the Risk of Banking Crises – Revised”, BIS Quarterly Review, March 2009, pp. 29–46, (Basel: Bank for International Settlements). - Bussiere, Matthieu, and Marcel Fratszcher, 2006, ”Towards a New Early Warning System of Financial Crises”, Journal of International Money and Finance, Vol. 25, pp. 953–973. - Candelon, Bertrand, Elena-Ivona Dumitrescu, and Christopher Hurlin, 2012, ”How to Evaluate an Early-Warning System: Toward a Unified Statistical Framework for Assessing Financial Crises Forecasting Methods” IMF Economic Review, Vol. 60, No.1, (Washington: International Monetary Fund). - Cerra, Valerie, and Sweta C. Saxena, 2008, ”Growth Dynamics: The Myth of Economic Recovery” American Economic Review, Vol. 98, No.1 (March 2008), pp. 439–457. - Comelli, Fabio, 2013, ”Comparing Parametric and Non-parametric Early Warning Systems For Currency Crises in Emerging Market Economies”, IMF Working Paper, No. WP/13/134, (Washington: International Monetary Fund). - Eichengreen, Barry, Andrew K. Rose, and Charles Wyplosz, 1995, ”Exchange Market Mayhem”, Economic Policy, Vol. 10, No. 21, pp. 249–312. - Frank, Nathaniel, and Heiko Hesse, 2009, ”Financial Spillovers to Emerging Markets during the Global Financial Crisis”, IMF Working Paper, No. WP/09/104 (Washington: International Monetary Fund). - Frankel, Jeffrey, and George Saravelos, 2012, ”Can Leading Indicators Assess Country Vulnerability? Evidence From the 2008-09 Global Financial Crisis”, Journal of International Economics, Vol. 87, pp. 216–231. - Goldman Sachs, 2013, ”’Sudden–stops’ in Capital Inflows and The Role of FX Reserves”, CEEMEA Economics Analyst, Issue No: 13/25. - Gourinchas, Pierre-Olivier, and Maurice Obstfeld, 2011, ”Stories of The Twentieth Century For The Twenty-first”, NBER Working Paper No. 17252, (Cambridge, Massachusetts: National Bureau of Economic Research). - International Monetary Fund, 2002, ”Global Financial Stability Report”, March 2002, Chapter 4, (Washington: International Monetary Fund). - Kaminsky, Graciela L., 1998, ”Currency and Banking Crises: The Early Warning of Distress”, International Finance Discussion Paper No. 629, (Washington: Board of Governors of the Federal Reserve System). - Kaminsky, Graciela L., Saul Lizondo and Carmen M. Reinhart, 1998, ”Leading Indicators for Currency Crisis”, IMF Staff Papers, Palgrave Macmillan Journals, 45(1). - Kaminsky, Graciela L., and Carmen M. Reinhart, 1999, ”The Twin Crises: The Causes of Banking and Balance-of-Payments Problems”, American Economic Review, Vol. 89, No. 3, pp. 473–500. - Llaudes, Ricardo, Ferhan Salman and Mali Chivakul, ”The Impact of the Great Recession on Emerging Markets”, International Monetary Fund Working Paper No. WP/10/237, (Washington: International Monetary Fund). - Milesi-Ferretti, Gian Maria, and Cédric Tille, 2011, ”The great retrenchment: international capital flows during the global financial crisis”, Economic Policy, Vol. 26, No. 66, pp. 289–346. - Minoiu, Camelia, Chanhyun Kang, V.S. Subrahmanian, Anamaria Berea, 2013, ” Does Financial Connectedness Predict Crises?”, IMF Working Paper, No. WP/13/267, (Washington: International Monetary Fund). - Rose, Andrew, and Mark M. Spiegel, 2012, ”Cross-country Causes and Consequences of the 2008 Crisis: Early Warning”, Japan and the World Economy, Vol. 24, pp. 1–16. - Stata Press, 2013, Stata Base Reference Manual, (College Station, TX: StataCorp LP). - Wooldridge, Jeffrey M., 2002, Econometric Analysis of Cross Section and Panel Data, MIT Press, (Cambridge: Massachusetts Institute of Technology Press). ### Annex — Country List - Argentina - Brazil - Bulgaria - Chile - China - Colombia - Croatia - Czech Republic - Egypt - Hungary - India - Indonesia - Kazakhstan - Korea - Malaysia - Mexico - Pakistan - Peru - Philippines - Poland - Romania - Russia - South Africa - Taiwan - Thailand - Turkey - Ukraine - Uruguay - Vietnam ### Annex — Description of the Variables - (1) Ratio between foreign exchange reserves and short term external debt, FXR/STED: This is calculated as the ratio between the stocks of foreign exchange reserves and short-term external debt (i.e. maturing within one year). Both numerator and denominator are expressed in U.S. dollars. It is a reserve adequacy ratio which is often used in early warning exercises. Quarterly data have been interpolated in order to have monthly time series. Source: Joint External Debt Hub (www.jedh.org). - (2) Current account balance as a percentage of GDP, CAB/Y. Ratio between the current account balance and nominal GDP. Both numerator and denominator are expressed in U.S. dollars. Annual data have been interpolated in order to have monthly time series. Source: World Economic Outlook Database, International Monetary Fund. - (3) Real GDP growth, ΔY: Annual percentage change in real GDP. Annual data have been interpolated in order to have monthly time series for real GDP growth. Source: World Economic Outlook Database, International Monetary Fund. - (4) Real effective exchange rate misalignment, REERM. The series has been obtained by taking the real effective exchange rate (REER) deviation from the three-year moving average. Monthly data. Source: International Financial Statistics, International Monetary Fund. - (5) Ratio between the stock of net foreign assets and nominal GDP, NFA/Y. Both numerator and denominator are expressed in U.S. dollars. Source: International Financial Statistics, International Monetary Fund. - (6) Ratio between private credit and nominal GDP, PRCR/Y. Available for most emerging economies only from January 2001 onwards. Source: International Financial Statistics, International Monetary Fund. ### Figures — Performance Metrics (as labeled in source) - Figure 2. Logit In-sample and Out-of-sample Performances - Panels for η=2 and η=3 - Out-of-sample years: 2007, 2008, 2009, 2010, 2011 - Performance metrics labeled: % cr. corr called; % non-cr. corr called; % missed crises; % false alarms; TME - In-sample and Out-of-sample comparisons shown - Figure 3. Probit In-sample and Out-of-sample Performances - Panels for η=2 and η=3 - Out-of-sample years: 2007, 2008, 2009, 2010, 2011 - Performance metrics labeled: % cr. corr called; % non-cr. corr called; % missed crises; % false alarms; TME - In-sample and Out-of-sample comparisons shown *Content derived from _wp1465 - REFERENCES (IMF).* --- _Source: https://www.imf.org/-/media/websites/imf/imported-full-text-pdf/external/pubs/ft/wp/2014/_wp1465.pdf_