## 1. MSE Ratios from Dollar-Based Exchange Rates ## Source details **Canonical URL:** [1. MSE Ratios from Dollar-Based Exchange Rates](https://www.imf.org/-/media/websites/imf/imported-full-text-pdf/external/pubs/ft/wp/2004/_wp0473.pdf) ## Other formats - [Markdown version](/-/media/websites/imf/imported-full-text-pdf/external/pubs/ft/wp/2004/_wp0473.pdf.md) - [Structured JSON version](/-/media/websites/imf/imported-full-text-pdf/external/pubs/ft/wp/2004/_wp0473.pdf.json) --- ### Introduction and motivation - The dollar and the euro have exhibited movements puzzling relative to standard models; the dollar characterized as overly “darling” and the euro repeatedly confounding predictions. - Recent models (net foreign asset–real exchange rate correlations, productivity differences) have not been subjected to Meese and Rogoff–style out-of-sample testing. - Rationale for systematic out-of-sample testing: - Models have risen in policy and financial prominence without systematic out-of-sample testing. - Recent treatments rely on limited sets of older models (monetary model, PPP, real interest differential). - Common evaluation metrics focus on first- and second-moment metrics while neglecting direction of change and other forecast attributes. ### Models, currencies, and scope of comparison - Five models compared to the driftless random walk benchmark: - Purchasing power parity (PPP). - Sticky-price monetary model (Dornbusch and Frankel; no coefficient restrictions). - Productivity-differentials model. - Interest rate parity specification (arbitrage relation, not estimated). - Composite specification (BEER/NATREX-style) incorporating multiple channels. - Currencies examined (U.S. dollar-based exchange rates): - Canadian dollar, British pound, Deutsche mark, Japanese yen. - Additional robustness check: corresponding yen-based rates examined but not reported. ### Estimation and forecasting design - Two econometric specifications: - First-difference specification. - Error-correction specification (ECM) with cointegrating vector estimated via Johansen within rolling windows. - Forecasting horizons evaluated: - 1-quarter, 4-quarter, 20-quarter. - Out-of-sample forecasting periods: - 1987q2 to 2000q4 (post-Louvre Accord). - 1983q1 to 2000q4 (post-end of U.S. monetary targeting). - Implementation notes: - Rolling regressions (Meese and Rogoff convention) to alleviate parameter instability. - OLS used throughout; instrumental variables not used. - Error-correction forecasts constructed to be true ex ante (cointegrating vector re-estimated as window rolls). - First-difference forecasts use ex post RHS values (informational advantage). ### Data - Frequency and sample: seasonally-adjusted quarterly data for United States, Canada, United Kingdom, Japan, Germany, Switzerland from 1973q2 to 2000q4. - Data sources and constructions: - Exchange rate, money, price, income: IMF International Financial Statistics. - Productivity: Bank for International Settlements (labor productivity indices, 1995=100). - Interest rates for IRP forecasts: essentially the same as Meredith and Chinn (1998). - NFA base stock year 1995 from Lane and Milesi-Ferretti (2001); quarterly stocks from IFS current account (Japan excluded). - Quarterly government debt constructed from annual IFS debt and quarterly deficit data; Canada and U.K. sources noted; Japan and Switzerland incomplete (no composite models estimated for them). - Five-year interest rates for Japan and Switzerland from Global Financial Data due to IMF gaps. - Exchange rate data: end-of-period; output in constant 1990 prices; CPI and PPI base 1990; inflation = 4-quarter log differences of CPI; real interest = 3-month nominal − lagged inflation. ### Forecast evaluation metrics and tests - Primary metric: - Mean squared error (MSE) ratio = MSE(structural model) / MSE(driftless random walk). - Interpretation: ratio < 1 implies structural model outperforms random walk; ratio > 1 implies random walk outperforms. - Statistical inference: Diebold-Mariano statistic (asymptotically standard normal); standard error of loss differential from weighted sum of sample autocovariances using quadratic spectral kernel and data-dependent bandwidth (Andrews, 1991). - Complementary metrics: - Direction of change statistic: proportion of correct sign predictions; tested against ½ (studentized version asymptotically standard normal). - Consistency criterion (Cheung and Chinn, 1998): forecast consistent if (1) forecast and actual have same order of integration; (2) they are cointegrated; (3) cointegrating vector satisfies unitary elasticity (1, −1). - Implementation nuances: - Quadratic spectral kernel used; Bartlett kernel and deterministic bandwidth yield qualitatively similar results. - Diebold-Mariano relied upon asymptotic results; finite-sample critical values not computed. ### Forecast performance — MSE results (exact aggregate counts and notable numeric cases) - Total MSE ratios computed: 216 (138 from ECM; 78 from first-difference). - Of 216 MSE ratios: - 151 are not significant at the 10 percent level. - 65 are significant. - Of the 65 significant cases, 63 show the random walk significantly better; 2 show structural model significantly better. - Patterns and notable numeric entries preserved from source: - Productivity model performs relatively better for the dollar–mark rate at 1- and 4-quarter horizons (not uniform). - PPP and interest rate parity often have MSE ratios < 1 only at the 20-quarter horizon. - Best reported MSE-case in text: interest rate parity at 20-quarter horizon for dollar–yen (post-1982) with MSE ratio = 0.57 (p-value = 0.17). - Related reported case (Cheung, Chinn, and Garcia Pascual, 2003): Canadian dollar–yen ratio = 0.48 (p-value = 0.04). ### Forecast performance — Direction of change results (exact counts and examples) - Total direction-of-change statistics: 216. - Significant cases at 10 percent level: - 50 cases significantly larger than ½ (structural model predicts direction correctly). - 23 cases significantly less than ½ (structural model predicts direction incorrectly). - Distributional details: - Approximately 58 percent of the 50 successful cases associated with ECM; remainder from first-difference. - Successes split evenly across the two forecasting periods. - Successful counts by model: PPP = 18; sticky-price = 10; composite = 9; productivity = 8; interest rate parity = 5. - Currency specificity: dollar–yen yields 13 of the 50 correct-direction forecasts; dollar–pound yields 4 of 50. - Horizon concentration: 20-quarter = 22 of 50; 4-quarter = 18; 1-quarter = 10. - Selected exact numeric examples preserved from tables: - Panel A: BP/$, ECM, Sample 1, Horizon 1: 0.527 (p-value 0.686). - Panel B: CAN$/$, ECM, Sample 1, Horizon 20: 0.944 (p-value 0.000). - Panel C: DM/$, FD, Sample 1, Horizon 1, IRP: 0.800 (p-value 0.000). - Panel E: Yen/$, ECM, Sample 1, Horizon 4 PPP: 0.673 (p-value 0.013). ### Cointegration and (1, −1) restriction results (exact counts and examples) - Cointegration (Johansen tests) between forecasts and actual spot rates: - 67 of 216 cases reject null of no cointegration at 10 percent (i.e., 31 percent cointegrated). - ECM accounts for 39 of 67 cointegrated cases; first-difference for 28. - By model (cointegration rates reported): - Sticky-price: 26 cointegrated series out of 60 (43 percent). - Composite: 39 percent of 36 series. - Productivity: 37 percent. - PPP: 33 percent. - Interest rate parity: 0 percent cointegrated. - By currency: dollar–pound and dollar–Canadian dollar have 19 and 17 cointegrated forecast series respectively; dollar–mark has 12. - By horizon: cointegration frequency declines with horizon (1-quarter: 35 of 67; 4-quarter: 20; 20-quarter: 12). - (1, −1) unitary elasticity restriction on cointegrating vector: - Of 67 cointegrated cases, 48 reject the (1, −1) restriction at 10 percent. - Only 19 of 67 satisfy (1, −1) at 10 percent (28 percent of cointegrated cases). - Combining cointegration and restriction: 9 percent of 216 dollar-based forecast series meet full consistency criterion. - Yen-based results (not shown) yield 12 percent meeting consistency criterion. - Selected exact numeric examples from tables: - Panel A: BP/$, ECM, Sample 2, Horizon 1 Johansen statistic: 32.83* (other entries include 5.25, 7.26, 0.77, 6.95, 12.64*). - Panel B: CAN$/US$, ECM, Sample 2, Horizon 1 Johansen statistic: 19.66*. - Panel C: DM/$, FD, Sample 1, Horizon 1 PPP Johansen statistic: 20.82*. - (1, −1) restriction examples: - Panel A: BP/$, ECM, Sample 1, Horizon 4 LR = 10.29 (p-value 0.00); Horizon 20 LR = 36.66 (p-value 0.00). - Panel B: CAN$/$, ECM, Sample 1, Horizon 20 LR = 76.59 (p-value 0.00); Sample 2 Horizon 20 LR = 82.26 (p-value 0.00). ### Discussion, interpretation, and limitations - Main synthesis: - Majority of MSE comparisons do not reject equality of forecasting performance; structural models rarely and inconsistently outperform the random walk under MSE. - Direction-of-change and consistency criteria identify more instances where structural models beat naive forecasts than MSE alone. - No single model/specification/currency/horizon combination is a consistent winner across samples. - Mechanisms and patterns: - Engel and West (2003) mechanism consistent with short-horizon unpredictability: near-unity discount factor and near unit-root in driving variable yield apparent random-walk behavior at short horizons but potential predictability at long horizons. - PPP and interest parity tend to do better at the 20-quarter horizon. - First-difference specifications often show the worst MSE performance for short-run dynamics estimation. - Stability and sensitivity: - Estimates vary widely over time; in-sample fit not necessarily predictive of out-of-sample performance. - Time-variation in error-correction responsiveness and cointegrating coefficients suggests need for research on parameter instability and restrictions. - Forecast performance sensitive to choice of out-of-sample period (shorter periods of dollar stability can improve some MSE outcomes but may harm direction-of-change performance). - Scope limitations noted in source: - Only linear models evaluated; nonlinearity and regime-switching excluded. - No panel regressions, systems estimation, or certain other methods implemented due to recursive cross-currency framework complexity. - Forecasting performance does not equal explanatory power; a misspecified model can outperform a correctly specified one in forecasting. ### Tables and appendices — structure and data notes (key elements) - Table 1 (MSE Ratios): Panels A–E for BP/$, CAN$/$, DM/$, SF/$, Yen/$; entries report MSE ratios and p-values; ECM and FD columns; horizons 1, 4, 20; samples 1987q2–2000q4 and 1983q1–2000q4; “.” indicates statistics not generated. - Table 2 (Direction of Change): proportion correct with p-values testing difference from ½; same panel/horizon/sample structure as Table 1; selected numeric examples preserved. - Table 3 (Cointegration Johansen stats): tests for null of no cointegration; “*” indicates 10 percent significance; tests for null of one cointegrating vector not rejected in all cases. - Table 4 ((1, −1) restriction): likelihood ratio statistics and p-values applied only to cointegrated cases in Table 3; many exact LR statistics and p-values reported; “.” indicates statistics not generated. - Appendix I data construction specifics and exceptions: - Five-year interest rate exceptions for Japan and Switzerland obtained from Global Financial Data. - NFA quarterly stocks constructed except for Japan. - Composite models not estimated for Japan and Switzerland due to incomplete government debt data. - Appendix II evaluation formulas: Diebold-Mariano loss functions and loss-differential expressions preserved as in source; direction-of-change test statistic formula preserved as given. ### Overall conclusion (source synthesis) - The paper’s summarized verdict: “perhaps.” - No model/specification combination consistently successful across the full set of evaluations. - Some models do well at certain horizons and by certain criteria for particular exchange rates. - Model performance is context-dependent (horizon, currency, metric, out-of-sample period). *Source: _wp0473 - MSE Ratios from Dollar-Based Exchange Rates (IMF working paper excerpt).* ### 1. MSE Ratios from Dollar-Based Exchange Rates ................................................................20 ### 1. MSE Ratios from Dollar-Based Exchange Rates ### Introduction - The dollar and the euro have exhibited movements that are puzzling relative to standard models; the dollar has been characterized as overly “darling” and the euro has repeatedly confounded predictions. - Several new models developed in the past decade (including those motivated by correlations between net foreign asset positions and real exchange rates and by productivity differences) have not been subjected to the rigorous out-of-sample testing exemplified by Meese and Rogoff. - A systematic examination of these newer empirical models is warranted because: - These models have risen in policy and financial prominence without systematic out-of-sample testing. - Recent academic treatments often rely on single or limited sets of older models (for example, monetary model, purchasing power parity, or real interest differential models). - Common evaluation metrics focus on first- and second-moment metrics (mean error, mean squared error) while neglecting other important aspects such as direction of change and other forecast attributes. ### Models and scope of comparison - Five models are compared against the random walk: - Purchasing power parity. - Sticky-price monetary model of Dornbusch and Frankel (the only structural model previously subjected to systematic analyses). - Productivity-differentials model. - Interest rate parity specification. - A composite specification incorporating a number of channels identified in differing theoretical models. - Currencies examined (U.S. dollar-based exchange rates): - Canadian dollar - British pound - Deutsche mark - Japanese yen - Additional robustness check: - Corresponding yen-based rates are examined but not reported to ensure conclusions are not dollar-specific. ### Estimation and forecasting design - Models are estimated in two ways: - First-difference specifications. - Error-correction specifications. - Forecasting horizons evaluated: - 1-quarter horizon ("1-quarter") - 4-quarter horizon ("4-quarter") - 20-quarter horizon ("20-quarter") - Sample periods evaluated: - Post-Louvre accord (Feb. 1987) - Post-1982 ### Forecast evaluation metrics and additional criteria - Conventional metrics: - First- and second-moment metrics such as mean error and mean squared error are considered. - Augmented metrics: - Direction of change statistic (emphasized as potentially more important from a market-timing perspective). - “Consistency” criterion of Cheung and Chinn (1998). - Methodological emphasis: - Focus on out-of-sample prediction to guard against data mining inherent in relying solely on in-sample inference. - Forecasting exercises are ex post simulations; in many instances contemporaneous values of right-hand-side variables are used to predict future exchange rates. ### Study extensions and structure - The study extends forecast comparison of exchange rate models along multiple dimensions described above. - The exchange rate models considered in the exercise are summarized in Section II of the source document. *Source: _wp0473 - 1. MSE Ratios from Dollar-Based Exchange Rates (IMF PDF).* ### Section III discusses the data, the estimation methods, and the criteria used to compare ### _wp0473 - Section III discusses the data, the estimation methods, and the criteria used to compare ### Theoretical models (overview retained from source) - Models evaluated meet criteria: (1) prominent in the economic and policy literature, (2) readily implementable and replicable, and (3) not previously evaluated in a systematic fashion. - Benchmark: random walk (driftless). - Models assessed (as specified in the source): - Relative purchasing power parity (PPP): p̂0t + = ŝt β (relative PPP using log CPI differences). - Sticky-price monetary model (no coefficient restrictions): - ŝt = β10 + β4 m̂t + β3 ŷt + β2 ît + β1 π̂t + u_t (equation (2) in source notation). - Variables: m = log money, y = log real GDP, i = interest rate, π = inflation rate, u_t = error term. - Productivity-differential model (Balassa-Samuelson vein): - ŝt = β10 + β5 z_t + β3 ît + β2 ŷt + β1 m̂t + u_t (equation (3) in source notation), where z = productivity differentials and ω = relative price of nontradables features in related specifications. - Composite model (BEER/NATREX-style; unitary coefficient on inter-country log price level p̂): - ŝt = β10 + β5 ω̂t + β6 r_t + β7 tot_t + β8 gdebt_t + β9 nfa_t + u_t (equation (4) in source notation), where ω = relative price of nontradables, r = real interest rate, gdebt = government debt to GDP ratio, tot = log terms of trade, nfa = net foreign asset. - Interest rate parity (arbitrage relation, not estimated): s_{t+k} = s_t + i_{t,k} − i_{t,k}^$ (equation (5) in source notation). - Conceptual distinctions noted: - Equation (2) (sticky-price monetary) assumes PPP in the long run. - Productivity-based model does not impose PPP; nominal exchange rate can drift from PPP determined by productivity differentials. - Composite model incorporates Balassa-Samuelson, overshooting (r), and portfolio balance effects (gdebt, nfa). ### Data (Section III.A) - Sample frequency and countries: - Quarterly data for the United States, Canada, United Kingdom, Japan, Germany, and Switzerland. - Sample period: 1973q2 to 2000q4. - Data sources: - Exchange rate, money, price, and income variables: IMF’s International Financial Statistics. - Productivity data: Bank for International Settlements. - Interest rates for interest rate parity forecasts: essentially the same as Meredith and Chinn (1998). - See the Data Appendix in source for more detail (not reproduced here). - Out-of-sample forecasting periods used: - 1987q2 to 2000q4 (post-Louvre Accord period). - 1983q1 to 2000q4 (post-end of U.S. monetary targeting). - Rationale: - Shorter out-of-sample (1987–2000) spans relative dollar stability (and mark appreciation). - Longer out-of-sample covers more than one “dollar cycle” (large dollar appreciation and subsequent depreciation against the mark; large dollar depreciation from 250 to 150 yen per dollar) and provides many forecast observations for inference. ### Estimation and forecasting methods (Section III.B) - Rolling regressions (Meese and Rogoff convention): - Estimate on a data window, generate out-of-sample forecast, roll the sample forward one observation, repeat until out-of-sample exhausted. - Benefit: potential alleviation of parameter instability effects. - Two specifications estimated for theoretical models: 1. Error-correction specification: - Two-step procedure: identify cointegrating relation using Johansen procedure to obtain estimated cointegrating vector (Γ̃); incorporate into error-correction term; estimate an error-correction model stripped of short-run dynamics via OLS (equation (8) in source). - Cointegrating vector allowed to vary as the data window moves (contrast with studies that estimate it over entire sample). - Produces true ex ante forecasts (contemporaneous RHS values not necessary). 2. First-differences specification: - Regression form: Δs_t = α + Γ ΔX_t + u_t (equation (7) in source notation). - Emphasizes effects of changes in macro variables on exchange rates. - Uses ex post values of RHS variables to generate predicted exchange rate change (gives informational advantage over error-correction specification). - Estimation method: - Ordinary least squares (OLS) used throughout. - Instrumental variables not used because prior experience shows consistency gains are outweighed by loss in prediction efficiency (Chinn and Meese, 1995). - Cointegration testing: - Johansen procedure applied within rolling estimation windows. - Error-correction forecasts and first-difference forecasts contrasted in informational requirements and forecasting character. - Implementation choices justified: - Short-run dynamics excluded from the error-correction model to produce true ex ante forecasts and to avoid cumbersome generation of RHS variables. - No coefficient restrictions imposed on many models because theory gives little guidance on exact parameter values. ### Forecast comparison criteria (Section III.C) - Primary metric: mean squared error (MSE) ratio - Ratio = MSE(structural model) / MSE(driftless random walk). - Ratio < 1 indicates structural model outperforms random walk; > 1 indicates random walk outperforms. - Statistical inference: Diebold-Mariano statistic (Diebold and Mariano, 1995) used to test null of no difference in accuracy (ratio = 1). The statistic is asymptotically standard normal. - Standard error of loss differential constructed from weighted sum of sample autocovariances using quadratic spectral kernel and data-dependent bandwidth (Andrews, 1991). - Alternative metrics (complementary): 1. Direction of change statistic: - Computed as number of correct sign predictions divided by total number of predictions. - Value above (below) 50 percent indicates better (worse) performance than naive equal-up-or-down model. - Test statistic per Diebold and Mariano (1995); studentized version asymptotically standard normal (equivalently, binomial test). - Rationale: relevant for profitability and market timing (Leitch and Tanner, 1991; Cumby and Modest, 1987). 2. Consistency criterion (Cheung and Chinn, 1998): - Forecast labeled consistent if: (1) forecast and actual have same order of integration; (2) they are cointegrated; and (3) cointegrating vector satisfies unitary elasticity of expectations (i.e., (1, −1) restriction). - Focuses on long-run comovement rather than pointwise precision. - Notes on inference: - Diebold-Mariano relies on asymptotic results; finite-sample critical values not computed due to infeasibility given the large number of cases. - Quadratic spectral kernel used for consistent variance estimation; Bartlett kernel and deterministic bandwidth experimented with and produced qualitatively similar results. ### Forecast performance — key findings (Sections IV.A–IV.D) - Scope of forecast comparisons: - 5 dollar-based currency pairs, 5 model specifications, 2 econometric specifications (error-correction and first-difference), 3 forecasting horizons (1-quarter, 4-quarter, 20-quarter), and 2 forecasting samples. - Total MSE ratios computed: 216 (138 from error-correction specification and 78 from first-difference specification). - For PPP and interest rate parity, only “error-correction specification” entries are reported because these relations are not estimated but used as parity predictions (deviation treated like error-correction term). - MSE (mean squared error) results (Section IV.A): - Of 216 MSE ratios: - 151 are not significant at the 10 percent level. - 65 are significant. - Of the 65 significant cases, 63 show the random walk significantly better than structural models; 2 cases show structural model significantly better. - Conclusion: majority of cases do not reject equality of forecasting performance; structural models rarely and inconsistently outperform random walk under MSE. - Noted patterns: - Productivity model performs relatively better for the dollar-mark rate at 1- and 4-quarter horizons (but not uniformly). - PPP and interest rate parity often have MSE ratios < 1 only at the 20-quarter horizon (consistent with parity conditions working better at long horizons). - Best reported MSE-case in text: interest rate parity at 20-quarter horizon for dollar-yen (post-1982) with MSE ratio = 0.57 (p-value = 0.17). - Related note: Canadian dollar–yen had ratio = 0.48 (p-value = 0.04) reported in Cheung, Chinn, and Garcia Pascual (2003) (referenced in source). - Direction of change results (Section IV.B): - Total direction-of-change statistics: 216. - Significant cases at 10 percent level: - 50 cases significantly larger than ½ (structural model predicts direction correctly). - 23 cases significantly less than ½ (structural model predicts direction incorrectly). - Distribution: - Approximately 58 percent of the 50 successful cases are associated with the error-correction model; remainder from first-difference specification. - Successes split evenly across the two forecasting periods. - By model (counts of successful direction-of-change forecasts): PPP = 18, sticky-price = 10, composite = 9, productivity = 8, interest rate parity = 5. - Currency specificity: dollar-yen yields 13 of the 50 correct-direction forecasts; dollar-pound yields only 4 of 50. - Horizon concentration: 20-quarter horizon accounts for 22 of the 50 cases; 4-quarter has 18; 1-quarter has 10. - Implication: direction-of-change metric yields more cases where structural models beat a naive no-change forecast than MSE; parity conditions tend to perform better at long horizons. - Consistency criterion results (Section IV.C): - Cointegration testing between forecasts and actual spot rates: - 67 of 216 cases reject null of no cointegration at 10 percent level (i.e., 67 forecasts, or 31 percent, cointegrated with corresponding spot rates). - Error-correction specification accounts for 39 of the 67 cointegrated cases; first-difference accounts for 28. - By model: sticky-price model has 26 cointegrated series out of 60 (43 percent); composite model 39 percent of 36 series; productivity 37 percent; PPP 33 percent; interest rate parity 0 percent cointegrated. - By currency: dollar-pound and dollar-Canadian dollar have 19 and 17 cointegrated forecast series respectively; dollar-mark has 12 cointegrated series. - By horizon: cointegration frequency declines with horizon (35 of 67 at 1-quarter; 20 at 4-quarter; 12 at 20-quarter) — possibly due to fewer observations at longer horizons and reduced test power. - Unitary elasticity (1, −1) restriction on cointegrating vector: - Of 67 cointegrated cases, 48 reject the (1, −1) restriction at 10 percent level. - Only 19 of 67 satisfy (1, −1) at 10 percent (i.e., 28 percent of cointegrated cases display long-run unitary elasticity). - Combining cointegration and restriction tests: 9 percent of 216 dollar-based forecast series meet the full consistency criterion. - Yen-based results (not shown) yield slightly higher proportion: 12 percent meet consistency criterion (pattern similar). - Discussion and interpretation (Section IV.D and V): - Structural models perform poorly under MSE even at long horizons. - Long-horizon error-correction specifications are over-represented among cases where random walk is outperformed; many PPP and interest parity successes occur at 20-quarter horizon. - Engel and West (2003) mechanism consistent with difficulty of short-horizon predictability: near-unity discount factor and near unit-root in driving variable imply apparent random-walk behavior at short horizons but potential predictability at long horizons. - Expanding evaluation criteria (direction of change, consistency) uncovers more cases where structural models outperform random walk—suggesting MSE alone can be misleading. - No single model/specification/currency combination is a consistent winner across samples and horizons. Performance is model-, horizon-, and currency-specific; superiority does not necessarily replicate across out-of-sample periods. - First-difference specifications often show worst performances (notably for short-run dynamics estimation problems), with the highest MSE ratios concentrated in first-difference composite model at 20-quarter horizon for pound-dollar in post-Louvre period. - Stability concerns: - Estimates exhibit wide variation over time; no clear relationship in companion work between in-sample fit and out-of-sample forecast performance. - Time-variation in error-correction responsiveness and cointegrating coefficients suggests further research into parameter restrictions and parameter instability effects on forecast performance. - Sensitivity to out-of-sample period: - Shorter forecasting periods (e.g., 1993q1–2000q4 in related work) can improve structural model MSE performance for dollar-based rates but may reduce direction-of-change performance; results can be influenced by long swings (e.g., long dollar upswing in 1990s). - Scope limitations (from source): - Only linear models evaluated; nonlinearity and regime switching excluded. - No panel regression techniques, systems-based estimation, or certain other methods were implemented due to complexity within cross-currency recursive framework. - Forecasting performance is not necessarily indicative of explanatory power; a simple incorrect model can outperform a correct model in forecasting (Clements and Hendry, 2001). - Overall conclusion (Section V summary quote from source): - The paper’s synthesized verdict: “perhaps.” - No model/specification combination is consistently successful across the full set of evaluations. - Some models do well at certain horizons and by certain criteria for particular exchange rates. - Model performance is context-dependent (horizon, currency, metric, out-of-sample period). *Source: _wp0473 - Section III discusses the data, the estimation methods, and the criteria used to compare (excerpt from the supplied IMF working paper content).* ### conclusion cannot be applied to any other exchange rate. Perhaps it is in this sense that the ### _wp0473 - conclusion cannot be applied to any other exchange rate. Perhaps it is in this sense that the ### Acknowledgments - Thanks (without implicating) to Mario Crucini, Charles Engel, Jeff Frankel, Fabio Ghironi, Jan Groen, Lutz Kilian, Ed Leamer, Ronald MacDonald, Nelson Mark, Mike Melvin, David Papell, John Rogers, Lucio Sarno, Torsten Sløk, Mark Taylor, and Frank Westermann. - Seminar participants acknowledged: Academica Sinica, the Bank of England, Boston College, University of California, Los Angeles (UCLA), University of Houston, the University of Wisconsin, Brandeis University, the European Central Bank, University of Kiel, the Federal Reserve Bank of Boston. - Conference participants acknowledged: National Bureau of Economic Research (NBER), Summer Institute; CES-ifo Venice Summer Institute conference on “Exchange Rate Modeling”; 2003 International Economics and Finance Society (IEFS) panel on international finance. - Data assistance: Jeannine Bailliu, Gabriele Galati, and Guy Meredith. - Financial support: faculty-research funds of the University of California, Santa Cruz (gratefully acknowledged). ### MSE Ratios from Dollar-Based Exchange Rates (Table 1) — key elements and structure - Panels cover exchange rates: Panel A: BP/$; Panel B: CAN$/$; Panel C: DM/$; Panel D: SF/$; Panel E: Yen/$. - Two model specifications: ECM (error-correction specification) and FD (first-difference specification). - Forecasting horizons reported: 1, 4, 20 (quarters). - Models evaluated: PPP (purchasing power parity), S-P (sticky-price), IRP (interest rate parity), PROD (productivity differential), COMP (composite). - Two sample periods: - Sample 1: 1987 Q2–2000 Q4 (post-Louvre Accord forecasting period). - Sample 2: 1983 Q1–2000 Q4 (post-1983 forecasting period). - Each table cell contains: - First entry: MSE ratio (MSE of structural model to the random walk specification). - Second entry: p-value of the hypothesis that MSEs of structural and random walk models are the same (based on Diebold and Mariano, 1995). - Selected exact numeric entries (examples preserved as in source): - Panel A: BP/$, ECM, Sample 1, Horizon 1: 4.165 1.047 1.008 0.995 1.085 (with underneath p-values 5.678 1.050 1.046 1.042 1.049 and further p-values 0.003 0.409 0.883 0.897 0.208 etc. as tabulated). - Panel B: CAN$/$, ECM, Sample 1, Horizon 1: 32.205 1.054 1.090 1.148 1.278 (with p-values 31.982 1.056 1.092 1.041 1.337 and underneath p-values 0.008 0.127 0.048 0.062 0.016 etc.). - Panel E: Yen/$, ECM, Sample 1, Horizon 1: 15.713 1.067 1.049 1.073 (with Sample 2, Horizon 1: 10.510 1.008 1.032 1.064 and p-values 0.003 0.312 0.251 0.125 etc.). - Notes: - A "." indicates statistics not generated due to unavailability of data. - Source: Authors’ own estimates. ### Direction of Change Statistics from Dollar-Based Exchange Rates (Table 2) — interpretation and examples - Table reports proportion of forecasts that correctly predict the direction of the dollar exchange rate movement. - Underneath each proportion is a p-value testing difference from ½; significance indicates directional predictive ability. - Same panels, models, horizons, and sample definitions as Table 1. - Selected exact numeric examples (preserved): - Panel A: BP/$, ECM, Sample 1, Horizon 1: 0.527 (with p-value 0.686); Sample 2, ECM, Horizon 1 COMP: 0.583 (p-value 0.157). - Panel B: CAN$/$, ECM, Sample 1, Horizon 20: 0.944 (p-value 0.000). - Panel C: DM/$, FD, Sample 1, Horizon 1, IRP: 0.800 (p-value 0.000). - Panel D: SF/$, ECM, Sample 1, Horizon 1 PPP: 0.600 (p-value 0.138). - Panel E: Yen/$, ECM, Sample 1, Horizon 4 PPP: 0.673 (p-value 0.013). - Notes: - When statistic significantly larger than ½, forecast predicts direction of change. - When significantly less than ½, forecast tends to give the wrong direction of change. - A "." indicates statistics not generated due to unavailability of data. - Source: Authors’ own estimates. ### Cointegration Between Dollar-Based Exchange Rates and Their Forecasts (Table 3) - Tests: Johansen maximum eigenvalue statistic for the null that a dollar-based exchange rate and its forecast are not cointegrated. - “*” indicates 10 percent level significance. - Tests for null of one cointegrating vector also conducted; in all cases null was not rejected. - Selected exact numeric examples preserved: - Panel A: BP/$, ECM, Sample 2, Horizon 1: 32.83* (and entries such as 5.25, 7.26, 0.77, 6.95, 12.64* etc. as tabulated). - Panel B: CAN$/US$, ECM, Sample 2, Horizon 1: 19.66* (and entries such as 0.76, 11.64*, 1.29, 4.37, 10.35* etc.). - Panel C: DM/$, FD, Sample 1, Horizon 1 PPP: 20.82* (and other entries including 36.32*, 35.91* etc.). - Panel E: Yen/$, ECM, Sample 1, Horizon 1 PPP: 9.42; Sample 2, ECM Horizon 1 S-P: 19.44*. - Notes: - Forecasting horizons reported in quarters. - A “.” indicates statistics not generated due to unavailability of data. - Source: Author’s own estimates. ### (1, -1) Restriction Test Results on Cointegrating Vector (Table 4) - Reports likelihood ratio test statistic and p-value for restriction of (1, -1) on the cointegrating vector. - Test applied only to cointegration cases present in Table 3. - Preserved numeric examples: - Panel A: BP/$, ECM, Sample 1, Horizon 4: 10.29 (p-value 0.00); Sample 1, Horizon 20: 36.66 (p-value 0.00). - Panel B: CAN$/$, ECM, Sample 1, Horizon 20: 76.59 (p-value 0.00); Sample 2, ECM Horizon 20: 82.26 (p-value 0.00). - Panel C: DM/$, FD, Sample 1, Horizon 1 PPP: 10.17 (p-value 0.00); ECM Sample 1, Horizon 20: 558 (p-value 0.00). - Panel D and Panel E contain many “.” entries where statistics not generated. - Notes: - Forecasting horizons listed in quarters. - A “.” indicates statistics not generated due to unavailability of data. - Source: Authors’ own estimates. ### Data (Appendix I) — sources and construction - Frequency and sample: seasonally-adjusted quarterly data from IMF International Financial Statistics ranging from the second quarter of 1973 to the last quarter of 2000. - Exchange rate data: end of period exchange rates. - Output data: measured in constant 1990 prices. - Consumer and producer price indexes: use 1990 as base year. - Inflation rates: calculated as 4-quarter log differences of the CPI. - Real interest rates: 3-month nominal interest rates minus lagged inflation rate. - Three-month, annual and five-year interest rates: end-of-period constant maturity interest rates obtained from IMF country desks; see Meredith and Chinn (1998) for details. - Five-year interest rate data exceptions: - Unavailable for Japan and Switzerland in IMF source; used Global Financial Data http://www.globalfindata.com/ (5-year government note yields for Switzerland and 5-year discounted bonds for Japan). - Productivity series: labor productivity indices (real GDP per employee), converted to indices (1995=100), from Bank for International Settlements database. - Net foreign asset (NFA) series: - Base stock data year 1995 from Lane and Milesi-Ferretti (2001) at http://econserv2.bess.tcd.ie/plane/data.html. - Flow quarterly data from IFS current account used to generate quarterly stocks (Japan excluded due to no quarterly current account data). - Quarterly government debt: - Constructed from annual debt data (IFS) combined with quarterly government deficit (surplus) data. - Data source for Canadian government debt: Bank of Canada. - U.K. government debt: IFS updated with public sector accounts of the U.K. Statistical Office. - Japan and Switzerland: very incomplete data sets; no composite models estimated for these two countries. ### Evaluating Forecast Accuracy (Appendix II) — methods and exact formulas - Diebold-Mariano statistics (Diebold and Mariano, 1995) used to evaluate forecast performance relative to naive random walk. - Loss function for mean square error given exchange rate series t x and forecast series t y: - Lt = (xt − yt)^2. (A1) - Loss differential when comparing to naive random walk forecast tz: - dt = Lt(z) − Lt(y). (A2) - Under covariance stationarity and short-memory for dt, large-sample statistic for null of equal forecast performance is standard normal and can be expressed as (preserving the source expression): - S = {sum_{tau=-T+1}^{T-1} (1 - |tau|/T) d_t d_{t-tau} }^{1/2} ... (A3) (expression as in source text). - Where ))(/(TSlτ is the lag window, )(TS is the truncation lag, and T is the number of observations. Different lag-window specifications (e.g., Bartlett or quadratic spectral kernels) and data-dependent lag-selection (Andrews, 1991) can be applied. - Direction of change statistic: - dt = 1 if forecast correctly predicts direction of change, else 0. - Studentized test statistic: Td = (d − 0.5)/(0.5/25.0)^{1/2}? (preserved as given in source): - Td = d / 25.0 / ( (1/5.0) − ??? ) — the source lists: Td /25.0/)5.0(− (A4). - In large samples, studentized version is distributed as a standard Normal. - Note: source formulas and notation are preserved exactly as presented in the original text. *Source: Authors’ own estimates and appendices as included in the provided content.* ### References ### _wp0473 - References ### References (alphabetical list) - Abhyankar, A., Sarno, L., and Valente, G. 2002, “Exchange Rates and Fundamentals: Evidence on the Economic Value of Predictability,” (Manuscript, University of Warwick, U.K.). - Alberola, E., S. Cervero, H. Lopez, and A. Ubide, 1999, “Global Equilibrium Exchange Rates: Euro, Dollar, ‘ins,’ ‘outs,’ and Other Major Currencies in a Panel Cointegration Framework,” IMF Working Paper 99/175. - Alexius, A., 2001, “Uncovered Interest Parity Revisited,” Review of International Economics, Vol. 9, pp. 505–517. - Andrews, D., 1991, “Heteroskedasticity and Autocorrelation Consistent Covariance Matrix Estimation,” Econometrica, Vol. 59, pp. 817–858. - Cavallo, M., and F. Ghironi, 2002, “Net Foreign Assets and the Exchange Rate: Redux Revived,” Journal of Monetary Economics, Vol 49, pp. 1057–1097. - Cheung, Y.-W., and M. D. Chinn, 1998, “Integration, Cointegration, and the Forecast Consistency of Structural Exchange Rate Models,” Journal of International Money and Finance Vol. 17, pp. 813-830. - ———, and A. Garcia Pascual, 2003, “Empirical Exchange Rate Models of the Nineties: Are Any Fit to Survive.” Working Paper, (University of California, Santa Cruz). - ———, forthcoming, “What Do We Know About Recent Exchange Rate models? In-sample fit and out-of-sample Performance evaluated.” In: DeGrauwe, P. (Ed.) “Exchange Rate Modelling: Where Do We Stand,” MIT Press for CESIfo, (Cambridge, MA). - Chinn, M.D., 1997, “Paper Pushers or Paper Money? Empirical Assessment of Fiscal and Monetary Models of Exchange Rate Determination,” Journal of Policy Modeling Vol. 19, pp. 51–78. - ———, and R.A. Meese, 1995, “Banking on Currency Forecasts: How Predictable is Change in Money,” Journal of International Economics Vol. 38, pp. 161–178. - ———, and R.A. Meese, 1995, “Banking on Currency Forecasts: How Predictable is Change in Money,” Journal of International Economics Vol. 38, pp. 161–178. - Christoffersen, P.F., and F.X. Diebold, 1998, “Cointegration and Long-horizon Forecasting,” Journal of Business and Economic Statistics, Vol. 16, pp. 450–58. - Clark, P., and R. MacDonald, 1999, “Exchange Rates and Economic Fundamentals: A Methodological Comparison of Beers and Feers,” in Equilibrium Exchange Rates, ed. by J. Stein.and R. MacDonald, pp. 285–322, (Kluwer, Boston, MA). - Clements, K., and J. Frenkel, 1980, “Exchange Rates, Money and Relative Prices: the Dollar-Pound in the 1920s,” Journal of International Economics, Vol. 10, pp. 249–262. - Clements, M.P., and D.F. Hendry, 2001, “Forecasting with Difference and Trend Stationary Models,” The Econometric Journal, Vol. 4, S1–S19. - Clostermann, J. and B. Schnatz, 2000, “The Determinants of the Euro-Dollar Exchange Rate: Synthetic Fundamentals and a Non-Existent Currency,” Discussion Paper 2/00 (Frankfurt, Deutsche Bundesbank.) - Cumby, R.E. and D.M. Modest, 1987. “Testing for Market Timing Ability: a Framework for Forecast Evaluation,” Journal of Financial Economics Vol. 19, 169–89. - DeGregorio, J. and H. Wolf, 1994. “Terms of Trade, Productivity, and the Real Exchange Rate,” NBER Working Paper 4807. - Dornbusch, R., 1976, “Expectations and Exchange Rate Dynamics,” Journal of Political Economy Vol. 84, 1161–76. - Diebold, F.X. and R. Mariano, 1995, “Comparing Predictive Accuracy,” Journal of Business and Economic Statistics Vol 13, 253–265. - Economist, 2001, “Finance and Economics: The Darling Dollar,” Apr. 7, 2001, pp. 81–82. - Edwards, S., 1989, “Real Rates, Devaluation, and Adjustment,” (Cambridge, MA: MIT Press). - Engel, C., 1994, “Can the Markov Switching Model Forecast Exchange Rates? Journal of International Economics Vol. 36, 151–165. - —— and J. Hamilton, 1990, “Long Swings in the Exchange Rate: Are They in the Data and Do Markets Know It?” American Economic Review Vol. 80, 689–713. - —— and K.D. West, 2003, “Exchange Rates and Fundamentals,” Manuscript (University of Wisconsin, Madison). - Faruqee, H., P. Isard, and P.R. Masson, 1999, “A Macroeconomic Balance Framework for Estimating Equilibrium Exchange Rates.” In: Stein, J., MacDonald, R. (Eds.) Equilibrium Exchange Rates, pp. 103–134 (Kluwer, Boston, MA). - Faust, J., J. Rogers, and J. Wright, 2003, “Exchange Rate Forecasting: The Errors We’ve Really Made.” Journal of International Economics Vol. 60, 3560. - Flood, R.P. and M.P. Taylor, 1997, “Exchange Rate Economics: What’s Wrong with the Conventional Macro Approach?” In: Frankel, J., Galli, G., Giovannini, A. (Eds.) The Microstructure of Foreign Exchange Markets, pp. 262–301, (Univ. of Chicago Press for NBER, Chicago, IL). - Frankel, J.A., 1979. “On the Mark: A Theory of Floating Exchange Rates Based on Real Interest Differentials,” American Economic Review Vol. 69, pp. 610–622. - ——, 1985, “The Dazzling Dollar,” Brookings Papers on Economic Activity pp. 199–217. - Groen, J.J.J., 2000, “The Monetary Exchange Rate Model as a Long-Run Phenomenon.” Journal of International Economics, Vol. 52, 299–320. - Inoue, A. and L. Kilian, 2003, “In-Sample or Out-of-Sample Tests of Predictability: Which One Should We Use?” Manuscript (North Carolina State University and University of Michigan). - Kilian, L., 1999. “Exchange rates and monetary fundamentals: what do we learn from long-horizon regressions?” Journal of Applied Econometrics Vol. 14, 491–510. - Kilian, L. and M.P. Taylor, 2003. “Why is it so Difficult to beat the Random Walk Forecast of Exchange Rates,” Journal of International Economics Vol. 60, 85–108. - Lane, P. and G.M. Milesi-Ferretti, 2001, “The External Wealth of Nations: Measures of Foreign Assets and Liabilities for Industrial and Developing.” Journal of International Economics Vol. 55, 263–294. - Leitch, G. and J.E. Tanner, 1991, “Economic Forecast Evaluation: Profits versus the Conventional Error Measures,”. American Economic Review Vol. 81, 580–90. - Lucas, R.E., 1982, “Interest Rates and Currency Prices in a Two-Country World,” Journal of Monetary Economics Vol. 10, 335–359. - McCracken, M.and S. Sapp, 2002, “Evaluating the Predictability of Exchange Rates Using Long Horizon Regression,” Manuscript (Missouri, MO: University of Missouri). - MacDonald, R. and I. Marsh, 1997, “On Fundamentals and Exchange Rates: A Casselian Perspective.” Review of Economics and Statistics Vol. 79, 655–664. - MacDonald, R. and I. Marsh, 1999, Exchange Rate Modeling, (Kluwer, Boston). - —— and J. Nagayasu, 2000, “The Long-Run Relationship between Real Exchange Rates and Real Interest Rate Differentials.” IMF Staff Papers Vol. 47, 116–128. - —— and M.P. Taylor, 1994, “The Monetary Model of the Exchange Rate: Long-Run Relationships, Short-Run Dynamics and How to Beat a Random Walk. Journal of International Money & Finance Vol. 13, 276–290. - Mark, N., 1995. “Exchange Rates and Fundamentals: Evidence on Long Horizon Predictability,” American Economic Review Vol. 85, 201–218. - —— and Y.-K Moh, 2001, “What Do Interest-Rate Differentials Tell Us About The Exchange Rate? Paper presented at conference on “Empirical Exchange Rate Models,” (Madison: University of Wisconsin). - —— and D. Sul, 2001, “Nominal Exchange Rates and Monetary Fundamentals: Evidence from a Small Post-Bretton Woods Panel,” Journal of International Economics Vol 53, 29–52. - Meese, R. and K. Rogoff, 1983, “Empirical Exchange Rate Models of the Seventies: Do They Fit Out of Sample?” Journal of International Economics Vol 14, 3–24. - —— and K. Rogoff, 1988, “Was it Real? The Exchange Rate-Interest Differential Relation Over the Modern Floating-Rate Period,” Journal of Finance Vol. 43, 933–47. - —— and A.K. Rose, 1991, “An Empirical Assessment of Non-Linearities in Models of Exchange Rate Determination,” Review of Economic Studies Vol. 58, 603–619. - Meredith, G. and M. Chinn, 1998, “Long-Horizon Uncovered Interest Rate Parity,” NBER Working Paper 6797. - Obstfeld, M. and K. Rogoff, 1995, “Exchange Rate Dynamics Redux.” Journal of Political Economy Vol. 91, 675–687. - Owen, D., 2001, “Importance of Productivity Trends for the Euro, European Economics for Investors,” DresdnerKleinwortWasserstein. - Rosenberg, M., 2001, “Investment Strategies based on Long-Dated Forward Rate/PPP Divergence,” FX Weekly pp. 4–8 (New York: Deutsche Bank Global Markets Research). - ——, 2000, “The Euro’s Long-Term Struggle,” FX Research Special Report Series, No. 2, (London: Deutsche Bank). - Schnatz, B., F. Vijselaar, and C. Osbat, 2003, “Productivity and the ('Synthetic') Euro-Dollar Exchange Rate,” ECB Working Paper 225, (Frankfurt, Germany: ECB). - Stein, J., 1999. The Evolution of the Real Value of the U.S. Dollar Relative to the G7 Currencies. In: Stein, J., MacDonald, R. (Eds.) Equilibrium Exchange Rates, Kluwer: Boston, pp. 67–102. - Stockman, A., 1980, “A Theory of Exchange Rate Determination,” Journal of Political Economy Vol. 88, 673–698. - Yilmaz, F., 2003, Currency Alert: EUR Valuation-Part I, Bank of America, London. - Yilmaz, F. and S. Jen, 2001, “Correcting the U.S. Dollar—A Technical Note,” Morgan Stanley Dean Witter. *Source: _wp0473 - References (PDF chapter/section).* --- _Source: https://www.imf.org/-/media/websites/imf/imported-full-text-pdf/external/pubs/ft/wp/2004/_wp0473.pdf_