Economic Forecasts: Too Smooth by Far?

Prakash Loungani and Jair Rodriguez
International Monetary Fund
World Economics
http://www.world-economics-journal.com/Default.aspx
Vol. 9, No. 2, April-June 2008

Two questions about the global economic outlook are a matter of active conjecture at the moment:

• First, will the U.S. economy go into a recession?

• Second, to what extent will the rest of the world be affected by developments in the U.S. economy? In the parlance of the day, will countries "decouple" from the United States?

To those turning to private sector economic forecasters for guidance on these questions, this article sounds a note of caution: the past performance of forecasters on these two questions leaves much to be desired. Few recessions have been forecast ahead of their arrival. Nor do forecasters take into account fully the dependence of economies on one another; we present evidence that the dependence on the U.S. economy of the other members of the G7 has been under-appreciated by forecasters in the past. In short, economies are coupled, but one country's forecasters often seem to be decoupled from those in other countries.

Why isn't the performance of economic forecasters better? There can be many reasons, but the one emphasized here is forecasters' excessive caution. Forecasts ought to be revised early and often so as to absorb new information promptly. But the evidence presented in this article suggests that forecasters take very long to reflect information in their forecasts. As a consequence, revisions to forecasts tend to be smooth.


This smoothness is not to be lauded: it suggests that forecasts are not efficient. Forecast efficiency requires that successive forecasts of the same event should be uncorrelated, a property emphasized by Nordhaus (1987) and explained lucidly by him as follows:

"If I can look at your most recent forecasts and accurately say, "Your next forecast will be 2% lower than today's", then you can surely improve your forecast" (Nordhaus, p. 673).

Efficient forecasts, Nordhaus demonstrated, should "appear jagged because they incorporate all news quickly. Inefficient forecasts appear smoother ... for they let the news seep in slowly." He conjectures that "the tendency to smooth one's forecasts is rooted in the way people think about the future ... We tend to break the good or bad news to ourselves slowly, taking too long to allow surprises to be incorporated into our forecasts."

Train[wreck]spotting

Recessions are the train derailments of economic activity. They are somewhat rare events but they can cause a lot of damage when they do occur. Spotting recessions requires unusual alertness on the part of forecasters to incoming economic information and a willingness to raise alarms about possible recessions, even at the risk that some of these calls will turn out to be wrong. However, the evidence shows that forecasters are unwilling or unable to signal that the economy is heading down the wrong track until a crash is imminent; and even then they initially underestimate the extent of the damage.

table1

Consider Table 1, which lists the G7 and seven large emerging market countries (dubbed the "EM7" in this article), which together account for nearly three-quarters of world GDP today. The table also lists the years since 1990 in which these economies experienced a recession, which is defined here as either a year in which real GDP declined (on a year-over-year basis) or a year in which there were two quarters of real GDP declines.1 The term recession thus has to be interpreted rather broadly to encompass cyclical downturns (as in the case of the United States in 1991 and 2001) and declines associated with crises of various kinds (e.g., the ERM crisis in 1992, the Mexican crisis in 1995 and the global financial crises in 1997-99). There were a total of 26 episodes of such recessions in our sample, accounting for about 12% of the total number of country-years in the sample.

To what extent were these recessions forecast in advance? To answer this question, we use data from the publication Consensus Forecasts, which has provided macroeconomic forecasts for industrialized countries on a monthly basis since October 1989. Over time, the coverage has expanded to encompass many emerging markets. For each country, the publication surveys a number of prominent financial and economic analysts, and reports their individual forecasts as well as simple statistics summarizing the distribution of forecasts. All the results in this paper are based on the average forecast (the so-called "consensus").

The event being forecast is annual average real GDP growth. Every month (or every other month in the case of many of the emerging markets), a new forecast is made of this event. To ensure consistency of treatment across the G7 and the EM7, we study the bi-monthly sequence of forecasts for each event. To take a concrete example, suppose that the event is 2007 real GDP growth. The sequence of forecasts that we study for this event are the twelve forecasts made between February 2006 and December 2007. The first six forecasts, the ones made during 2006, are referred to as year-ahead forecasts; the six forecasts made during 2007 are called current-year forecasts.

table2

Let's return now to the question posed earlier: how well did the consensus fare in predicting recessions?2 Only two recessions were predicted a year in advance and one of those predictions came toward the turn of the year, as shown in column 3 of Table 2. Requiring recessions to be predicted a year ahead may seem like an unreasonably high bar to set. Lowering the bar to the start of the year in which the recession occurred does indeed improve the performance somewhat: 8 of the 26 recessions were predicted in February of the year in which they occurred and 16 were predicted by August. But even as the year drew to a close, 6 of 26 recessions remained undetected by forecasters.

Moreover, while forecasters increasingly start to recognize recessions in the year in which they occur, the results in column 4 show that the magnitude of the recession is underpredicted in the vast majority of cases. For instance, even as late as December of the year of the recession, the forecast is more optimistic than the outcome in 15 cases.

The remaining columns of Table 2 show the average forecast error at the different forecast horizons over all 26 episodes and also for the G7 and EM7 countries separately. Note that average forecast errors continue to be quite substantial even for forecasts made fairly late in the year of the recession. For instance, in August the average forecast error is 0.6 percentage points for industrialized countries and 2.4 percentage points for developing countries.

Figure 1 shows the evolution of forecasts on average across the 26 episodes. The forecast in February of the year before is for about 2.5% growth. This forecast is slowly lowered over the course of the year and by the start of the year of the recession the average forecast is for a small decline in real GDP. It is only as the year is drawing to a close that the average forecast catches up with the reality of the recession. The impression one is left with is that of forecasters chasing the data rather than a step ahead of it.

figure1

Too smooth a ride

The tendency not to revise forecasts promptly, while particularly costly in a recession, is a feature of forecasts in all years. Our inspection and analysis of forecasts for the 14 countries in non-recession years shows that forecasts are revised in a very smooth fashion: the forecast revisions—the changes in the forecast between successive horizons—are smooth.

In the two decades since the 1987 paper by Nordhaus that was mentioned at the outset, further evidence has accumulated of excessive smoothness in forecasts in a variety of different contexts. One interesting recent study is that of crop production forecasts made by the U.S. Department of Agriculture (USDA) over a thirty-year period [Isengildina, Irwin and Good (2006)]. The authors cite a representative of a large agribusiness firm who commented, in the spirit of Nordhaus's quotes, that

"he had noticed that, if [USDA] corn forecasts go up from September to October, they almost always go up from October to November."

He appears to have been right. After subjecting the data on forecasts revisions to a battery of careful statistical tests, the authors concluded that there was "systematic under-adjustment of forecasts": "overall, the results ... revealed the presence of smoothing in most corn and soybean production forecast revisions." Since USDA forecasts play a big role in "business decisions by farmers and agribusiness firms and also have an impact on government policy," this smoothing behavior can be costly unless users become aware of this tendency and correct for it.

Nordhaus's idea of inspecting forecast revisions can also be used to show that forecasters tend to live in silos in the sense that they are reluctant to absorb quickly information from other countries. In other words, while economies are coupled, forecasters tend to be somewhat decoupled: forecasters for one country's growth take a long time to absorb relevant information from other countries.

Decoupled forecasters?

Evidence for such decoupling can be presented by looking at the relationship between revisions for a country's forecasts and the revisions for other countries. If forecasts incorporate all the available information efficiently, forecast revisions for a country's forecasts should be uncorrelated not only with past values of that country's forecast revisions but with past values of the forecast revisions for other countries. As Isiklar, Lahiri and Loungani (2006) suggest, a simple way of testing whether this is the case is to estimate a vector autoregressive model for the forecast revisions for a set of countries.3 Implementation of this idea for the G7 countries yields the results discussed below.

To begin with, the estimated model pinpoints the countries in which it is particularly important that forecasters pay attention to events outside their own borders. Figure 2 shows the forecast error variance decompositions into own (i.e. domestic) and foreign news at different months. The countries break out into three groups. In the first group are the United States and Japan, countries where real growth is influenced more by domestic developments than foreign news. The second group consists of France and Canada, where the influence of foreign news is quite important. In France, at a horizon of ten months, domestic and foreign news are equally important in contributing to the forecast error variance, and this is nearly the case in Canada as well. For the countries in the third group—the United Kingdom, Germany and Italy—the influence of foreign news is somewhat greater than is the case for Japan and the U.S. but not as important as in France and Canada. This suggests that, among the G7 economies, forecasters in the second group and perhaps also in the third group should be particularly alert to developments outside their own economies.

figure2

In the past, however, forecasters have not given foreign news the degree of attention it deserves and are slow in absorbing news from other countries. Figure 2 shows selected impulse responses from the estimated model. Under perfect efficiency, forecast revisions should respond fully to the shocks immediately; the impulse responses should be zero beyond the first month. If the forecast revisions do not respond to the shocks immediately, i.e. if the impulse response values are not zero for months beyond the first, forecasts are not efficiently using the information immediately, and some of the information is being utilized in the later forecast revisions. In other words, the impulse responses of the forecast revisions show the dynamics of how news gets absorbed in the forecast revisions over time. The longer it takes for the responses to go to zero, the lower is the degree of forecast efficiency. The top left panel (labeled "Response of UK to UK") shows the extent to which forecast revisions for UK growth are correlated with its past revisions. This reiterates the points made earlier in the article that forecasters are slow to respond to news, even about their own country.

figure3

The other three panels show, for selected cases, how revisions are correlated with revisions for other countries. The top right-hand panel shows how slowly forecasts of UK growth absorb U.S. news—even well past the 12 month horizon, UK growth forecast revisions are correlated with US forecast revisions. The bottom panels show that forecasts for Canadian growth and French growth respond very slowly to US news and German news, respectively. This inefficiency can be quite costly given the importance of foreign news for Canada and France that was noted in Figure 2.

Summing-up

This article has used data on revisions of growth forecasts to demonstrate that for 14 major economies there is excessive smoothness in forecasts. Of course, some caution is desirable given the fact that monthly and quarterly data on economic activity are often extensively revised and the structure of economies is always changing. But the rate of absorption of news is so slow that it limits the usefulness of forecasts. This can be particularly costly around turning points, such as at the present conjuncture.

References

Clements, M. P. (1995), "Rationality and the Role of Judgment in Macroeconomic Forecasting," Economic Journal, 105: 410-420.

Clements, M. P. (1997), "Evaluating the Rationality of Fixed-Event Forecasts," Journal of Forecasting, 16: 225-239.

Davies, A. and K. Lahiri (1995), "A New Framework for Analyzing Survey Forecasts Using Three-Dimensional Panel Data," Journal of Econometrics, 68: 205-227.

Davies, A. and K. Lahiri (1999), "Reexamining the Rational Expectations Hypothesis Using Panel Data on Multi-Period Forecasts," in Analysis of Panels and Limited Dependent Variable Models (eds. C. Hsiao et al.), Cambridge University Press: Cambridge, 226-254.

Fintzen, David, and H. O. Stekler (1999), "Why Did Forecasters Fail to Predict the 1990 Recession?" International Journal of Forecasting, 15: 309-323.

Isengildina, Olga, Scott H. Irwin and Darrel L. Good (2006), "Are Revisions to USDA Crop Production Forecasts Smoothed?" American Journal of Agricultural Economics, 88: 1091-1104.

Isiklar, Gultekin, Kajal Lahiri and Prakash Loungani (2006), "How Quickly Do Forecasters Incorporate News? Evidence from Cross-Country Surveys," Journal of Applied Econometrics, 21: 703-25.

Nordhaus, W. (1987), "Forecasting Efficiency: Concepts and Applications," The Review of Economics and Statistics, 69: 667-674.
ACKNOWLEDGEMENTS

An earlier version of this paper was presented at the Research Program on Forecasting seminar at George Washington University. We thank Fred Joutz, Tara Sinclair, Herman Stekler and other participants at the seminar for very useful comments. We are also grateful for review by our IMF colleagues Selim Elekdag and Subir Lall.


1 An exception is the 2001 U.S. recession-it does not meet our criteria but we use it nevertheless as it is widely regarded as a recession and was termed as such by the National Bureau of Economic Research.

2 Fintzen and Stekler (1999, p.309) note that "one of the most disturbing findings of forecast evaluations is that, in the United States, recessions have generally not been predicted prior to their occurrence." One country's macroeconomic history only yields a few observations of recessions, but the cross-country sample used here demonstrates that the failure to forecast recessions is a common feature of growth forecasts.


3 See the Appendix in the original article for a brief technical description of this procedure.



IMF EXTERNAL RELATIONS DEPARTMENT

Public Affairs    Media Relations
E-mail: publicaffairs@imf.org E-mail: media@imf.org
Fax: 202-623-6220 Phone: 202-623-7100