## IMPLEMENTATION PLAN IN RESPONSE TO THE BOARD-ENDORSED RECOMMENDATIONS FOR THE IEO EVALUATION OF IMF FORECASTS—PROCESS, QUALITY, AND COUNTRY PERSPECTIVE

## Source details

**Canonical URL:** [IMPLEMENTATION PLAN IN RESPONSE TO THE BOARD-ENDORSED RECOMMENDATIONS FOR THE IEO EVALUATION OF IMF FORECASTS—PROCESS, QUALITY, AND COUNTRY PERSPECTIVE](https://www.imf.org/-/media/files/publications/pp/pp091815c-implementation-plan.pdf)

## Other formats

- [Markdown version](/-/media/files/publications/pp/pp091815c-implementation-plan.pdf.md)
- [Structured JSON version](/-/media/files/publications/pp/pp091815c-implementation-plan.pdf.json)

---

### Executive summary and objectives
- Management’s response outlines an implementation plan (MIP) addressing the five IEO recommendations that received broad Board support:
  - maintain commissioning external evaluations of IMF forecasts;
  - enhance processes and incentives for learning from past forecast performance;
  - extend guidance to desk economists on forecasting methodologies;
  - publish a description of the WEO forecasting process; and
  - improve public availability of data related to forecasts and outturns.
- Several actions already implemented; others underway.
- Implementation monitoring: forthcoming Periodic Monitoring Reports (PMRs), regularly commissioned external evaluations, and staff evaluations of ICD training courses.

### Implementation actions by IEO recommendation
- Recommendation 1 — Maintain external evaluations (every 5 to 8 years):
  - Commissioned reports: external evaluation to be released by end-2015; RES to commission and publish similar studies at regular intervals every 5 to 8 years.
  - Terms of reference to require: (i) spell out recommendations, (ii) include an assessment of implementation status of previously accepted recommendations, (iii) include an assessment of the forecasting process itself.
  - Staff responses to the studies’ recommendations will be published.
  - Circulation: commissioned external evaluation reports will be issued to the Executive Board for information; Board briefing can follow if requested.

- Recommendation 2 — Enhance learning from past forecast performance:
  - More systematic review of forecasts:
    - RES developed a beta data processing tool to display forecast errors for key variables.
    - Reports on forecast errors for real GDP growth, inflation, and current account balances generated by this tool are provided to country desks at the beginning of each WEO round (practice started in July 2014).
    - From January 2016 onward, policy consultation meetings for Article IV consultations and program reviews will discuss underlying causes of significant forecast errors for key macroeconomic variables (SPR reviewers will monitor implementation).
    - WEO will discuss forecast errors following significant revisions to global growth forecasts (practice initiated with the October 2014 WEO).
    - Modalities for SPR review process to be developed in consultation with area and relevant functional departments and posted by end-November on the SPR website about the Review process.
  - Knowledge transfer / handover:
    - All area and functional departments have issued guidelines with a checklist for handover of country assignments (including forecasting process).
    - SPR to conduct a survey in the fall of 2015 to assess implementation; staff guidelines updated as warranted.
    - Handover guidelines to be integrated in area department data management guidelines by end-2016 and monitored as part of the bi-annual audit of data management guidelines.
    - Managers expected to include assessment of desk transitions in the Annual Performance Review (APR) of desk economists.
  - Training:
    - ICD and HRD mandated to strengthen learning environment.
    - Interdepartmental working group led by ICD revamped the internal economics training program (IET) into a structured curriculum with courses in surveillance, finance, monetary and fiscal policy, macro-financial linkages, statistics and empirical methods at multiple levels.
    - An online course on macroeconomic forecasting was added in May 2015 and will be opened to outside audiences in October 2015 as part of the Fund’s MOOCs.
    - Five self-contained modules on advanced empirical methods complement the course.
    - Over 130 Fund staff signed up for the first offering of the online forecasting course.

- Recommendation 3 — Extend guidance on forecasting methodologies (short- and medium-term):
  - RES to establish a process for top-down guidance as an input in medium-term forecasts.
  - A multi-country macro-econometric forecasting model with stock-flow consistency (e.g., between current account balances and net foreign assets) has been developed.
  - Pilot schedule:
    - Piloted with several country desks in the run-up to the October 2015 WEO.
    - Extended to more desks for the April 2016 WEO.
    - Gradually applied to most major economies and country groups.
  - Medium-term consistency checks will complement existing short-term checks.
  - The revamped curriculum on empirical methods will support desk economists in incorporating advances in forecasting methodologies.

- Recommendation 4 — Publish WEO forecasting process description:
  - A general description of the WEO forecasting process will be posted on the external IMF website with the release of the October 2015 WEO.
  - Material will be periodically reviewed and revised.

- Recommendation 5 — Public availability of forecast and outturn data:
  - WEO data:
    - A database showing current and past forecasts and outcomes for real GDP growth, CPI inflation, and current account balances was posted on the IMF’s external website in April 2015.
    - The database will be updated annually and could be later extended to other series depending on demand and resource costs.
    - Reference: https://www.imf.org/external/pubs/ft/weo/2015/01/weodata/index.aspx
  - Program data:
    - The Monitoring of Fund Arrangements (MONA) database was extended to include all past forecasts linked to each program review and will be updated continuously.
    - Reference: http://www.imf.org/external/np/pdr/mona/index.aspx

### Monitoring, governance, and accountability
- Overall coordination: SPR in close collaboration with RES and ICD.
- Monitoring mechanisms:
  - Periodic Monitoring Reports (PMRs) to report progress.
  - Commissioned external evaluations to regularly assess forecast quality and process.
  - ICD to evaluate internal course relevance using participant feedback.
- Annex I — implementation accountability summary (selected):
  - Recommendation 1: RES responsible for commissioning studies, preparing staff responses, and implementing follow-up actions. Next study due by end-2015; recurring every 5–8 years.
  - Recommendation 2: RES to monitor forecast errors; SPR to monitor compliance with handover procedures and conduct a Fall 2015 survey; area departments to discuss significant forecast errors in policy consultation meetings; ICD to deliver and update forecasting course offerings. Handover guidelines to be integrated by end-2016. From January 2016, policy consultations to discuss significant forecast errors.
  - Recommendation 3: RES to pilot top-down guidance for medium-term forecasts beginning Fall 2015; pilots in run-up to October 2015 WEO and extended for April 2016 WEO.
  - Recommendation 4: General description of WEO forecasting process to be posted with October 2015 WEO and periodically reviewed.
  - Recommendation 5: WEO database posted April 2015; updated annually. MONA database extended and updated continuously.

### Resource implications and staffing (numeric details preserved)
- Resources allocated so far (including in the FY2016 budget) for MIP implementation amount to about 7 FTEs and an additional 0.25 FTE would still need to be accommodated (Annex 2).
- The bulk of costs relate to development and rollout of the medium-term macro-econometric forecasting model (model development was part of a RES capital project).
- Recurrent annual costs from FY2017 onward for the MIP measures amount to about 4.9 FTEs, with an additional ¼ FTE required every 5–8 years to prepare the commissioned study on IMF forecasts.
- Estimates include desk time to implement MIP recommendations and process guidance from the medium-term forecasting model.
- Training costs assumed to be accommodated within existing resources, possibly by refocusing training priorities.
- Resource implications will be reassessed after a critical mass of pilot cases for the medium-term forecast model.

- Annex II — summary table of resource figures (as listed in source):
  - Recommendation 1:
    - Resource Cost through FY 2016: 0.25
    - Future Recurrent Costs: 0.25 (every 5–8 years)
  - Recommendation 2 activities (itemized FTEs as listed in source):
    - 0.60
    - 0.10
    - 0.05
    - 0.05
    - 0.10
    - 0.25
    - 0.05
    - 0.35
    - 0.10
    - 0.05
    - 0.15
    - 0.00
    - 0.00
    - 0.05
  - Recommendation 3 activities and resource lines (FTEs as listed):
    - 6.25
    - 3.00
    - 0.25
    - 4.50
    - 3.00
    - 1.50
  - Initial development of multi-country macro-econometric forecasting model:
    - Resource Cost through FY 2016: 3.00
    - Future Recurrent Costs: 0.00
  - Recommendation 4:
    - Resource Cost through FY 2016: 0.01
    - Future Recurrent Costs: 0.00
  - Recommendation 5 activities (WEO/MONA development and maintenance; FTEs as listed):
    - Resource Cost through FY 2016: 0.20; 0.10; 0.05; 0.05; 0.05 (individual items as listed)
    - Future Recurrent Costs (FTEs per year): 0.00; 0.00; 0.05

*Source: pp091815c-implementation-plan - Section 1 and Section 2 (Implementation Plan in Response to the IEO Evaluation of IMF Forecasts), September 18, 2015.*

### Section 1

### IMPLEMENTATION PLAN IN RESPONSE TO THE BOARD-ENDORSED RECOMMENDATIONS FOR THE IEO EVALUATION OF IMF FORECASTS—PROCESS, QUALITY, AND COUNTRY PERSPECTIVE

### Executive Summary
- Management’s response outlines an implementation plan (MIP) addressing the five IEO recommendations that received broad Board support:  
  - maintain commissioning external evaluations of IMF forecasts;  
  - enhance processes and incentives for learning from past forecast performance;  
  - extend guidance to desk economists on forecasting methodologies;  
  - publish a description of the WEO forecasting process; and  
  - improve public availability of data related to forecasts and outturns.  
- Several actions have already been implemented following the Board discussion; others are underway.  
- Implementation monitoring: forthcoming Periodic Monitoring Reports (PMRs), regularly commissioned external evaluations, and staff evaluations of ICD training courses.

### Introduction
- The MIP responds to the IEO evaluation “IMF Forecasts: Process, Quality, and Country Perspectives.”  
- Board reaction: Directors welcomed the evaluation, noted broadly positive findings on forecast quality and country confidence, and supported recommendations to improve learning from forecast errors, transparency, and adoption of best practices.  
- Monitoring: PMRs, regular external evaluations, and ICD course evaluations will track implementation and relevance for desk economists.

### IEO Recommendations and Board Reactions
- The five high-level IEO recommendations (exact wording preserved):  
  - Maintain the practice of commissioning external evaluations of IMF forecasts by recognized experts.  
  - Enhance processes and incentives for learning from past forecast performance.  
  - Extend guidance to desk economists about how best to incorporate advances in forecasting methodologies for both short- and medium-term forecasts.  
  - Publish online a general description of the WEO forecasting process, which should be reviewed and revised as needed.  
  - Make available to the public data related to forecasts and outturns that are readily available internally.  
- Board endorsement: Executive Directors broadly endorsed all five recommendations. Directors noted resource constraints in facilitating public access to historical forecast and outturn data and emphasized weighing benefits against costs. The Managing Director supported all five recommendations but cautioned on feasibility and resource payoffs for some suggested actions.

### Implementation Plan — Actions and Monitoring
- Overall coordination: Implementation will be coordinated by SPR in close collaboration with RES and ICD. Commissioned external evaluations will monitor forecast quality and process; ICD will evaluate internal course relevance.

- Recommendation 1 — Maintain external evaluations (every 5 to 8 years):  
  - Commissioned reports: An external evaluation will be released by end-2015. RES will commission and publish similar studies at regular intervals every 5 to 8 years. Terms of reference will require studies to (i) spell out recommendations, (ii) include an assessment of implementation status of previously accepted recommendations, and (iii) include an assessment of the forecasting process itself. Staff responses to the studies’ recommendations will be published.  
  - Circulation to the Board: Commissioned external evaluation reports will be issued to the Executive Board for information; a Board briefing can follow if requested.

- Recommendation 2 — Enhance learning from past forecast performance:  
  - More systematic review of forecasts: RES developed a beta data processing tool to display forecast errors for key variables. Reports on forecast errors for real GDP growth, inflation, and current account balances generated by this tool are provided to country desks at the beginning of each WEO round (practice started in July 2014). From January 2016 onward, policy consultation meetings for Article IV consultations and program reviews will discuss underlying causes of significant forecast errors for key macroeconomic variables (SPR reviewers will monitor implementation). WEO will discuss forecast errors following significant revisions to global growth forecasts (practice initiated with the October 2014 WEO). Modalities for SPR review process will be developed in consultation with area and relevant functional departments and posted by end-November on the SPR website about the Review process.  
  - Knowledge transfer / handover: All area and functional departments have issued guidelines with a checklist for handover of country assignments (including forecasting process). SPR will conduct a survey in the fall of 2015 to assess implementation; staff guidelines will be updated as warranted. Handover guidelines will be integrated in area department data management guidelines by end-2016 and monitored as part of the bi-annual audit of data management guidelines. Managers are expected to include assessment of desk transitions in the Annual Performance Review (APR) of desk economists.  
  - Training: ICD and HRD mandated to strengthen learning environment. An interdepartmental working group led by ICD revamped the internal economics training program (IET) into a structured curriculum with courses in surveillance, finance, monetary and fiscal policy, macro-financial linkages, statistics and empirical methods at multiple levels. An online course on macroeconomic forecasting was added in May 2015 and will be opened to outside audiences in October 2015 as part of the Fund’s MOOCs. Five self-contained modules on advanced empirical methods complement the course. Over 130 Fund staff signed up for the first offering of the online forecasting course.

- Recommendation 3 — Extend guidance on forecasting methodologies (short- and medium-term):  
  - Top-down guidance for forecasts: RES will establish a process for top-down guidance as an input in medium-term forecasts. A multi-country macro-econometric forecasting model with stock-flow consistency (e.g., between current account balances and net foreign assets) has been developed. The process will be piloted with several country desks in the run-up to the October 2015 WEO, extended to more desks for the April 2016 WEO, and gradually applied to most major economies and country groups. These medium-term consistency checks will complement existing short-term checks. The revamped curriculum on empirical methods will support desk economists in incorporating advances in forecasting methodologies.

- Recommendation 4 — Publish WEO forecasting process description:  
  - A general description of the WEO forecasting process will be posted on the external IMF website with the release of the October 2015 WEO, drawing on inputs from the IEO report. The material will be periodically reviewed and revised.

- Recommendation 5 — Public availability of forecast and outturn data:  
  - WEO data: A database showing current and past forecasts and outcomes for real GDP growth, CPI inflation, and current account balances was posted on the IMF’s external website in April 2015. The database will be updated annually and could be later extended to other series depending on demand and resource costs. (See https://www.imf.org/external/pubs/ft/weo/2015/01/weodata/index.aspx.)  
  - Program data: The Monitoring of Fund Arrangements (MONA) database was extended to include all past forecasts linked to each program review and will be updated continuously. (See http://www.imf.org/external/np/pdr/mona/index.aspx.)

- Monitoring and follow-up: Progress will be reported in future PMRs. Commissioned external evaluations will regularly assess forecast quality and process. ICD will evaluate internal course relevance using participant feedback.

### Resource Implications
- Resources allocated so far (including in the FY2016 budget) for MIP implementation amount to about 7 FTEs and an additional 0.25 FTE would still need to be accommodated (Annex 2).  
- The bulk of costs relate to development and rollout of the medium-term macro-econometric forecasting model (model development was part of a RES capital project).  
- Recurrent annual costs from FY2017 onward for the MIP measures amount to about 4.9 FTEs, with an additional ¼ FTE required every 5–8 years to prepare the commissioned study on IMF forecasts.  
- Estimates include desk time to implement MIP recommendations and process guidance from the medium-term forecasting model. Training costs assumed to be accommodated within existing resources, possibly by refocusing training priorities. Resource implications will be reassessed after a critical mass of pilot cases for the medium-term forecast model.

### Annex I — Implementation Accountability (summary)
- Recommendation 1: RES responsible for commissioning studies, preparing staff responses, and implementing follow-up actions. Next study due by end-2015; recurring every 5–8 years.  
- Recommendation 2: RES responsible for monitoring forecast errors; SPR to monitor compliance with handover procedures and conduct a Fall 2015 survey; area departments to discuss significant forecast errors in policy consultation meetings; ICD to deliver and update forecasting course offerings. Handover guidelines to be integrated by end-2016. From January 2016, policy consultations to discuss significant forecast errors.  
- Recommendation 3: RES to pilot top-down guidance for medium-term forecasts beginning Fall 2015; pilots in run-up to October 2015 WEO and extended for April 2016 WEO.  
- Recommendation 4: A general description of the WEO forecasting process to be posted with October 2015 WEO and periodically reviewed.  
- Recommendation 5: WEO database posted April 2015; updated annually. MONA database extended and updated continuously.

*Source: pp091815c-implementation-plan - Section 1 (Implementation Plan in Response to the IEO Evaluation of IMF Forecasts), September 18, 2015.*

### Section 2

### pp091815c-implementation-plan - Section 2

### Overview of responses to IEO recommendations
- The Implementation Plan (MIP) addresses IEO evaluation recommendations related to the Fund’s forecasts, with Director responses and planned follow-up actions.
- A multi-country macro-econometric forecasting model was developed in FY 2015 to ensure medium-term consistency of forecasts.
- Consistency checks will be broadened gradually for most major economies and country groups to complement existing consistency checks for short-term forecasts; some Directors noted resource implications.

### Specific recommendations, Director responses, and follow-up actions
- Recommendation 4: Publish online a general description of the WEO forecasting process, which will be reviewed and revised as needed.
  - Directors supported this recommendation.
  - Action: A general description of the WEO forecasting process will be posted on the external IMF website at the time of the October 2015 WEO. The material will be reviewed and revised periodically.
  - Responsibility: RES.
- Recommendation 5: Make available to the public data related to forecasts and outturns that are readily available internally.
  - Directors supported facilitating public access to historical forecast and outturn data, while noting resource constraints and weighing expected benefits to users and costs; some Directors thought dissemination costs would be low.
  - Actions:
    - A WEO database showing current and past forecasts and outcomes for real GDP growth, CPI inflation, and current account balances has been created and posted on the IMF website in April 2015. The database will be updated annually.
    - The Monitoring of Fund Arrangements (MONA) database has been extended and now includes all past forecasts linked to each program review. The database will be updated on a regular basis.
  - Responsibilities: RES and SPR will be responsible for updating the WEO and MONA databases, respectively.

### Annex II — Resource implications (summary of IEO recommendations and resource costs)
- Table headers: IEO Recommendation | Resource Cost through FY 2016, (FTEs) | Future Recurrent Costs (FTEs per year)

- 1. Maintain the practice of commissioning external evaluations of IMF forecasts by recognized experts.
  - Resource Cost through FY 2016: 0.25
  - Future Recurrent Costs: 0.25 (every 5–8 years)

- 2. Enhance processes and incentives for learning from past forecast performance.
  - Activities listed:
    - Development of a data processing tool for forecasts.
    - Dissemination of forecast errors to desks and discussion in WEO publications.
    - Discussion of major forecast errors in policy consultation meetings by country desks starting in January 2016.
    - Staff Survey on handover process.
    - Development of an online course on macroeconomic forecasting.
    - Teaching online course.
  - Resource costs and future recurrent costs (as listed, in FTEs and broken down across items in the source):
    - 0.60
    - 0.10
    - 0.05
    - 0.05
    - 0.10
    - 0.25
    - 0.05
    - 0.35
    - 0.10
    - 0.05
    - 0.15
    - 0.00
    - 0.00
    - 0.05

- 3. Extend guidance to desk economists about how best to incorporate advances in forecasting methodologies for both short- and medium-term forecasts.
  - Activities and resource notes:
    - Refining, maintaining and applying the model to pilot country cases (does not include country desk time).
    - Processing and responding to the guidance provided by the medium-term forecasting model by country desks in pilot country cases.
  - Resource Cost through FY 2016 and Future Recurrent Costs (FTEs):
    - 6.25
    - 3.00
    - 0.25
    - 4.50
    - 3.00
    - 1.50

- Initial development of multi-country macro-econometric forecasting model.
  - Resource Cost through FY 2016: 3.00
  - Future Recurrent Costs: 0.00

- 4. Publish online a general description of the WEO forecasting process, which will be reviewed and revised as needed.
  - Resource Cost through FY 2016: 0.01
  - Future Recurrent Costs: 0.00

- 5. Make available to the public data related to forecasts and outturns that are readily available internally.
  - Activities:
    - Developing WEO historical database
    - Extending MONA database
    - Maintaining WEO and MONA databases
  - Resource Cost through FY 2016 (FTEs):
    - 0.20
    - 0.10
    - 0.05
    - 0.05
    - 0.05
  - Future Recurrent Costs (FTEs per year):
    - 0.00
    - 0.00
    - 0.05

*Source: https://www.imf.org/-/media/files/publications/pp/pp091815c-implementation-plan.pdf*

---


_Source: https://www.imf.org/-/media/files/publications/pp/pp091815c-implementation-plan.pdf_