## Implementation Plan in Response to Board-Endorsed Recommendations for the IEO Evaluation of Research at the IMF—Relevance and Utilization ## Source details **Canonical URL:** [Implementation Plan in Response to Board-Endorsed Recommendations for the IEO Evaluation of Research at the IMF—Relevance and Utilization](https://www.imf.org/-/media/websites/imf/imported-full-text-pdf/external/np/pp/eng/2012/_102312.pdf) ## Other formats - [Markdown version](/-/media/websites/imf/imported-full-text-pdf/external/np/pp/eng/2012/_102312.pdf.md) - [Structured JSON version](/-/media/websites/imf/imported-full-text-pdf/external/np/pp/eng/2012/_102312.pdf.json) --- ### Introduction - Purpose: forward-looking implementation plan for the IEO report on IMF research (IEO Evaluation of Research at the IMF—Relevance and Utilization). - Board view: many IMF analytical papers are of high quality and widely read, but research is uneven in quality and perceived as message driven; scope to enhance relevance, technical quality, openness to alternative views, and coordination. ### IEO recommendations and Board reactions - Four recommendation clusters: - Improve the management of IMF research. - Enhance the relevance of research. - Promote openness to alternative perspectives. - Enhance the technical quality of analytical work. - Directors: support improving management but urged examination of past coordination failures (see Box 1 findings on prior committees). - Monitoring: Board monitoring through Periodic Monitoring Reports and broader discussions of strategic human resource initiatives for cultural changes. ### A. Improving the management of IMF research - IEO recommendation: designate a senior Research Coordinator reporting annually to the membership and Board. - Actions taken: - Economic Counselor convened interdepartmental meeting in October 2011 to coordinate departmental work agendas; meetings to be held twice a year going forward. - These meetings replace the Committee on Research Priorities (CRP) deemed overly bureaucratic. - Economic Counselor to update interested Executive Directors about twice a year. - Historical coordination (Box 1 findings): - Committee on Research Priorities (CRP) established November 2, 1999; inactive after 2001. - Research Committee established 2004; faded by 2007. - Working Group on Fund Research (WGFR) has provided a lower-profile clearinghouse since 1989. - Conclusion: formal committees proved costly relative to benefits; many functions achieved through normal management interactions and lower-profile mechanisms; some CRP-initiated items (IMF Annual Research Conference, IMF Research Bulletin) endured. ### B. Enhancing the relevance of IMF research - IEO recommendation: conduct periodic strategic reviews and prepare an indicative medium-term research agenda in consultation with members. - Planned strategic review: - Will be based on the IEO framework; scheduled in about 18 months. - Will assess allocation of resources across research product lines; balance quantity and quality; consider quality control, internal review processes, and incentives; account for budget constraints. - Consultation: Executive Directors, country authorities, and external stakeholders to the extent time/resources permit. - Interim adjustments and consultations: - Departments have reduced frequency/discontinued some Regional Economic Outlooks to reallocate resources. - Departments will step up consultations with country authorities and endeavor to discuss preliminary results early; staff retain flexibility to research issues they deem important. - Progress to be monitored through surveillance update in autumn of 2012 and in the next strategic review. - Mission-team stability and mobility: - Assignments expected to last at least three years going forward. - For staff with seven or more years in a department, a centralized mobility support program will complement market mechanisms. - HRD and TGS put in place a system to monitor mission chief and team tenure; results to be shared with the Board as part of briefings on strategic human resources. ### C. Promoting openness to alternative perspectives - IEO recommendation: allow researchers to explore issues without preconceived conclusions; foster environment encouraging innovative research and establish incentives. - Cultural and human-resources initiatives: - Reforms cover leadership, management training, mobility, and accountability (reference to recent MIP and Managing Director’s statement on the IEO Crisis Evaluation). - A statement of workplace values agreed by September 2012 for employees will emphasize people management and diversity and encourage staff to contribute ideas that may challenge groupthink. - Research product innovations: - Staff Discussion Note series published to elicit comment and debate; examples cited: - Managing Capital Inflows: What Tools to Use? (April 2011). - What Happens to Social Spending in IMF-Supported Programs? (August 2011). - Rethinking Macroeconomic Policy (February 2010). - Board and Directors views (from Section 2): - Researchers should be allowed to explore issues without preconceived conclusions or messages. - After a thorough quality review, staff should be able to publish WPs and other academic style products even when results are not well aligned with messages in surveillance documents. - Directors found worrisome the perception that IMF research is message-driven and emphasized addressing internal culture and institutional values to protect credibility. ### D. Enhancing the technical quality of analytical work - IEO recommendation: allocate adequate time/resources to projects; establish clear standards for technical quality; strengthen incentives for quality. - Quality-control measures: - WP-producing departments continue responsibility for outputs and will nominate a senior staff member responsible for ensuring processes verify each Working Paper (WP) is of sufficient quality and ensure accountability of senior staff approving papers. - Department-level processes with shared best-practices; departments encouraged to solicit early feedback from other relevant departments when research overlaps. - Strategic review scope: - Will assess these procedures and technical quality across other research products (Selected Issues Papers, Regional Economic Outlooks). - Will consider an enhanced role for external review. - Staff annual performance evaluations to place increased weight on research quality for research-producing staff. - Section 2 reiteration: - “...the IMF needs clear standards for technical quality of different research projects...incentives to improve the quality of research should be strengthened.” - New procedures have been adopted to ensure the quality of Working Papers. - The forthcoming strategic review of IMF research will assess technical quality more broadly. ### Resource implications - Overall resource requirement: initiatives should not need significant additional resources; most can be accommodated within the existing budget envelope. - Quality control: additional staff time required is not expected to be very large but may necessitate being more judicious/selective on number of working papers produced. - Context and constraints: - Conditions differ from 1999–2008 period; downsizing and crisis workload have significantly reduced resources for non-flagship research products and this trend is expected to continue at least in the next fiscal year. - Strategic review will assess whether resource changes are needed to further enhance research output and improve quality for a given quantity. ### Monitoring and reporting - Regular monitoring commitments: - Interdepartmental coordination meetings twice a year. - Economic Counselor updates to Executive Directors about twice a year. - Surveillance update in autumn of 2012 to monitor consultation commitments. - Strategic review in about 18 months to assess broader issues and inform medium-term agenda. - HRD/TGS tenure monitoring results to be shared with the Board as part of strategic HR briefings. *Prepared by the Strategy, Policy, and Review Department (in consultation with the Research and Other Departments); Approved by Ranjit Teja; October 23, 2012. Source: _102312.* ### Section 1 ### Implementation Plan in Response to Board-Endorsed Recommendations for the IEO Evaluation of Research at the IMF—Relevance and Utilization ### Introduction - Purpose: forward-looking implementation plan for the IEO report on IMF research (IEO Evaluation of Research at the IMF—Relevance and Utilization). - Board view: report found many IMF analytical papers are of high quality and widely read, but research is uneven in quality and perceived as message driven; scope to enhance relevance, technical quality, openness to alternative views, and coordination. ### IEO recommendations and Board reactions - Four recommendation clusters: - Improve the management of IMF research. - Enhance the relevance of research. - Promote openness to alternative perspectives. - Enhance the technical quality of analytical work. - Directors: support improving management but urged examination of past coordination failures (see Box 1 findings on prior committees). - Monitoring: Board monitoring through Periodic Monitoring Reports and broader discussions of strategic human resource initiatives for cultural changes. ### Implementation Plan — A. Improving the management of IMF research - IEO recommendation: designate a senior Research Coordinator reporting annually to the membership and Board. - Actions taken: - Economic Counselor convened interdepartmental meeting in October 2011 to coordinate departmental work agendas; meetings to be held twice a year going forward. - These meetings replace the Committee on Research Priorities (CRP) deemed overly bureaucratic. - Economic Counselor to update interested Executive Directors about twice a year. - Box 1 findings (historical coordination efforts): - Committee on Research Priorities (CRP) established November 2, 1999; inactive after 2001. - Research Committee established 2004; faded by 2007. - Working Group on Fund Research (WGFR) has provided a lower-profile clearinghouse since 1989. - Conclusion: formal committees proved costly relative to benefits; many functions achieved through normal management interactions and lower-profile mechanisms; some CRP-initiated items (IMF Annual Research Conference, IMF Research Bulletin) endured. ### Implementation Plan — B. Enhancing the relevance of IMF research - IEO recommendation: conduct periodic strategic reviews and prepare an indicative medium-term research agenda in consultation with members. - Planned review: - Will be based on the IEO framework; scheduled in about 18 months. - Will assess allocation of resources across research product lines; balance quantity and quality; consider quality control, internal review processes, and incentives; account for budget constraints. - Consultation: Executive Directors, country authorities, and external stakeholders to the extent time/resources permit. - Interim adjustments: departments have reduced frequency/discontinued some Regional Economic Outlooks to reallocate resources. - Staff-country consultation (Selected Issues Papers and bilateral/regional surveillance): - Departments will step up consultations with country authorities and endeavor to discuss preliminary results early; staff retain flexibility to research issues they deem important. - Progress to be monitored through surveillance update in autumn of 2012 and in the next strategic review. - Mission-team stability and mobility: - Assignments expected to last at least three years going forward. - For staff with seven or more years in a department, a centralized mobility support program will complement market mechanisms. - HRD and TGS put in place a system to monitor mission chief and team tenure; results to be shared with the Board as part of briefings on strategic human resources. ### Implementation Plan — C. Promoting openness to alternative perspectives - IEO recommendation: allow researchers to explore issues without preconceived conclusions; foster environment encouraging innovative research and establish incentives. - Cultural and human-resources initiatives: - Reforms cover leadership, management training, mobility, and accountability (reference to recent MIP and Managing Director’s statement on the IEO Crisis Evaluation). - A statement of workplace values agreed by September 2012 for employees will emphasize people management and diversity and encourage staff to contribute ideas that may challenge groupthink. - Research product innovations: - Staff Discussion Note series published to elicit comment and debate; examples cited: - Managing Capital Inflows: What Tools to Use? (April 2011). - What Happens to Social Spending in IMF-Supported Programs? (August 2011). - Rethinking Macroeconomic Policy (February 2010). ### Implementation Plan — D. Enhancing the technical quality of analytical work - IEO recommendation: allocate adequate time/resources to projects; establish clear standards for technical quality; strengthen incentives for quality. - Quality-control measures: - WP-producing departments continue responsibility for outputs and will nominate a senior staff member responsible for ensuring processes verify each Working Paper (WP) is of sufficient quality and ensure accountability of senior staff approving papers. - Department-level processes with shared best-practices; departments encouraged to solicit early feedback from other relevant departments when research overlaps. - Strategic review scope: - Will assess these procedures and technical quality across other research products (Selected Issues Papers, Regional Economic Outlooks). - Will consider an enhanced role for external review. - Staff annual performance evaluations to place increased weight on research quality for research-producing staff. ### Resource implications - Overall resource requirement: initiatives should not need significant additional resources; most can be accommodated within the existing budget envelope. - Quality control: additional staff time required is not expected to be very large but may necessitate being more judicious/selective on number of working papers produced. - Context and constraints: - Conditions differ from 1999–2008 period; downsizing and crisis workload have significantly reduced resources for non-flagship research products and this trend is expected to continue at least in the next fiscal year. - Strategic review will assess whether resource changes are needed to further enhance research output and improve quality for a given quantity. ### Monitoring and reporting - Regular monitoring: - Interdepartmental coordination meetings twice a year. - Economic Counselor updates to Executive Directors about twice a year. - Surveillance update in autumn of 2012 to monitor consultation commitments. - Strategic review in about 18 months to assess broader issues and inform medium-term agenda. - HRD/TGS tenure monitoring results to be shared with the Board as part of strategic HR briefings. *Prepared by the Strategy, Policy, and Review Department (in consultation with the Research and Other Departments); Approved by Ranjit Teja; October 23, 2012.* ### Section 2 ### _102312 - Section 2 ### Indicative medium-term research agenda and periodic strategic review - An indicative medium term research agenda should be prepared in consultation with member countries and the Executive Board. - “Directors generally saw merit in conducting a periodic strategic review of research products. Management and staff were encouraged to focus on how best to allocate resources among the various research product lines...and to strengthen quality controls, the internal review process, and incentives to enhance the technical content of research, while taking into account budget constraints.” - “Most Directors also supported the IEO recommendation to set an indicative medium-term research agenda”. - The next review will build on the IEO’s framework and is expected in about 18 months. The review will focus on many of those issues highlighted by the IEO, and will be used to inform the setting of an indicative medium term research agenda. ### Consultation with country authorities and mission staffing - 2. Staff should consult country authorities on topics for SIPs and other research. [they] should also be able to research other relevant topics. Preliminary results should be discussed with authorities and other in country-experts. - “Directors broadly supported the IEO recommendation to consult more with country authorities on research topics prepared for bilateral and regional surveillance, particularly for Selected Issues Papers....Directors agreed that staff should remain free to research the issues that they feel are most important... Longer country assignments for mission members could facilitate collaboration with authorities and enhance familiarity with country specific conditions.” - Departments will step up efforts to consult more with country authorities on research topics, recognizing that staff will need flexibility to research issues that they feel are important. - Longer country assignments would also contribute by enabling greater familiarity with country conditions, as would collaboration with country authorities on research projects. - Reforms have recently been announced to ensure that an appropriate balance is struck between the need for cross country and fresh perspectives and mission team stability. ### Promoting openness to alternative perspectives - 1. Researchers should be allowed to explore issues without preconceived conclusions or messages. The Board, Management and senior staff should actively foster an environment that encourages innovative research and should establish incentives for staff to pursue such research. - After a thorough quality review, staff should be able to publish WPs and other academic style products even when the results of their analysis are not well aligned with the messages in surveillance documents. - “Directors considered worrisome the finding that there is a widely held perception that IMF research is message-driven, or that policy conclusions do not always follow from the analysis. While recognizing that research produced by the IMF will inevitably carry policy messages....they considered it critical for the credibility of the institution that the conclusions of in-house research are not biased by the IMF’s position... In this context, many Directors underlined the importance of addressing concerns about the internal culture and institutional values—identified in previous IEO evaluations...” - The forthcoming statement of workplace values should emphasize, among other things, that staff should contribute their ideas to the work program of departments, even if these are not in line with the Fund’s “groupthink.” ### Enhancing the technical quality of analytical work - Management and staff need to allocate adequate time and resources to each research project. - “...the IMF needs clear standards for technical quality of different research projects...incentives to improve the quality of research should be strengthened.” - New procedures have been adopted to ensure the quality of Working Papers. - The forthcoming strategic review of IMF research will assess technical quality more broadly. *Source: _102312 - Section 2* --- _Source: https://www.imf.org/-/media/websites/imf/imported-full-text-pdf/external/np/pp/eng/2012/_102312.pdf_