February 20, 2004

The IEO provides objective and independent evaluation on issues related to the IMF. The Office operates independently of IMF management and at arm's length from the IMF's Executive Board.

IEO Work Program for Fiscal Year 2005

1. This note outlines the work program of the IEO for FY2005 as determined by the Director, on the basis of consultations with the Executive Board, management, staff, and external stakeholders. The work program is being submitted to the Executive Board for review, as required under the IEO terms of reference.

I. Consultation Process for Determining the FY2005 Work Program

2. In November 2001, the IEO had indicated a list of 15 topics from which the work program for the next three years would be chosen.¹ Six of these were chosen in the first two years (FY 2003 and FY 2004). The first stage in determining the FY2005 work program was to review the original list and prepare a short list of topics for FY2005. The short list consisted of the following, of which 4 would be chosen.²

• The IMF's experience with financial sector assessment programs (FSAP) and the associated financial sector stability assessments (FSSA);
  
  
• A case study of the role of the IMF in Turkey;
  
  
• The IMF's approach to capital account liberalization;
  
  
• Structural conditionality in IMF-supported programs;
  
  
• Private sector involvement in the prevention and resolution of crises;
  
  
• The role of multilateral surveillance; and
  
  
• A low-income country case study

II. Work Program for FY2005

4. The consultations undertaken by the IEO revealed that there was support for all seven projects and a careful balancing of pros and cons was needed to choose four of them. On further consideration, it was felt that two of the projects--an evaluation of structural conditionality and the Turkey case study--should be taken up next year.

5. In the case of structural conditionality, there was very wide support for an IEO evaluation of this topic. However, the Executive Board is already scheduled to discuss the biennial review of conditionality in the summer of 2004. To avoid parallel efforts, it was felt that it would be more useful for the IEO evaluation to take account of the outcome of this review, including comments and concerns expressed by Executive Directors. This implies that work on this evaluation should begin sometime in the Fall of 2004 and should be ready for Board consideration by the end of 2005.

6. In the case of Turkey, the case for postponement rests on the fact that there is an ongoing program which will conclude only in December 2004. The project was included in the short list on the assumption that the evaluation would be limited to the period up to 2002 leaving out the ongoing program. However, on further reflection, it was felt that for an evaluation report presented in 2005 to stop at 2002 would leave a number of important issues unexamined; consequently it would be better to undertake a fuller evaluation of Turkey later. A decision on this project will therefore be taken at the time of formulating the FY 2006 work program.

7. From the remaining five projects in the shortlist, it has been decided to choose the following:

(i) financial sector assessment program (FSAP) and financial sector stability assessments (FSSA);

(ii) the IMF's approach to capital account liberalization;

(iii) the role of multilateral surveillance;

(iv) country case study.

The issues to be addressed in each evaluation, including the proposed evaluation of structural conditionality on which work will commence in FY2005 but which is part of the work program for FY2006, are briefly spelt out below.

A. Financial Sector Assessment Program (FSAP) and Financial Sector

Stability Assessments (FSSA)

8. The FSAP/FSSA is a major initiative undertaken by the IMF, in collaboration with the World Bank, designed to strengthen bilateral surveillance of member countries and contribute to crisis prevention by helping to identify financial sector vulnerabilities and generate policy recommendations to overcome them. The experience under this initiative was last reviewed by the Executive Board in March 2003, based on an internal joint assessment by IMF and World Bank staff. Approximately 70 assessments are expected to be completed by the end of FY 2004. An independent evaluation of this important, and resource-intensive, activity is therefore timely.

9. The evaluation would focus on the adequacy of objectives under the Fund's core responsibilities; on design and implementation of the program so far, including the nature of assessment reports and linkages to technical assistance (TA) and surveillance activities; and on policy changes achieved vis-à-vis objectives of the initiative. Some of the issues to be addressed could include:

• How do FSAP objectives and Fund involvement in financial sector stability issues aligned with its core macro stability mandate? What has been the overarching framework guiding the design of the FSAP/FSAA and its integration into the Fund's surveillance function?
  
  
• On risk assessment; what has been the approach to risk and vulnerability assessment under the FSAP and has this approach been effective? How has the evaluation of financial risks integrated with the overall macroeconomic assessment of vulnerabilities? What is the role for re-assessment and continuous financial surveillance of vulnerabilities?
  
  
• With respect to vulnerabilities, how well have FSAPs identified the major risks to a country's financial system? Are recommendations articulated as a coherent "blueprint" for reform with proper regard to suitability, sequencing, and implementation capacity? Have actions/reforms been effectively carried out? Has the exercise led to a better understanding of TA needs in the context of strengthening the financial sector, and has this led to appropriate IMF response in the provision of TA?
  
  
• Regarding resource allocation, what have been the selection criteria for countries under the initiative and has the actual selection been appropriate, in light of the objectives of the initiative? What has been the coverage of topics across countries? What has been the division of labor and coordination with the World Bank and other IFIs?
  
  
• Regarding value added: Have FSAP/FSSAs deepened the quality and coverage of discussions with authorities on financial sector issues? What has been the role of the FSAP and FSSAs in recent crisis intervention cases? How has the private sector reacted to published FSSAs and how has it impacted its activities and risk perceptions?

10. The evaluation will also seek to identify ways in which the effectiveness of the IMF's role in the initiative could be improved with regard to each of these questions.

B. The IMF's Approach to Capital Account Liberalization

11. The issue of capital account liberalization has received a great deal of attention in the aftermath of recent capital account crises. Critics have argued that the IMF has encouraged emerging market countries to liberalize capital flows prematurely, and subjected them to the unexpected volatility of these flows. Some have argued that rapid liberalization, with insufficient attention to sequencing and establishing the preconditions for effective integration with global financial markets, has been responsible for a great deal of the financial instability and associated economic distress experienced by many emerging market countries. The IMF has argued that its approach to capital account liberalization has become much more pragmatic since the East Asian crisis and there is now a clear recognition of the need to build strong financial systems as a precondition for capital account liberalization.

12. Unlike liberalization of current account transactions, which is an explicit part of the mandate of the Fund, encouraging liberalization of the capital account is not explicitly a part of the Fund's mandate. The proposed evaluation would assess whether indeed the Fund's policy advice on the scope, pace and composition of capital account liberalization has changed after the experience of the late 1990s. The evaluation would be largely based on a selective study of surveillance and program documents in a representative subset of countries before and after the East Asian crisis, supplemented by an examination of internal documents related to these countries.

13. Specific issues which could be addressed by the evaluation include the following:

• What has been the content of the IMF's advice regarding liberalization of the capital account? Have policies towards the capital account changed in recent years in the light of experience with capital account crises? More specifically, how is this reflected in IMF advice to countries that have not undertaken significant liberalization of their capital account?
  
  
• Has the Fund's policy advice on capital account liberalization across countries reflected a consistent approach based on a clear recognition of the two-way linkage between policy towards capital flows and policy towards the exchange rate regime?
  
  
• What has this implied for policy advice on the sequencing of reforms, especially vis-à-vis the financial sector? Have trade-offs between capital account liberalization and the benefit from strengthening domestic financial intermediation been explicitly weighed against the dangers from heightened vulnerabilities when the financial sector is not sufficiently strong?⁴
  
  
• Is surveillance—including multilateral surveillance—giving sufficient attention to the risks emanating from the supply side, especially with regard to the volatility of capital flows? Has surveillance helped to identify corrective steps needed on the supply side and has it been effective in addressing these issues?
  
  
• Given that occasional crises are inevitable, has the IMF taken a consistent and defensible position on the role of temporary capital account restrictions as part of crisis response and prevention? Has the IMF objectively assessed the usefulness and effectiveness of these restrictions in situations where they have been imposed?

C. The Role of Multilateral Surveillance

14. Multilateral surveillance is a core activity of the IMF aimed at analyzing the forces driving the world economy, identifying global vulnerabilities, and advising on appropriate policies, especially in the most systemically important countries. Potential issues to be addressed would include the following:

• What has been the effectiveness of multilateral surveillance in giving "early warning" of potential global vulnerabilities, such as cross-country asset booms and sharp fluctuations in capital flows to emerging markets? Does the staff produce sufficiently pointed analysis and recommendations on these issues; how effectively does the IMF follow up on them; and what has been the actual impact of IMF policy advice on member country policies?
  
  
• Has there been an effective interaction between multilateral surveillance and bilateral surveillance, particularly in the major industrial countries? How could this cross-fertilization be improved?⁵ How has the interaction of multilateral and bilateral surveillance dealt with the increasing "globalization" of some vulnerabilities, and does this suggest that changes in the way such surveillance is conducted are needed?
  
  
• What has been the role of the World Economic Outlook and International Capital Markets Report exercises in forecasting prospects for the global economy and identifying global vulnerabilities? How well are these exercises integrated into the IMF's other operational work, including the design of IMF-supported programs?
  
  
• How effective a role has the International Capital Markets Department (ICM) played in enhancing the coverage and integration of financial market issues into multilateral surveillance and in identifying systemic weaknesses originating in the financial sector.
  
  
• What has been the impact of the increased emphasis on capital market issues in multilateral surveillance and the enhanced interactions with private financial market participants? Has it led to a clearer focus on areas where action is needed on the supply side to mitigate volatility?
  
  
• What has been the effectiveness of the IMF's inputs into the deliberations of various international groups, such as the G-7, APEC, the Manila Framework Group, and various regional groupings?
  
  
• How has the product of multilateral surveillance compared with similar assessments produced by other public and private institutions? How has it been received by the international community as well as by the private sector?

D. Country Case Study

15. The proposed country case study is meant to provide an example of evaluation in situations other than capital account crises. The evaluation will serve two main purposes: (i) provide an illustration of the scope for learning from a systematic ex post assessment of IMF operations in a country; and (ii) investigate in some detail what the case suggests about the IMF's recent approach to program design, including the structure and focus of conditionality. How surveillance and technical assistance activities relate to program activities in the particular situation of the country will also be examined.

16. It is proposed to select either a low-income country that is not among the existing case studies for the PRSP/PRGF, or a middle-income country that is not a "capital account crisis" case.⁶ In order to allow for some assessment of preliminary outcomes, the choice will be limited to countries with an IMF-supported program that ended no later than end-2003. The final selection will be made after further consultations.

E. Structural Conditionality in IMF-Supported Programs

17. The IMF was criticized in many quarters for the widening of both the scope and the volume of structural conditionality over the last decade. The Managing Director responded to this criticism by issuing, in the Fall of 2000, a set of interim guidelines for staff on streamlining conditionality and, shortly thereafter, launching a thorough review of conditionality. This led to the adoption by the Executive Board of revised conditionality guidelines in September 2002. These guidelines call for streamlining conditionality to achieve parsimony while ensuring that conditionality covers all policies critical to the achievement of macro-economic objectives. They recognize that critical policies would include policies for which other institutions, notably the World Bank, have primary responsibility. They also emphasize the importance of ownership for conditionality to be effective.

18. The evaluation could examine the internal consistency of the new guidelines, including (a) the possible tension between emphasizing parsimony and also covering all policies critical to the achievement of macroeconomic objectives including sustainability, and (b) the tension between an approach that emphasizes country-owned adjustment programs but also retains the traditional dependence on conditionality. It could assess the extent to which the new guidelines have brought about a significant change in actual practice in focusing in some core areas of structural conditionality such as fiscal, trade, financial and privatization. This would require comparison of the situation before September 2002 (as already documented in a staff review) and the current position reflected in programs approved after the September 2002 guidelines were issued.

19. Specific issues to be investigated could include:

• What is the underlying rationale for conditionality, and what is the evidence about the appropriateness (design) and effectiveness (implementation) of structural conditionality in light of the expected objectives? How have programs dealt with the issue of ownership and what are the implications for the design of structural conditionality? Has a more focused conditionality enhanced the ownership of programs? In addressing these issues, the evaluation will draw upon any lessons emerging from the ongoing evaluation of the PRSP/PRGF Initiative.
  
  
• How has the potential conflict between the objective of achieving parsimony and the objective of ensuring that conditionality covers all policies critical to achieving macroeconomic objectives of the program been resolved in practice? Have programs identified core essential elements and non-core desirable elements while limiting conditionality explicitly to the former?
  
  
• How have staff approached the task of assessing macroeconomic relevance in deciding whether or not to include particular structural measures in IMF conditionality? Are there situations where streamlining has been interpreted too narrowly --leaving out measures impacting on critical aspects of the "quality" of macroeconomic adjustment, in particular sustainability and minimum considerations of equity?
  
  
• Has the concept of division of labor between IFIs led to a redesign of programs to give greater emphasis to the areas of conditionality that are within the competence of the IMF, with less emphasis being given to the others. Where other areas of conditionality remain important, how is the objective of parsimony achieved in collaboration with other institutions?
  
  
• How has enhanced collaboration with the World Bank in the design of conditionality been implemented in practice?

As in the past, a more detailed issues paper will be prepared on each subject as each evaluation commences and comments will be invited on the issues paper before finalizing the terms of reference. Comments on this work program, including suggestions relating to topics for future evaluations are invited and can be submitted by e-mail at: ieo@imf.org