October 23, 2003
FOR FISCAL YEAR 2005
1. The work program of the IEO is determined by the Director
on the basis of consultations with a broad range of stakeholders. To facilitate
these consultations, this paper presents a short list of topics from which
the program for FY 2005 (May 2004 to April 2005) will be chosen. It is
expected that this work program will consist of four evaluations. 2. The short list has been chosen from the medium-term
work program published by the IEO in November 2001, which suggested 15
possible evaluation topics (see Appendix 1). Six of these have already
been taken up in the first two full years of operation. The remaining
9 topics (nos. 7 to 15 in the Appendix) remain relevant today. A tenth
topic which may be added is a case study of Turkey, which was originally
envisaged as an alternative to Argentina.
1. The work program of the IEO is determined by the Director on the basis of consultations with a broad range of stakeholders. To facilitate these consultations, this paper presents a short list of topics from which the program for FY 2005 (May 2004 to April 2005) will be chosen. It is expected that this work program will consist of four evaluations.
2. The short list has been chosen from the medium-term work program published by the IEO in November 2001, which suggested 15 possible evaluation topics (see Appendix 1). Six of these have already been taken up in the first two full years of operation. The remaining 9 topics (nos. 7 to 15 in the Appendix) remain relevant today. A tenth topic which may be added is a case study of Turkey, which was originally envisaged as an alternative to Argentina.
3. The selection criteria used earlier gave priority to topics that (i) have been the subject of controversy or criticisms; (ii) have the greatest interest for a wide range of the membership; (iii) have the greatest learning potential; and (iv) for which the IEO has a clear comparative advantage. Applying these criteria, we have identified the following short list of seven topics.
A. The IMF's experience with financial sector assessment programs (FSAP) and the associated financial sector stability assessments (FSSA);
4. The IEO
welcomes suggestions on which of the topics listed above should be included
in the final list of four evaluations to be undertaken in FY 2005. Topics
not included in the short list can also be considered if there are good
reasons for preferring them on the basis of the criteria outlined above.
Comments can be addressed to firstname.lastname@example.org.
A. Financial Sector Assessment Program (FSAP) and Financial Sector Stability Assessments (FSSA)
6. The FSAP/FSSA is a major initiative undertaken by the IMF, in collaboration with the World Bank, in response to the crises of the late 1990s. It is designed to strengthen bilateral surveillance of member countries and contribute to crisis prevention by helping to identify financial sector vulnerabilities and generate policy recommendations to overcome them. The experience under this initiative was last reviewed by the Executive Board in March 2003, based on an internal joint assessment by IMF and World Bank staff. Approximately 70 assessments are expected to be completed by the end of FY 2004. An independent evaluation of this important, and resource-intensive, activity is therefore timely.
7. The proposed evaluation could focus on areas of broad program design and integration with surveillance functions, as well as implementation aspects, including:
8. The evaluation would likely have to be conducted at two levels. Some issues would be examined on the basis of a cross country assessment of experience in all the FSAP cases. This cross-country assessment would have to be supplemented by in-depth study of a sample of country cases.
9. Since the FSAPs are undertaken jointly with the World Bank, it would be desirable to undertake the study in parallel with an evaluation by the World Bank's Operations Evaluation Department of the Bank's experience in this regard. Preliminary discussions with OED have been held to this end.
B. A Case Study of Turkey
10. Turkey provides another example of a capital account crisis involving large access, similar in some ways to the cases in Korea, Indonesia, and Brazil already studied by IEO and the study of Argentina which is currently under way. The country experienced severe financial instability in 1999 leading to the request for a stand-by arrangement for SDR 9.3 billion (US$12.7 billion) at the end of 1999, to protect the fixed exchange rate which was viewed as an essential nominal anchor. The amount under the arrangement was subsequently augmented, but the effort to maintain the exchange rate peg proved unsustainable and was abandoned in February 2001 when the lira was floated. The 1999 arrangement, with SDR 3.3 billion undrawn, was cancelled in February 2002, and a new SBA for SDR 12.8 billion was put in place. This program is expected to end in December 2004.
11. An evaluation of Turkey's experience, including the process by which key decisions on IMF-supported programs were taken, should provide additional lessons on the effectiveness of Fund surveillance in crisis prevention and the Fund's approach to crisis resolution. It would be most useful if the study could cover a longer period after the initial crisis than was possible in the earlier IEO studies of crisis cases, as this would enable the IEO to focus on the post-crisis period in greater depth. The exact period to be covered under the evaluation will be chosen so as to avoid interfering with ongoing operations. The experience with the original 1999 arrangement which was cancelled in February 2002 can be examined without interfering with ongoing operations. The evaluation could be extended to cover issues relating to the second 2002, arrangement after December 2004, when the present arrangement is set to expire.
12. The evaluation would seek to address several questions related to the adequacy of IMF diagnosis and policy advice, including:
(i) Surveillance in the pre-crisis period will be critically examined to assess whether it identified the problems which surfaced later, especially regarding inflation stabilization, the use of the exchange rate as a nominal anchor, and consideration of vulnerabilities, risks, and exit strategies.
the programs well designed from the point of view of assessing fiscal
sustainability, both before the crisis and during the period of crisis
(iv) Were programs designed on the basis of a realistic assessment of the extent to which the proposed reforms were owned by the authorities, as well as the authorities' capacity to implement key measures?
(v) Did crisis management, including the defense of the exchange rate regime, take adequate account of banking sector problems?
(vi) How well did IMF policy advice in the post-crisis period cope with the challenges of stabilization and continuing vulnerabilities, including debt sustainability and banking soundness? Parallels with Argentina in this respect would be interesting.
C. The IMF's Approach to Capital Account Liberalization
13. The issue of liberalization of the capital account has been the focus of a great deal of attention and generated considerable criticism of the IMF. Liberalization of capital account transactions is not an explicit part of the mandate of the Fund but policies relating to liberalization of the capital account have figured in Fund surveillance. Critics have argued that the IMF has encouraged emerging market countries to liberalize capital flows in a manner that has subjected them to severe stress because of the unexpected volatility of these flows. Some have argued that rapid liberalization, with insufficient attention to sequencing and establishing the preconditions for effective integration with global financial markets, has been responsible for a great deal of the financial instability and associated economic distress experienced by many emerging market countries.
14. The IMF has argued that its approach to capital account liberalization has become much more pragmatic after the East Asian crisis and there is now a clear recognition of the need to build strong financial systems as a precondition for capital account liberalization. The proposed evaluation would assess whether indeed the Fund's policy advice on the scope, pace and composition of capital account liberalization has changed after the late 1990s. The evaluation would be largely based on a selective study of surveillance and program documents in a representative subset of countries before 1997 and after 1999, supplemented by an examination of other internal documents, if any.
15. Specific issues which could be addressed by the evaluation include the following:
D. Structural Conditionality in IMF-Supported Programs
16. The IMF was criticized in many quarters for the widening of both the scope and the volume of structural conditionality over the last decade. The Managing Director responded to this criticism by issuing, in the Fall of 2000, a set of interim guidelines for staff on streamlining conditionality and, shortly thereafter, launching a thorough review of its conditionality. This led to the adoption by the Executive Board of revised conditionality guidelines in September 2002. These guidelines call for streamlining conditionality to achieve parsimony while ensuring that conditionality covers all policies critical to the achievement of macro-economic objectives. They recognize that critical policies would include policies for which other institutions, notably the World Bank, have primary responsibility. They also emphasize the importance of ownership for conditionality to be effective.
17. The evaluation could examine the internal consistency of the new guidelines, including (a) the possible tension between emphasizing parsimony and also covering all policies critical to the achievement of macroeconomic objectives, and (b) the tension between an approach that emphasizes country-owned adjustment programs but also retains the traditional dependence on conditionality. It could assess the extent to which the new guidelines have brought about a significant change in actual practice in focusing on the four core areas of structural conditionality—fiscal, trade, financial and privatization. This would require comparison of the situation before September 2002 (as already documented in a staff review) and the current position reflected in programs approved after the September 2002 guidelines were issued. Specific issues to be investigated could include:
E. Private Sector Involvement (PSI) in Crisis Resolution
18. The need to involve the private sector effectively in the resolution of financial crises in order to contain the size of access and also to create market-based systems of resolution with the right mix of incentives has been much discussed. In principle, an evaluation of recent country programs could be undertaken to address issues such as:
While these are important issues, an argument against undertaking an evaluation at this stage is that policy in this area is currently being discussed and there are significant differences within the official community on how the issues should be handled. The management initiative on introducing a Sovereign Debt Restructuring Mechanism was potentially an important element of private sector involvement but there is no agreement on this proposal as yet. There is general agreement on the usefulness of Collective Action Clauses (CAL), but too little experience.
F. The Role of Multilateral Surveillance
19. Multilateral surveillance is a core activity of the IMF that aims to provide a basis for analyzing the forces driving the world economy, identifying global vulnerabilities, and advising on appropriate policies, especially in the most systemically important countries. Potential issues to be addressed would include the following:
G. An Additional Country Case Study
20. The IEO could undertake an in-depth examination of a recent country program with the focus on the effectiveness of program design and conditionality. The primary goal would not be to assess whether the IMF's actions were right or wrong in a particular case (although that would be an additional outcome), but to investigate in some detail what the case suggests about the IMF's recent approach to program design as well as the structure and focus of its conditionality. Programs chosen could be from either 2000 or 2001, allowing evaluation of outcomes in the post-program period. Two possible options could be considered:
(i) An investigation of program design in a middle-income country, but not one of the major "capital account crisis" cases, which are being examined in other evaluations. Possibilities would include Bulgaria, Peru, Romania, Sri Lanka, or the Ukraine.
(ii) A low-income country that has not been able to advance to the PRSP stage owing to weak policy formulation processes, fragile institutions, and a variety of governance and other political problems, including post-conflict situations4. An in-depth examination of the IMF's role in one such case could help generate lessons on what the IMF's role and approach should be in such cases.
* * * * *
Comments are invited from all interested parties on the priority that should be attached to the seven topics discussed in this paper in choosing 4 topics for the IEO work program for FY 2005. Suggestions for including topics not discussed in the paper are also invited.
The deadline for submitting comments is Monday, December 22, 2003. Comments can be submitted through the IEO website www.imf.org/ieo. The final work program will be determined after taking account of comments received and will be posted on the IEO website sometime in January 2004.
1The list of topics is taken from the original list of 15 topics in "Proposed Work Program of the Independent Evaluation Office (IEO)" November 21, 2001 which can be accessed at www.imf.org/ieo. The order in which the topics appear has been changed to reflect the selections already made in the FY 2003 and FY 2004 work programs.