The Executive Board of the International Monetary Fund has decided to request independent external experts to conduct an evaluation of certain aspects of ESAF with the purpose of assessing its operation to date and making recommendations for improving the ESAF instrument and the design and implementation of ESAF-supported programs in a manner consistent with the overall purposes of the IMF.

2.  Structure and Focus of the Evaluation

There will be one external evaluation with three topics as well as unified conclusions for the design and implementation of ESAF-supported programs and the ESAF instrument:

The three focus topics are:

	(i)	Developments in countries' external position during ESAF-supported programs.
	(ii)	Social policies and the composition of government spending during ESAF-supported programs.
	(iii)	The determinants and influence of differing degrees of national ownership of ESAF-supported programs.

 
These topics will be evaluated on the basis of case studies by independent external experts, as indicated in Section 3, below, who will be responsible for a section of the final report, after consultation with the other evaluators. The purpose of each evaluation is to objectively assess developments during ESAF-supported programs and draw operational conclusions for future programs. At their full discretion, the evaluators may wish to take into account the views of concerned country authorities and social partners; of parliamentarians; of representatives of multilateral development banks, bilateral donors, and non-governmental organizations; of academic experts; and of Fund Executive Directors and staff. Unified conclusions will be prepared jointly by the evaluators, drawing on their three studies, as well as on the results of the internal evaluation of experience with ESAF-supported programs to be completed in the spring of 1997 by the Policy Development and Review Department.

The three evaluations should include, but not necessarily be limited to, consideration of the following issues:

(i)  Developments in countries' external position during ESAF-supported programs

• How, and based on what indicators, should vulnerability and viability in a country's external position be assessed?

• To what extent has external vulnerability increased or diminished in the countries under study over the specified period? What are the main factors that explain the different outcomes across the sample?

• Has the evolution of the countries' external positions diverged from what was expected in successive ESAF-supported programs? If so, what explains the divergences?

• Has the evolution of countries' external position been consistent with progress toward sustainable economic growth?

• To what extent were other developments, and policies pursued outside the context of Fund-supported programs, responsible for holding back, or contributing to, the achievement of external viability?

• What lessons can be drawn from the design of Fund-supported programs, and more generally for countries' external sector policies?

• How have social indicators evolved in countries undertaking ESAF-supported programs, and can one identify what impact adjustment policies have had on these social indicators? If possible, comparisons should be made with the trends of social indicators in the years before the ESAF-supported programs and a judgment made about how social indicators would have evolved in a non-adjustment scenario. What social indicators did Fund staff use in assessing priorities and progress?

• To what extent have social conditions been enhanced by strengthened external viability (where applicable) and improved conditions for non-inflationary growth?

• To what extent and how has the social impact of ESAF-supported adjustment policies and reforms influenced the use and design of social safety nets, the level of public spending on social services like education, health and rural development, and the design of the adjustment and reform measures themselves? How effective have the safety nets been in mitigating the effects of macroeconomic adjustment on the poorest members of society?

• How appropriate was the Fund advice on social safety nets, social indicators and social policies? To what extent has staff advice and government policy conformed to the guidance provided by the Executive Board?

• To what extent have government social expenditures been cut in ESAF-supported programs to achieve fiscal adjustment objectives, and how did outcomes in this regard differ from program plans? How were decisions on the level of social spending influenced by judgments about the overall balance between expenditure and revenue measures in programs?

• Is there any evidence on changes in social conditions or other economic outcomes in countries that have persevered with adjustment under ESAF-supported programs?

• What have been the main constraints in addressing social issues?

• To what extent have Fund staff collaborated with other institutions that have expertise on social issues? Has such collaboration been fruitful and how can it be enhanced?

• Given the purpose of Fund-supported programs to promote external viability and thereby facilitate economic growth, should such programs take into account a broader range of social issues than the current policy would dictate? Have programs been moving in this direction?

• On what criteria can national ownership of an ESAF-supported program be assessed?

• What evidence is there in the cases being studied that programs were "owned" by the country concerned? Was ownership confined to the national government or did it extend to other social partners?

• What efforts have been made by Fund staff and/or the national authorities to promote ownership of programs, and how successful have these efforts been? In what circumstances is the active involvement of Fund staff in this area most constructive, and are there instances where it is unhelpful?

• To what extent have other aspects of the Fund's operational practices inhibited the development of a sense of ownership by the countries' concerned?

• What have been the consequences for the implementation and sustainability of policies of strong or weak ownership?

• From the cases reviewed, what lessons can be drawn for the importance of ownership, and the approach the Fund should take to promote it further?

The following independent experts have agreed to conduct the evaluation:

Professor Koichi Hamada, with focus on the topic: "Developments in countries' external position during ESAF-supported programs."

Professor Paul Collier and Professor Jan Willem Gunning, with focus on the topic: "Social policies and the composition of government spending during ESAF-supported programs."

Dr. Kwesi Botchwey, with focus on the topic: "The determinants and influence of differing degrees of national ownership of ESAF-supported programs."

Dr. Botchwey has agreed to act as "convenor" for discussions among the evaluators concerning an appropriate degree of coordination in the selection of countries for the case studies, the contacts with country authorities, the design of work programs, and the drafting of the unified conclusions.

The evaluators shall conduct their work freely and objectively and shall render impartial judgment to the best of their professional abilities.

4.  Selection of Countries for the Case Studies

The selection of country cases for each topic will be the responsibility of the evaluator(s) for each of the three topics in consultation with the other evaluators and with concerned country authorities, subject to the following guidelines: (1) the number of countries for each topic should be in the range of 4 to 7, with as much overlap as feasible in the choice of countries for the three topics; (2) the groups of countries should be geographically diverse; and (3) the sample should contain both strong and weak performing countries.

5.  Access to Confidential Information and Protection of Confidentiality

	(i)	Each evaluator shall have access to any information in possession of the Fund as needed for carrying out the evaluation. This may include, but will not necessarily be limited to, access to staff reports, internal memoranda and studies, existing data bases, briefing papers and debriefing reports and other communications with management, as well as minutes of Executive Board proceedings. The Chairman of the Evaluation Group of Executive Directors (Evaluation Group) shall make all necessary arrangements to facilitate and assist the procurement by the evaluators of relevant information in possession of the Fund. The evaluators shall be invited to attend Executive Board consideration of the internal evaluation of ESAF-supported programs to be concluded in the spring of 1997 by the Policy Development and Review Department, as referenced in Section 2, above.
	(ii)	Each evaluator undertakes not to disclose, deliver, or use for personal gain or for the benefit of any person or entity without the consent of the IMF, any restricted or confidential information in possession of the Fund that he receives in the course of the evaluation. The Chairman of the Evaluation Group will ensure that the draft evaluation report will be reviewed at an early stage by an appropriate officer of the Fund with the express purpose of ensuring that restricted or confidential information will not be disclosed inadvertently.
	(iii)	Each evaluator is free to request information from country authorities and other sources outside the Fund as he deems appropriate.

 
6.  Evaluation Report: Publication, Executive Board Consideration and Comments

	(i)	The evaluators are invited to prepare their report on the assumption that it would be published by the Fund. The Executive Board will decide at a later stage whether or not the evaluation report will be made public. The Fund reserves the exclusive right to publish the report, and the evaluators undertake, individually and collectively, not to publish any part of the report separately.
	(ii)	In accordance with generally accepted practices for the conduct of audits and evaluations, the Chairman of the Evaluation Group will ensure that those whose actions and advice are the subject of evaluation shall have the opportunity to respond to relevant parts of the evaluation report in draft form, as well as in final form. The evaluators are free to take account of, or ignore, any comments on their draft evaluation report.
	(iii)	Comments on the final evaluation report shall be considered part of the official record. If the Executive Board decides to make public the final evaluation report, it may also decide to make public the comments thereon, including the conclusions of the Executive Board consideration of the report.