## 3. For each policy

## Source details

**Canonical URL:** [3. For each policy](https://www.imf.org/-/media/websites/imf/imported-full-text-pdf/external/np/pp/eng/2016/_061716.pdf)

## Other formats

- [Markdown version](/-/media/websites/imf/imported-full-text-pdf/external/np/pp/eng/2016/_061716.pdf.md)
- [Structured JSON version](/-/media/websites/imf/imported-full-text-pdf/external/np/pp/eng/2016/_061716.pdf.json)

---

### Objectives and agreed approach
- Directors broadly agreed that policy and thematic reviews should define at the outset:
  - the objectives of the review; and
  - what would constitute policy success.
- Directors did not support spelling out ex ante how the self-assessment of such reviews should be conducted, noting that this would allow plans for such reviews to be:
  - integrated within the Fund’s overall planning framework; and
  - adapted to take account of the changing needs facing the institution.

### Management commitment and implementation
- Current best practices will be applied more consistently with respect to self-evaluation in policy and thematic reviews.
- Best practices will be implemented, including defining at the concept note stage:
  - the objectives of the review, including any self-evaluation elements;
  - what would constitute policy success; and
  - whether the work of the review will include drawing lessons on program design and results, and IMF performance.
- These best practices are outlined in the statement of principles for self-evaluation (Annex III).
- Departments conducting policy and thematic reviews will implement and monitor these practices.

### Deliverables and timelines referenced
- The Small Group on Knowledge Management (SGKM) is tasked to:
  - (i) produce concrete plans to make available high-quality, web-based information on critical topics for Fund staff;
  - (ii) enhance internal information classification and retrieval tools;
  - (iii) provide recommendations on the respective responsibilities of area, functional, and other departments, as well as each staff member, in strengthening KM; and
  - (iv) propose a permanent KM structure.
- The final report of the SGKM is expected to be delivered to management in Summer 2016.
- The Institute for Capacity Development (ICD) is developing a common evaluation framework, including self-evaluation, for IMF capacity development; ICD to implement.
- SPR has launched an exercise to identify and disseminate examples of good practices in Article IV staff reports; SPR to continue implementation.
- The What’s Important in Review group (comprised of SPR country reviewers) was created in 2015 to facilitate learning from experience and disseminate lessons learned in policy reviews.
- The Risk Management Unit (established in 2014) produces reports on the Fund’s risk profile at regular intervals and highlights areas for additional risk analysis and mitigation efforts as needed.

### Resource implications (Annex II — Resource Cost through FY 2017, Future Recurrent Costs)
- Recommendation 1: Adopt a broad policy or general principles to establish an explicit, institution-wide framework for self-evaluation in the IMF, including its goals, scope, outputs, utilization, and follow-up.
  - Resource Cost through FY 2017 (FTEs): 0.25
  - Future Recurrent Costs (FTEs per year): 0
- Recommendation 2: Conduct self-assessments for every IMF-supported program, giving country authorities the opportunity to express their views on program design and results, and IMF performance.
  - Resource Cost through FY 2017 (FTEs): 0
  - Future Recurrent Costs (FTEs per year): 0
- Recommendation 3: For each policy and thematic review, explicitly set out a plan for how the policies and operations it covers will be self-evaluated going forward.
  - Resource Cost through FY 2017 (FTEs): 0
  - Future Recurrent Costs (FTEs per year): 0
- Recommendation 4: Develop products and activities aimed at distilling and disseminating evaluative findings and lessons in ways that highlight their relevance for staff work and that facilitate learning.
  - Resource Cost through FY 2017 (FTEs): TBD
  - Future Recurrent Costs (FTEs per year): TBD

### Annex III — Principles and Best Practices (high-level points)
- Overarching objective: develop a culture of continuously learning from experience to assess compliance with policies, gauge effectiveness, consider improvements, and promote accountability.
- The Fund relies on a combination of internal self-evaluation and the Independent Evaluation Office, and takes self-evaluation findings into account when designing and implementing operational work and the Fund’s policy agenda.
- Policy and Thematic Reviews:
  - Strategic approach: step back occasionally to assess whether existing policies need adjustment.
  - Reviews conducted too frequently may lower ability to focus on key strategic issues and be resource intensive; most reviews are conducted on an as-needed basis, with need assessed after a predetermined amount of time.
  - Best practices when undertaking reviews include:
    - Defining at the concept note stage:
      - the objectives the policy was expected to achieve;
      - the scope of the review, including key questions and any self-evaluation elements;
      - what would constitute policy success; and
      - whether the review will draw lessons on program design and results, and IMF performance;
    - Providing cross-cutting findings and lessons to inform future policy development;
    - Paying explicit attention to staff practices as well as broader institutional policies and operations;
    - Using a wide variety of evaluative techniques, including analysis of previous staff reports, and the use of surveys and interviews;
    - Considering external views, ranging from commentary and analysis outside the Fund to external consultants and advisory groups.
- IMF Lending Arrangements:
  - Ex Post Evaluations (EPEs) are required within a year after the end of any arrangement under facilities in the General Resources Account that involve exceptional access.
  - For countries with longer-term program engagement (LTPE: at least seven of the past ten years), a peer-reviewed assessment is undertaken if a request for a new arrangement is made.
  - For all programs, country authorities will be given opportunities to express their views on program design and performance after program conclusion, both during the Article IV mission and at the Board meeting.
    - At the Article IV consultation immediately subsequent to the end of each program, country teams are encouraged to discuss authorities’ views on program design and results, and IMF performance; these discussions would typically take place at the time of the Article IV consultation but would not formally be part of the Article IV consultation process.
    - Staff will reflect lessons learned, where appropriate, in the relevant sections of the Article IV staff report and in future country work.
    - It is strongly encouraged that country authorities express their views at the Board meeting for the Article IV consultation subsequent to the end of each program (via Executive Director’s Buff statement or orally).
- Surveillance:
  - The Comprehensive Review of Surveillance is conducted periodically; learning from experience should be continuous.
  - Best practice: country teams and review departments reflect on effectiveness of surveillance activities and collect lessons learned for broader policy or thematic reviews.
  - SPR has launched an exercise to identify and disseminate good practice examples across Article IV staff reports; results will be disseminated to area departments.
- Capacity Development:
  - IMF is developing a common evaluation framework for capacity development, including self-evaluation by staff and external evaluations.
  - Framework will draw on OECD DAC criteria (relevance, effectiveness, efficiency, impact, and sustainability) and include processes for distilling and disseminating lessons from evaluations.
- Utilization and dissemination:
  - All policy and thematic reviews are expected to be presented within the Fund and, where applicable, to outside audiences and stakeholders.
  - Evaluations of all types are expected to be easily accessible upon completion and findings broadly shared among Fund staff so lessons learned can feed into future delivery.
  - Reviewing departments should place high value on dissemination of lessons from previous self-evaluation exercises.
  - The Small Group on Knowledge Management is producing concrete plans to make available high-quality, web-based information on critical topics for Fund staff, and to enhance internal information classification and retrieval tools.

*Source: _061716 - 3. For each policy (PDF chapter/section)*

### EXECUTIVE SUMMARY

### EXECUTIVE SUMMARY

### INTRODUCTION
- Paper sets out Management’s response to the Independent Evaluation Office’s (IEO) evaluation report “Self-Evaluation at the IMF.”
- Board discussion of the IEO report (September 18, 2015) welcomed the evaluation and noted:
  - There is considerable self-evaluation at the IMF that is generally of high quality and contributes usefully to reforms.
  - Gaps and weaknesses exist in the Fund’s self-evaluation; strengthening current mechanisms is needed.
  - Importance of a clearly articulated approach to self-evaluation that builds on current processes, takes due account of resource constraints, and adapts over time.
  - Need to better disseminate lessons from self-evaluation.
- This management implementation plan (MIP) proposes actions to address the IEO recommendations and explains how implementation will be monitored.
- Date: June 17, 2016.

### IEO RECOMMENDATIONS AND BOARD REACTIONS
- The IEO evaluation contains four key recommendations to enhance self-evaluation by strengthening learning, transparency, and accountability:
  - Adopt a broad policy or general principles to establish an explicit, institution-wide framework for self-evaluation in the IMF, including its goals, scope, outputs, utilization, and follow-up.
  - Conduct self-assessments for every IMF-supported program, giving country authorities the opportunity to express their views on program design and results, and IMF performance.
  - For each policy and thematic review, explicitly set out a plan for how the policies and operations it covers will be self-evaluated going forward.
  - Develop products and activities aimed at distilling and disseminating evaluative findings and lessons in ways that highlight their relevance for staff work and that facilitate learning.
- Executive Directors’ key views during the Board discussion:
  - Many Directors favored building on existing processes and integrating self-evaluation into the IMF’s institution-wide strategic planning framework rather than creating a separate architecture.
  - Directors recognized the value of drawing lessons from country experiences and integrating country authorities’ views. Most Directors favored a selective, risk-based approach rather than self-assessing every program.
  - Directors broadly agreed that policy and thematic reviews should define objectives and what constitutes policy success at the outset, but they did not favor prescribing ex ante how self-assessments of such reviews should be conducted.
  - There was broad support to develop products and revamp knowledge management practices to better distill and share lessons.

### IMPLEMENTATION PLAN
- Management proposes specific actions to address the Board-endorsed IEO recommendations and sets out monitoring arrangements. Proposed measures will be reviewed and adjusted as warranted.

- Recommendation 1: Adopt a broad policy or general principles for self-evaluation (goals, scope, outputs, utilization, follow-up).
  - Management’s statement of principles for self-evaluation is attached (Annex III) and will be posted on the internal website as a stand-alone document.
  - The statement:
    - Describes goals, scope, outputs, utilization, and follow-up of self-evaluation at the IMF.
    - Builds on existing processes to avoid creating a separate self-evaluation architecture.
    - Serves as practical input and reference for staff and reviewers, outlining general principles and pointing to best practices without establishing a checklist of requirements.
    - Allows application to evolve and avoids excessive rigidity.

- Recommendation 2: Conduct self-assessments for every IMF-supported program; give country authorities opportunities to express their views.
  - Management does not propose standalone self-assessments for every program, consistent with most Executive Directors’ views and recent measures replacing Ex-Post Assessments for LTPE countries with a succinct, peer-reviewed assessment.
  - Procedures to give authorities opportunities to express views:
    - Country teams are encouraged to discuss, at the Article IV consultation immediately subsequent to the end of each program, the authorities’ views on program design and results, and IMF performance. These discussions would take place at the time of the Article IV consultation but would not formally be part of the Article IV consultation process.
    - Management welcomes authorities’ expressions of views on program design, results, and IMF performance at the Board meeting for the Article IV consultation subsequent to the end of each program. These views could be given in the Executive Director’s Buff statement (part of the publication bundle) or provided orally by the Executive Director at the Board meeting.
  - Lessons learned:
    - Staff will reflect lessons learned from these discussions, where appropriate, in the relevant sections of the Article IV staff report and in subsequent country work.
    - Policy and thematic reviews (e.g., crisis program reviews, conditionality reviews, reviews of engagement with fragile states and small states) will continue to draw cross-cutting lessons from program analyses across countries and from individual country case studies.
  - Exceptions and notes:
    - In cases where a successor program is requested before the Article IV consultation subsequent to the end of a program, the proposed procedures would be conducted at that time.
    - Procedures apply to all programs except where more in-depth assessments are required. For countries with LTPE, the required peer-reviewed assessment would be conducted. Programs with exceptional access are subject to an ex post evaluation.

- Recommendation 3: For each policy and thematic review, explicitly set out a plan for how covered policies and operations will be self-evaluated going forward.
  - Mandate and best practices for policy and thematic reviews:
    - Define at the concept note stage:
      - (i) the objectives the policy was expected to achieve;
      - (ii) the scope of the review, e.g., key questions the review will cover, including any self-evaluation elements;
      - (iii) what would constitute policy success; and
      - (iv) whether the review will include drawing lessons on program design and results, and IMF performance.
    - These standards have been incorporated into the statement of principles for self-evaluation (Annex III).
    - Actual implementation will be monitored via the interdepartmental review process.

- Recommendation 4: Develop products and activities to distill and disseminate evaluative findings and lessons for staff work and learning.
  - Knowledge Management (KM):
    - Management established a Small Group on KM (SGKM) to develop a plan for enhancing KM at the Fund through “quick wins” and more fundamental proposals.
    - SGKM tasks:
      - Produce concrete plans to make available high-quality, web-based information on critical topics for Fund staff.
      - Enhance internal information classification and retrieval tools.
      - Provide recommendations on responsibilities of area, functional, and other departments, and each staff member, in strengthening KM for internal and external audiences.
      - Design a proposed permanent KM structure.
    - SGKM final report expected to be delivered to management in Summer 2016.
  - Capacity development evaluations:
    - The Institute for Capacity Development, working closely with other departments, is developing a common evaluation framework, including self-evaluation, for IMF capacity development (endorsed by the Board in its 2013 discussion of the IMF’s capacity development strategy).
    - The framework aims for more focused and comparable evaluations while allowing flexibility, and emphasizes accessibility and broad sharing of findings so lessons feed into future delivery.
  - Identifying good practices in surveillance:
    - SPR launched an exercise to identify and disseminate examples of good practices in Article IV staff reports across all policy areas; results will be disseminated to area departments.
    - A similar approach has been applied to mainstream macro-financial surveillance.
  - What’s Important in Review (WIR):
    - Group of SPR country reviewers, created in 2015, meets periodically to discuss important issues in country review to facilitate learning and disseminate lessons.
  - Risk Management Unit (RMU):
    - Established in 2014; produces reports on the Fund’s risk profile at regular intervals and highlights areas for additional risk analysis and mitigation as needed.

### MONITORING IMPLEMENTATION
- Coordination and monitoring:
  - Implementation of the MIP will be coordinated by the Strategy, Policy, and Review Department (SPR), in close collaboration with the Office of Budget and Planning for budget-related issues.
  - Monitoring will be primarily through future Periodic Monitoring Reports, which provide updates on implementation of IEO evaluations.
  - SPR will ensure via the regular interdepartmental review process that:
    - Country teams discuss program performance with authorities in the subsequent Article IV.
    - Policy and thematic reviews follow best practices in self-evaluating policies and operations covered by the review.
  - The Secretary’s Department will ensure the Compendium of Executive Board Work Procedures makes explicit that Directors are encouraged to convey the views of their authorities on program performance as described.

### RESOURCE IMPLICATIONS
- Overall resource implications described as small.
- Specifics:
  - Proposals to ensure self-assessment and inclusion of authorities’ views, and to more consistently apply self-evaluation best practices in policy reviews, will likely require some reprioritization and are expected to be broadly accommodated under the existing resource envelope.
  - One-time resource cost for formulation of this MIP and creation of the statement of principles and best practices: approximately 0.25 FTE in FY2016.
  - Development of products and activities to distill and disseminate evaluative findings and lessons will be implemented as part of the broader KM enhancement effort, with resource costs to be defined during the course of FY2017.

### ANNEX I (SUMMARY)
- Annex I summarizes the IEO recommendations, Executive Directors’ responses, proposed follow-up, timelines, and accountability arrangements.
- Examples from Annex I:
  - Recommendation 1 follow-up: high-level statement of principles (Annex III) posted internally; Management will update the statement as and when necessary.
  - Recommendation 2 follow-up: country teams to discuss authorities’ views at the Article IV subsequent to program end; lessons reflected in staff reports where appropriate; policy and thematic reviews to draw cross-cutting lessons; SPR to monitor implementation in normal review process; departments conducting policy and thematic reviews will implement and monitor.

*Source: MIP in Response to IEO Evaluation of Self-Evaluation at the IMF, June 17, 2016.*

### 3. For each policy

### 3. For each policy

### Objectives and agreed approach
- Directors broadly agreed that policy and thematic reviews should define at the outset:
  - the objectives of the review; and
  - what would constitute policy success.
- Directors did not support spelling out ex ante how the self-assessment of such reviews should be conducted, noting that this would allow plans for such reviews to be:
  - integrated within the Fund’s overall planning framework; and
  - adapted to take account of the changing needs facing the institution.

### Management commitment and implementation
- Current best practices will be applied more consistently with respect to self-evaluation in policy and thematic reviews.
- Best practices will be implemented, including defining at the concept note stage:
  - the objectives of the review, including any self-evaluation elements;
  - what would constitute policy success; and
  - whether the work of the review will include drawing lessons on program design and results, and IMF performance.
- These best practices are outlined in the statement of principles for self-evaluation (Annex III).
- Departments conducting policy and thematic reviews will implement and monitor these practices.

### Deliverables and timelines referenced
- The Small Group on Knowledge Management (SGKM) is tasked to:
  - (i) produce concrete plans to make available high-quality, web-based information on critical topics for Fund staff;
  - (ii) enhance internal information classification and retrieval tools;
  - (iii) provide recommendations on the respective responsibilities of area, functional, and other departments, as well as each staff member, in strengthening KM; and
  - (iv) propose a permanent KM structure.
- The final report of the SGKM is expected to be delivered to management in Summer 2016.
- The Institute for Capacity Development (ICD) is developing a common evaluation framework, including self-evaluation, for IMF capacity development; ICD to implement.
- SPR has launched an exercise to identify and disseminate examples of good practices in Article IV staff reports; SPR to continue implementation.
- The What’s Important in Review group (comprised of SPR country reviewers) was created in 2015 to facilitate learning from experience and disseminate lessons learned in policy reviews.
- The Risk Management Unit (established in 2014) produces reports on the Fund’s risk profile at regular intervals and highlights areas for additional risk analysis and mitigation efforts as needed.

### Resource implications (Annex II — Resource Cost through FY 2017, Future Recurrent Costs)
- Recommendation 1: Adopt a broad policy or general principles to establish an explicit, institution-wide framework for self-evaluation in the IMF, including its goals, scope, outputs, utilization, and follow-up.
  - Resource Cost through FY 2017 (FTEs): 0.25
  - Future Recurrent Costs (FTEs per year): 0
- Recommendation 2: Conduct self-assessments for every IMF-supported program, giving country authorities the opportunity to express their views on program design and results, and IMF performance.
  - Resource Cost through FY 2017 (FTEs): 0
  - Future Recurrent Costs (FTEs per year): 0
- Recommendation 3: For each policy and thematic review, explicitly set out a plan for how the policies and operations it covers will be self-evaluated going forward.
  - Resource Cost through FY 2017 (FTEs): 0
  - Future Recurrent Costs (FTEs per year): 0
- Recommendation 4: Develop products and activities aimed at distilling and disseminating evaluative findings and lessons in ways that highlight their relevance for staff work and that facilitate learning.
  - Resource Cost through FY 2017 (FTEs): TBD
  - Future Recurrent Costs (FTEs per year): TBD

### Annex III — Principles and Best Practices (high-level points)
- Overarching objective: develop a culture of continuously learning from experience to assess compliance with policies, gauge effectiveness, consider improvements, and promote accountability.
- The Fund relies on a combination of internal self-evaluation and the Independent Evaluation Office, and takes self-evaluation findings into account when designing and implementing operational work and the Fund’s policy agenda.
- Policy and Thematic Reviews:
  - Strategic approach: step back occasionally to assess whether existing policies need adjustment.
  - Reviews conducted too frequently may lower ability to focus on key strategic issues and be resource intensive; most reviews are conducted on an as-needed basis, with need assessed after a predetermined amount of time.
  - Best practices when undertaking reviews include:
    - Defining at the concept note stage:
      - the objectives the policy was expected to achieve;
      - the scope of the review, including key questions and any self-evaluation elements;
      - what would constitute policy success; and
      - whether the review will draw lessons on program design and results, and IMF performance;
    - Providing cross-cutting findings and lessons to inform future policy development;
    - Paying explicit attention to staff practices as well as broader institutional policies and operations;
    - Using a wide variety of evaluative techniques, including analysis of previous staff reports, and the use of surveys and interviews;
    - Considering external views, ranging from commentary and analysis outside the Fund to external consultants and advisory groups.
- IMF Lending Arrangements:
  - Ex Post Evaluations (EPEs) are required within a year after the end of any arrangement under facilities in the General Resources Account that involve exceptional access.
  - For countries with longer-term program engagement (LTPE: at least seven of the past ten years), a peer-reviewed assessment is undertaken if a request for a new arrangement is made.
  - For all programs, country authorities will be given opportunities to express their views on program design and performance after program conclusion, both during the Article IV mission and at the Board meeting.
    - At the Article IV consultation immediately subsequent to the end of each program, country teams are encouraged to discuss authorities’ views on program design and results, and IMF performance; these discussions would typically take place at the time of the Article IV consultation but would not formally be part of the Article IV consultation process.
    - Staff will reflect lessons learned, where appropriate, in the relevant sections of the Article IV staff report and in future country work.
    - It is strongly encouraged that country authorities express their views at the Board meeting for the Article IV consultation subsequent to the end of each program (via Executive Director’s Buff statement or orally).
- Surveillance:
  - The Comprehensive Review of Surveillance is conducted periodically; learning from experience should be continuous.
  - Best practice: country teams and review departments reflect on effectiveness of surveillance activities and collect lessons learned for broader policy or thematic reviews.
  - SPR has launched an exercise to identify and disseminate good practice examples across Article IV staff reports; results will be disseminated to area departments.
- Capacity Development:
  - IMF is developing a common evaluation framework for capacity development, including self-evaluation by staff and external evaluations.
  - Framework will draw on OECD DAC criteria (relevance, effectiveness, efficiency, impact, and sustainability) and include processes for distilling and disseminating lessons from evaluations.
- Utilization and dissemination:
  - All policy and thematic reviews are expected to be presented within the Fund and, where applicable, to outside audiences and stakeholders.
  - Evaluations of all types are expected to be easily accessible upon completion and findings broadly shared among Fund staff so lessons learned can feed into future delivery.
  - Reviewing departments should place high value on dissemination of lessons from previous self-evaluation exercises.
  - The Small Group on Knowledge Management is producing concrete plans to make available high-quality, web-based information on critical topics for Fund staff, and to enhance internal information classification and retrieval tools.

*Source: _061716 - 3. For each policy (PDF chapter/section)*

---


_Source: https://www.imf.org/-/media/websites/imf/imported-full-text-pdf/external/np/pp/eng/2016/_061716.pdf_