IMF Executive Board Discussed Use of Third-Party Indicators in Fund Reports
IMF News, November 22, 2017
Source details
- Canonical URL
- IMF Executive Board Discussed Use of Third-Party Indicators in Fund Reports
Other formats
Bibliographic details
- Published: November 22, 2017
Summary of the Board discussion
- On November 10, 2017, the Executive Board of the International Monetary Fund (IMF) discussed a staff paper on “Use of Third-Party Indicators in Fund Reports.”
- Press Release No. 17/456
- Publication date on the page: November 22, 2017
- MEDIA RELATIONS PRESS OFFICER: Raphael Anspach; Phone: +1 202 623-7100; Email: MEDIA@IMF.org
- The use of comparative indicators developed by other organizations (third‑party indicators, TPIs) is widespread in core areas of Fund surveillance and is increasing in emerging macro‑critical areas.
Rationale for using TPIs
- Staff rely on TPIs to measure concepts such as:
- business environment
- competitiveness
- quality of governance
- progress toward social development goals
- TPIs support evidence‑based work in priority areas such as:
- risks and spillovers
- macro‑financial surveillance
- structural issues
- emerging macro‑critical areas like inequality, gender, and climate change
- program work
- Staff are expected to continue drawing on other institutions’ expertise and estimates, consistent with Executive Board guidance to leverage outside expertise where internal expertise is limited or lacking.
Framework proposed by staff
- The paper outlines a three‑element framework to promote best practice in TPI use:
- (i) a principles‑based Guidance Note for staff (forthcoming);
- (ii) an “Indicators Digest” (a database for internal use only) compiling selected indicators’ characteristics to inform staff’s judgment, based on the adapted Data Quality Assessment Framework (DQAF);
- (iii) the review process to ensure best practice in Fund reports.
- Scope of application:
- The framework will apply to all country documents, policy documents, and multi‑country documents that are subject to the Fund’s Transparency Policy.
- Staff are encouraged to follow similar guidelines for other Fund documents not subject to the Transparency Policy (e.g., Regional Economic Outlooks and Staff Discussion Notes).
Executive Board assessment — key observations and concerns
- Directors welcomed discussion of TPIs and noted TPIs’ usefulness for:
- facilitating cross‑country comparisons
- identifying evidence to inform analysis
- complementing assessments where the institution may not gather primary data
- Concerns and challenges highlighted:
- Varied qualities of TPIs present challenges and risks to the Fund’s credibility.
- Some TPIs are based on hard data; others are based on qualitative assessment by experts, experience, perception, or composites.
- Opacity of some TPIs’ sources and methodologies is a concern.
- Quality and reliability concerns for TPIs based on perception or value judgment and those produced by private entities.
- Lack of transparency in staff’s choice of some TPIs or their components.
- Directors welcomed a more structured approach to promote best practice, consistency, and evenhandedness.
Endorsed principles and recommended practices
- Directors endorsed three complementary principles for best practice:
- transparency in the selection and interpretation of the indicators;
- robustness checks;
- presentation of the views of stakeholders.
- Specific recommendations and emphases from Directors:
- Provide clear justification whenever TPIs are used; a few Directors suggested including an appropriate disclaimer.
- Discuss key characteristics and limitations of indicators and how results inform the overall assessment.
- Use indicators as one of many inputs, complementing qualitative discussion that draws on multiple indicators and sources measuring similar concepts to promote robustness.
- Caution that indicators derived from the same underlying data might be correlated and biased in the same direction.
- Present staff’s independent and candid views while including views of authorities and other key stakeholders when interpretations differ or additional context exists.
- Prefer data‑based TPIs over perception‑based ones, though some Directors noted potential value of perception‑based indicators.
- Ensure evenhanded use based on macro‑criticality and country‑specific context.
- Many Directors considered it important that the Board be able to access TPI data used by staff.
- In emerging macro‑critical areas relevant to the Fund’s mandate, many Directors favored developing in‑house expertise; others cautioned about resource implications.
- Directors underlined the importance of collaboration with other international organizations.
Indicators Digest — role and characteristics
- Directors appreciated staff efforts to assemble the Indicators Digest, applying an adapted version of the Board‑endorsed Data Quality Assessment Framework.
- Clarifications about the Indicators Digest:
- It is an internal, centralized quality assessment database.
- It is not an ex‑ante positive or negative list of acceptable indicators for use in staff analysis and Fund work.
- It is not an exhaustive compilation of all possible TPIs staff may employ nor a validation of those TPIs.
- It is a living database; future expansion should be demand‑driven.
- Suggestions from Directors for Digest improvement:
- Include greater detail on the nature of data, methodology used for compiling each indicator, and specific components that are actionable for governments.
Next steps and engagement
- Directors’ views will provide inputs into the forthcoming Guidance Note for staff.
- Directors look forward to a further opportunity to engage with staff before the Guidance Note’s finalization.
International Monetary Fund — Executive Board discussion summary as presented on the IMF web page.