## _052113a

## Source details

**Canonical URL:** [_052113a](https://www.imf.org/-/media/websites/imf/imported-full-text-pdf/external/np/pp/eng/2013/_052113a.pdf)

## Other formats

- [Markdown version](/-/media/websites/imf/imported-full-text-pdf/external/np/pp/eng/2013/_052113a.pdf.md)
- [Structured JSON version](/-/media/websites/imf/imported-full-text-pdf/external/np/pp/eng/2013/_052113a.pdf.json)

---

### Background and scope
- Supplement provides background information for the main Board paper, The Fund’s Capacity Development Strategy—Better Policies through Stronger Institutions.
- Divided into nine notes/sections (A–I), focused on: institutions and growth (A); changes in CD landscape since the late 1990s (B); measurement issues with CD data (C); historical perspective of Fund TA (D); prior efforts to prioritize CD and assessment of RSNs and country pages (E); comparison of TA funding models (F); current monitoring and evaluation practices for TA and training (G); synergies between TA and training (H); and the revolution in online learning (I).
- Prepared by ICD’s Strategy and Evaluation Division; date in document: May 21, 2013.

### Institutions, economic development, and the Fund’s role
- Core premise: institutional quality is important for economic development; renewed empirical interest; 1993 Nobel Prize to Douglass North cited.
- Empirical challenges:
  - Positive correlation between institutional indicators and income per capita does not by itself establish causality due to endogeneity.
  - Researchers use instrumental variables (e.g., mortality rates, geographic/historical variables) to address endogeneity.
- Quantified impact (from cited WEO 2003 example):
  - Improving sub-Saharan Africa’s institutions to the world average would result in a two-and-a-half time’s increase in income per capita (over time).
  - Such institutional improvement would increase GDP growth by 1.7 percentage points.
- Fund activities related to institutions:
  - Technical assistance (TA) is a core Fund activity and a channel for institutional strengthening.
  - The Capability Assessment Program (CAP) is a pilot program to evaluate whether institutions at the core of macroeconomic policy can formulate and implement sound monetary and fiscal policies (Box 2).
  - More research needed on the effects of Fund TA on institutional improvement and on subsequent growth and stability.

### Stylized facts on capacity development (CD) and financing
- CD composition and historical shares:
  - CD (composed of TA and training) accounted for 23% of the Fund’s activities in FY 2012.
  - CD share was 14 percent in FY 1998 (as reported in the 1999 Review of Fund Technical Assistance).
- Two sources of CD financing: internal Fund financing and donor financing.
  - Donor partnerships started in the 1980s, grew since the 1990s, and intensified since 2008.
  - Across departments, most of the increase in TA delivery since FY 2009 has been financed by donors.
  - Donor support for training is important, mostly as co-sponsorships of regional training centers (RTCs).
- Key FY 2012 cost and financing figures (preserve numeric values exactly):
  - CD was 23 percent of Fund output (ACES basis (Table 1)).
  - Donor financing contributed about $103 million to CD activities, covering about half of the total direct costs of $215 million.
  - When accounting for support costs as total costs rise to $272 million (pre-ACES basis (Table 1)), donor financing contribution falls to about 40 percent.
  - Donor financing is expected to rise to $130–$150 million per year for the next five years, reflecting existing commitments and continued funding of RTACs and TTFs.
  - Donors pay a seven percent trust fund management (TFM) fee; the $103 million figure includes a seven percent TFM fee.
  - Donor-financed activities reported do not include support or governance; measured in direct costs (inclusive of TFM fees), donor financing was about $103 million while the Fund’s share was $112 million—hence donor financing was about 50 percent of direct cost of total CD.
  - When the Fund’s contribution to support and governance is included, the Fund’s share rises to $169 million and total CD costs are $272 million; donor-financed activities then account for nearly 40 percent of total cost.
- Measurement and methodological notes:
  - TA and training data are collected in physical delivery units: field delivery years (TA) and participant weeks (training).
  - Conversion to dollar costs uses average salaries and travel expenses to estimate direct costs; these omit Fund-wide support and governance costs.
  - Support and governance costs are estimated to represent almost one-third of total costs and are crucial to include for institutional completeness.
  - ACES provides the best information for measuring total CD costs and comparing with other Fund outputs but is only available since FY 2011; pre-ACES was used for longer historical time series.
  - Uncertainties: classification of TA depends on how a person’s time is reported; likely underreporting of TA volume with unknown magnitude. Legal and budgetary definitions of TA differ; change in budget classification from output areas to responsibility areas resulted in reclassification of some Fund work from CD to surveillance.
- Next steps to improve measurement:
  - Review methodology for allocating overhead, particularly to donor-financed CD.
  - Ensure TA backstopping and project management tasks are more accurately identified.
  - Improve data quality by addressing reporting and accounting issues and clarifying definitions.
  - The second phase of the ACES project is expected to address these issues.

### TA delivery by department and modality (selected metrics)
- Department shares and changes (selected highlights):
  - MCM delivered the most TA in FY 2007; MCM’s share fell from 38 percent in FY 2007 to 27 percent in FY 2012 (in dollar or share terms), although in person years MCM’s TA remained relatively stable over that period.
  - FAD increased from 36 percent (70 person years) of TA delivered in FY 2007 to 50 percent (120 person years) in FY 2012.
- Delivery mechanism shifts (FY 2000 to FY 2012):
  - Short-term resident experts increased from about 10 percent of total field delivery in FY 2000 to about 25 percent in FY 2012.
  - Long-term resident experts fell from about 60 percent in FY 2000 to about 40 percent in FY 2012.
  - Regional experts (RTACs) rose from 5 percent in FY 2000 to roughly 38 percent in FY 2012.
  - Shares of peripatetic expert visits and TA missions remained broadly stable.

### Training venues, curriculum, and distribution by income and region
- Training venues and shifts:
  - Training shifted from Fund headquarters to overseas venues over time, tailored to regional needs.
  - FY 2012 training venue shares:
    - RTCs accounted for nearly 50 percent of total training volume.
    - Other overseas training accounted for about 30 percent of total training volume.
    - IMF headquarters delivered about 20 percent of total training volume.
  - Of the 30 percent in other overseas training in FY 2012, about 13 percent was delivered through the RTACs (courses typically restricted to officials from a specific country or limited countries).
  - New RTC addition noted in FY 2012: RTC in Kuwait (CEF).
- Curriculum and providers:
  - ICD curriculum aims to support effective macroeconomic management by combining policy-oriented lectures with hands-on workshops drawing from Fund experience.
  - ICD-provided courses—in financial programming and policies, macroeconomic management, and finance—account for about 60 percent of training.
  - Other providers include STA, FAD, MCM, the Finance Department (FIN), the Legal Department (LEG), and the RTACs.
  - Departmental participant-week shares (FY 2012):
    - ICD: 55
    - STA: 23
    - Other departments: 9
    - RTACs: 13
  - The share of finance courses increased from 2 percent in FY 2008 to 10 percent in FY 2012 in response to post-crisis member needs.
- Distribution by income and region:
  - TA remains concentrated in low- and lower-middle income countries, while training is focused on middle income countries.
  - TA share to low and lower-middle income countries fell from about 70 percent in FY 2008 to about 60 percent in FY 2012, although the volume of TA delivered to these countries increased over the same period.
  - Training recipients in FY 2012:
    - Middle income countries received about 70 percent of total training.
    - Low-income countries received about 20 percent of total training.
  - Regional distribution changes:
    - Africa (AFR) and the Western Hemisphere (WHD) have received growing shares of total TA delivery since FY 2000, partly through donor financing of new RTACs.
    - In FY 2012, levels of training delivered to AFR, MCD, and APD were roughly comparable; however, adjusting for the number of countries, training to AFR stood at just three-quarters of training to MCD and APD in FY 2012.
    - The recent establishment of the Africa Training Institute (ATI) in Mauritius is expected to boost delivery of training to AFR.

### Notable programs, tools, and acronyms
- Capability Assessment Program (CAP) — pilot program to evaluate core macroeconomic policy institutions; CAP will start with pilot cases including Tunisia, Libya, and Oman; participation is voluntary.
- Results-based and costing tools referenced: Analytical Costing and Estimation System (ACES); Resource Allocation Plan (RAP); Results-Based Management (RBM).
- Major institutional actors and centers referenced: AFRITACs (East, West, West 2, Central, South), CAPTAC-DR, CARTAC, METAC, PFTAC; RTCs: ATI, BTC, CTP, CEF, JPA, JVI, STI; India Training Program (Pune) delivered FY 2007–FY 2013.
- Top donors (FY 2010–14 commitments, as noted in the source): Japan, the EU, the United Kingdom, Canada, and Kuwait (contributions included support for training, bilateral TA, TTFs, and RTACs).

### Review of prior prioritization efforts and RSN/country page assessments
- Prior initiatives and lessons:
  - Resource Allocation Plan (RAP), introduced FY 1993, allocates TA resources on basis of priorities in RSNs but is not a strategic prioritization tool.
  - Prioritization filters (2001–2005) were discontinued in 2005 as "over determined."
  - Technical Assistance Consultations (TCs), Technical Cooperation Action Plans (TCAPs), and Technical Assistance Country Strategy Notes (TACSNs) were piloted and later discontinued or superseded.
  - Charging for TA (introduced 2008) was decided in 2011 not to be used as a prioritization tool.
  - Regional Strategy Notes (RSNs), introduced FY 2008, together with RAP form the current basis for prioritization; starting FY 2013, area departments added training prioritization to RSNs.
- Assessment of RSNs (FY 2008–13) — Table 2 scores (reported exactly):
  - Discussion of prioritization at regional level (TA):
    - FY 2008: 1.6
    - FY 2009: 1.2
    - FY 2010: 1.6
    - FY 2011: 1.6
    - FY 2012: 1.6
    - FY 2013: 2.0
  - Discussion of prioritization at regional level (training):
    - FY 2008–FY 2012: NA
    - FY 2013: 1.6
  - Discussion of unresourced TA demand at regional level:
    - FY 2008: 0.6
    - FY 2009: 0.4
    - FY 2010: 0.4
    - FY 2011: 0.4
    - FY 2012: 0.6
    - FY 2013: 0.6
  - Identification of regional short vs. medium-term needs/priorities:
    - FY 2008: 0.0
    - FY 2009: 0.0
    - FY 2010: 0.6
    - FY 2011: 0.6
    - FY 2012: 0.8
    - FY 2013: 1.6
  - Integration of regional training needs/priorities with regional TA needs/priorities:
    - FY 2008: 0.2
    - FY 2009: 0.2
    - FY 2010: 0.0
    - FY 2011: 0.4
    - FY 2012: 0.4
    - FY 2013: 1.0
  - Discussion of regional TA delivery (e.g., RTACs):
    - FY 2008: 0.8
    - FY 2009: 0.8
    - FY 2010: 1.0
    - FY 2011: 1.0
    - FY 2012: 1.0
    - FY 2013: 2.0
- Health check of RSN country pages (sample of intensive TA users, FY 2008–13) — Table 3 scores (reported exactly):
  - Identification of country short vs. medium-term needs/priorities:
    - FY 2008: 1.7
    - FY 2009: 1.6
    - FY 2010: 1.5
    - FY 2011: 1.3
    - FY 2012: 1.8
    - FY 2013: 2.0
  - Integration of country training needs/priorities with country TA needs/priorities:
    - FY 2008: 0.3
    - FY 2009: 0.3
    - FY 2010: 0.2
    - FY 2011: 0.1
    - FY 2012: 0.0
    - FY 2013: 0.8
  - Discussion of progress in implementing past TA:
    - FY 2008: 0.7
    - FY 2009: 0.5
    - FY 2010: 0.3
    - FY 2011: 0.3
    - FY 2012: 0.2
    - FY 2013: 2.5
  - Discussion of unresourced TA demand:
    - FY 2008: 0.0
    - FY 2009: 0.0
    - FY 2010: 0.0
    - FY 2011: 0.0
    - FY 2012: 0.0
    - FY 2013: 0.2
- Use of surveillance products for prioritization:
  - Spring 2008 Vulnerability Exercise for Emerging Markets (VEE) results showed very little difference in TA received before and after being classified as highly vulnerable in overall and fiscal dimensions; some evidence of increased TA after financial vulnerability ratings possibly driven by crisis intensification and program engagement.
  - Implication: VEE results may not have been actively used for TA prioritization at the time; scope exists to use RAMs in Article IV reports and vulnerability exercises more actively when discussing CD needs.

### Capability Assessment Program (CAP) — Box 2
- Objective and scope:
  - Assess countries’ ability to effectively formulate and implement sound monetary and fiscal policies.
  - CAP will evaluate the extent to which core macroeconomic policy institutions (including the ministry of finance and the central bank) can fulfill mandates, deliver on objectives, and respond to pressing policy challenges.
  - CAP will start with a few pilot cases, including Tunisia, Libya, and Oman. Participation is voluntary.
- Purpose and audience:
  - Provide a new way to assist country authorities in identifying urgent capacity needs across main macroeconomic policy institutions.
  - Main audience: country authorities; the assessment should help set CD priorities in consultation with multilateral and bilateral development partners.
- Nature of analysis:
  - CAP is intended as a high-level assessment of relative strengths and weaknesses; it complements existing diagnostic tools by covering the main institutions engaged in fiscal and monetary policy design and implementation.
  - CAP analysis will be less detailed than traditional diagnostic tools and will be informed by findings from Article IV consultations and FSAPs.

### Monitoring, evaluation, quality control, and RBM
- Monitoring and evaluation — summary findings:
  - Monitoring of outputs (particularly quality) and evaluations are undertaken but not done consistently to track progress, explain shortfalls, or point to areas/circumstances where Fund TA and training could be improved.
  - Most recent evaluations have been driven by donor requirements and often focus on specific projects or RTACs.
  - With the expansion of externally financed CD projects, the number of project- or location-specific evaluations continues to grow, while higher-level assessments are less frequently undertaken owing to cost considerations.
- Quality control of TA delivery and training:
  - Backstopping role increased with scaling up of external funding and greater use of short-term experts; backstopping defined as core work by Fund staff to support experts in the field to ensure consistency with international best practice and Fund standards and policies.
  - Departments have internal review processes; FAD conducts ex-post quality reviews of a random selection of TA reports.
  - Feedback practices vary: some departments solicit routine feedback; STA sends questionnaires six or 12 months after mission requesting status of implementation of recommendations.
  - ICD course development and review frameworks include a Curriculum Development Committee (CDC), division chief review, and testing of new workshops; ultimate training quality depends on trainers’ capabilities and experience.
- Evaluation instruments:
  - Fund-wide periodic reviews and an IEO independent evaluation have been undertaken.
  - Executive Board established a TA Evaluation Program in July 2002 to present three to four evaluations a year to the Board; the last summary report was provided in 2010.
  - Departments have undertaken evaluations not guided by donor financing (examples provided in source).
  - RTAC/TTF evaluations are routinely conducted shortly after the mid-point and no later than 40 months into funding cycles; only one completed evaluation of a TTF thus far.
- Training evaluation instruments and limitations:
  - Course participant feedback via formal questionnaires for every ICD-led course; follow-up independent surveys sent one year to 18 months after a course; triennial survey of sponsoring government agencies continues since 1995.
  - Limitations: evaluations focus on satisfaction rather than impact; dissemination of results is limited; until the 2012 triennial survey, questionnaires did not differentiate across regions or income groups.
- Performance assessment improvements:
  - Accountability framework introduced to align departments’ objectives with Fund-wide goals.
  - Results-based management (RBM) endorsed by the Executive Board to strengthen TA planning and results monitoring; RBM expected to capture outcome information and standardize measurement across TA departments.
  - RSNs: updated guidance on preparing RSNs introduced in FY 2013 to improve discussion of progress in implementing past TA.

### TA–Training synergies — findings, examples, and recommended elements
- Rationale:
  - Training for country officials and Fund staff helps member countries implement TA advice by providing analytical skills, supporting institutional buy-in, and facilitating dialogue with country teams.
- Existing synergies by design (examples by department):
  - FAD: training supports implementation of TA advice or assists policymakers to understand fiscal policy challenges; end-of-mission workshops include practical exercises.
  - LEG: in AML/CFT, training strengthens capacities of groups supporting enforcement of legal frameworks.
  - MCM: training supports implementation of TA on financial stability, stress testing, risk-based supervision; TA missions often include hands-on workshops.
  - STA: training on statistical methodology and compilation normally in response to earlier or ongoing TA; hands-on workshops important.
- Successful models:
  - Modernizing monetary policy frameworks in sub-Saharan Africa: coordinated technical assistance; training for mid- and higher-level officials; conferences; internal staff training aligned with TA and country training.
  - Jobs and growth agenda: coordinated CD activities aligned with surveillance objectives.
- Suggested elements for harnessing synergies:
  - Training for staff should present both latest research and practical tools applicable to country analysis.
  - Training for country officials and staff should be well aligned with TA.
  - Training should present cross-country experiences and best practices in implementing reform measures.
  - High-level seminars or short courses for policymakers can help ensure institutional buy-in.
  - Better alignment required between bilateral TA delivery and ICD’s regional training in applied macroeconomics and finance; complementary training for Fund staff could better support absorption of external TA and training by member countries.

### Revolution in online learning — MOOCs and implications for IMF training
- MOOCs features and opportunities:
  - MOOCs leverage technology to reach large audiences at relatively low marginal cost; offered free to anyone with internet access.
  - MOOCs are interactive with graded exercises and active discussion forums; lectures broken into eight to 12 minute segments interspersed with exercises, usually coupled with longer weekly assignments or exams.
  - Assignments typically machine graded; some humanities courses use peer assessment of essays.
  - MOOCs offer opportunities for blended learning and for research on how technology transforms learning since every answer and click is stored.
- Fund engagement:
  - The Fund negotiated a partnership with edX to pilot courses on the edX platform.
- Reach of MOOC providers (As of April 7, 2013) — Table 8 figures preserved as presented:
  - edX / Coursera
  - Partner institutions (number) 126 / 2
  - Courses (number) 323 / 13
  - Unique learners (number in millions) 0.8 / 1.1
  - Course enrollments (number in millions) 1.1 / 3.1
  - of which: international enrollments (share in percent) 70 / 70
- Notable enrolment examples:
  - Stanford Summer 2011 free artificial intelligence course attracted 160,000 students worldwide.
  - At first anniversary of their launch, edX and Coursera had enrolled 1.1 million and 3.1 million students, respectively.
- Revenue models:
  - Major MOOC providers (edX, Coursera, Udacity) committed to offering courses free and are seeking alternative revenue sources such as fees for optional verified certificates, job referrals for top students, and licensing courses to third parties.

*THE FUND’S CAPACITY DEVELOPMENT STRATEGY—BACKGROUND PAPER, INTERNATIONAL MONETARY FUND*

### REFERENCES ________________________________________________________________________ 35

### _052113a - REFERENCES ________________________________________________________________________ 35

### Background and scope
- Supplement provides background information for the main Board paper, The Fund’s Capacity Development Strategy—Better Policies through Stronger Institutions.
- Divided into nine notes/sections (A–I), focused on: institutions and growth (A); changes in CD landscape since the late 1990s (B); measurement issues with CD data (C); historical perspective of Fund TA (D); prior efforts to prioritize CD and assessment of RSNs and country pages (E); comparison of TA funding models (F); current monitoring and evaluation practices for TA and training (G); synergies between TA and training (H); and the revolution in online learning (I).
- Prepared by ICD’s Strategy and Evaluation Division; date in document: May 21, 2013.

### Institutions, economic development, and the Fund’s role
- Core premise: institutional quality is important for economic development; renewed empirical interest; 1993 Nobel Prize to Douglass North cited.
- Empirical challenges:
  - Positive correlation between institutional indicators and income per capita does not by itself establish causality due to endogeneity.
  - Researchers use instrumental variables (e.g., mortality rates, geographic/historical variables) to address endogeneity.
- Quantified impact (from cited WEO 2003 example):
  - Improving sub-Saharan Africa’s institutions to the world average would result in a two-and-a-half time’s increase in income per capita (over time).
  - Such institutional improvement would increase GDP growth by 1.7 percentage points.
- Fund activities related to institutions:
  - Technical assistance (TA) is a core Fund activity and a channel for institutional strengthening.
  - The Capability Assessment Program (CAP) is a pilot program to evaluate whether institutions at the core of macroeconomic policy can formulate and implement sound monetary and fiscal policies (Box 2).
  - More research needed on the effects of Fund TA on institutional improvement and on subsequent growth and stability.

### Stylized facts on capacity development (CD) and financing
- CD (composed of TA and training) accounted for 23% of the Fund’s activities in FY 2012 (Figure 1).
- Historical comparison:
  - CD share was 14 percent in FY 1998 (as reported in the 1999 Review of Fund Technical Assistance).
- Two sources of CD financing: internal Fund financing and donor financing.
  - Partnerships with donors started in the 1980s, grew since the 1990s, and intensified since 2008.
  - Across departments, most of the increase in TA delivery since FY 2009 has been financed by donors.
  - Donor support for training is important, mostly as co-sponsorships of regional training centers (RTCs).

### TA delivery by department and modality (selected metrics)
- Department shares and changes:
  - MCM delivered the most TA in FY 2007; MCM’s share fell from 38 percent in FY 2007 to 27 percent in FY 2012 (in dollar or share terms), although in person years MCM’s TA remained relatively stable over that period (Figure 2d).
  - FAD increased from 36 percent (70 person years) of TA delivered in FY 2007 to 50 percent (120 person years) in FY 2012.
- Delivery mechanism shifts (FY 2000 to FY 2012):
  - Short-term resident experts increased from about 10 percent of total field delivery in FY 2000 to about 25 percent in FY 2012.
  - Long-term resident experts fell from about 60 percent in FY 2000 to about 40 percent in FY 2012.
  - Regional experts (RTACs) rose from 5 percent in FY 2000 to roughly 38 percent in FY 2012.
  - Shares of peripatetic expert visits and TA missions remained broadly stable.

### Training venues and trends (selected metrics)
- Training shifted from Fund headquarters to overseas venues over time, tailored to regional needs.
- FY 2012 training venue shares:
  - RTCs accounted for nearly 50 percent of total training volume.
  - Other overseas training accounted for about 30 percent of total training volume.
  - IMF headquarters delivered about 20 percent of total training volume.
- Of the 30 percent in other overseas training in FY 2012, about 13 percent was delivered through the RTACs (courses typically restricted to officials from a specific country or limited countries).
- RTC network and changes:
  - RTCs deliver ICD program courses jointly managed by ICD and co-sponsors.
  - New RTC addition noted in FY 2012: RTC in Kuwait (CEF).

### Notable programs, tools, and acronyms highlighted
- Capability Assessment Program (CAP) — pilot program to evaluate core macroeconomic policy institutions (Box 2).
- Results-based and costing tools referenced in the document: Analytical Costing and Estimation System (ACES); Resource Allocation Plan (RAP); Results-Based Management (RBM).
- Major institutional actors and centers referenced: AFRITACs (East, West, West 2, Central, South), CAPTAC-DR, CARTAC, METAC, PFTAC; RTCs: ATI, BTC, CTP, CEF, JPA, JVI, STI; India Training Program (Pune) delivered FY 2007–FY 2013.

### Figures and tables (items enumerated in source)
- Figures listed include: Share of Costs of Major IMF Activities, FY 2012; IMF CD Financing, TA Delivery, and Training Venues; IMF Global CD Presence; IMF Training Providers and CD Recipients; IMF TA and Training, FY 1964–2012; Contributions by Major Donors, FY 2010–14; Internet Users by Country Income Level, FY 2000–13.
- Tables listed include: Comparison of Allocation Methodologies for Fund Outputs, FY 2012; Assessment of RSNs, FY 2008–13; Assessment of Country Pages, FY 2008–13; TA for Countries Rated as Highly Vulnerable (Overall/Fiscal/Financial) in the Spring 2008 VEE; Comparison of TA Funding Models; Reach of MOOC Providers.

*May 21, 2013 — THE FUND’S CAPACITY DEVELOPMENT STRATEGY—BACKGROUND PAPER*

### 16.      The training curriculum has been adapting to members’ needs. The Fund is a niche

### 16.      The training curriculum has been adapting to members’ needs. The Fund is a niche provider of training

### Training curriculum and providers
- The ICD curriculum aims to support effective macroeconomic management in member countries by combining policy-oriented lectures with hands-on workshops that draw from Fund experience.
- ICD-provided courses—in financial programming and policies, macroeconomic management, and finance—account for about 60 percent of training.
- Other providers include STA, FAD, MCM, the Finance Department (FIN), the Legal Department (LEG), and the RTACs.
- Departmental participant-week shares (FY 2012, Figure 4a):
  - ICD: 55
  - STA: 23
  - Other departments: 9
  - RTACs: 13
- The share of finance courses increased from 2 percent in FY 2008 to 10 percent in FY 2012 in response to post-crisis member needs.

### TA and training distribution by income and region
- TA remains concentrated in low- and lower-middle income countries, while training is focused on middle income countries.
- TA share to low and lower-middle income countries fell from about 70 percent in FY 2008 to about 60 percent in FY 2012, although the volume of TA delivered to these countries increased over the same period (Figure 4c).
- Training recipients in FY 2012:
  - Middle income countries received about 70 percent of total training.
  - Low-income countries received about 20 percent of total training (Figure 4d).
- Regional distribution changes:
  - Africa (AFR) and the Western Hemisphere (WHD) have received growing shares of total TA delivery since FY 2000, partly through donor financing of new RTACs (Figure 4e).
  - In FY 2012, levels of training delivered to AFR, MCD, and APD were roughly comparable (Figure 4f); however, adjusting for the number of countries, training to AFR stood at just three-quarters of training to MCD and APD in FY 2012.
  - The recent establishment of the Africa Training Institute (ATI) in Mauritius is expected to boost delivery of training to AFR.

### Measurement issues: basic facts, costing methods, and donor financing
- Basic Facts—FY 2012:
  - CD was 23 percent of Fund output (ACES basis (Table 1)).
  - Donor financing contributed about $103 million to CD activities, covering about half of the total direct costs of $215 million.
  - The contribution of donor financing falls to about 40 percent when accounting for support costs as total costs rise to $272 million (pre-ACES basis (Table 1)).
- Direct and total costs:
  - TA and training data are collected in physical delivery units: field delivery years (TA) and participant weeks (training).
  - Physical units are useful for time series but cannot be compared across activities without conversion to dollars.
- Dollar measures:
  - Conversion to dollar costs uses average salaries and travel expenses to estimate direct costs; these omit Fund-wide support and governance costs.
  - Support and governance costs are estimated to represent almost one-third of total costs and are crucial to include for institutional completeness.
- Allocation methodologies:
  - Pre-ACES approach: support and governance costs distributed proportionately based on each activity’s share of Fund-financed direct costs.
  - ACES approach: apportions support and governance costs based more closely on their use; methodological changes caused a three percentage point difference in shares due to (i) more granular pricing of direct labor costs; (ii) more accurate allocation of support; and (iii) change in allocation of governance costs to proxy for Board time.
  - ACES provides the best information for measuring total CD costs and comparing with other Fund outputs but is only available since FY 2011; pre-ACES was used for longer historical time series.
- Donor financing specifics:
  - Donors pay for staff and expert time for delivery, quality-control backstopping, and travel; donors also pay a seven percent trust fund management (TFM) fee.
  - The $103 million figure includes a seven percent TFM fee.
  - Donor-financed activities reported do not include support or governance; measured in direct costs (inclusive of TFM fees), donor financing was about $103 million while the Fund’s share was $112 million—hence donor financing was about 50 percent of direct cost of total CD.
  - When the Fund’s contribution to support and governance is included, the Fund’s share rises to $169 million and total CD costs are $272 million; donor-financed activities then account for nearly 40 percent of total cost.
  - Donor financing is expected to rise to $130–$150 million per year for the next five years, reflecting existing commitments and continued funding of RTACs and TTFs.
- Contributions in kind:
  - Donors or host countries sometimes provide facilities and other in-kind services; these are not captured in the Fund’s financial systems and are not included in budget data.
- Uncertainties in measurement:
  - Classification of TA depends on how a person’s time is reported (stand-alone TA mission versus surveillance mission team), implying likely underreporting of TA volume with unknown magnitude.
  - Legal and budgetary definitions of TA differ; some activities legally TA (e.g., non-mandatory FSAPs, ROSCs, G20 MAP) are not classified as TA in the Fund’s budget.
  - Change in budget classification from output areas to responsibility areas resulted in reclassification of some Fund work from CD to surveillance, affecting reported outputs.
- Next steps to improve measurement:
  - Review methodology for allocating overhead, particularly to donor-financed CD.
  - Ensure TA backstopping and project management tasks are more accurately identified.
  - Improve data quality by addressing reporting and accounting issues and clarifying definitions.
  - The second phase of the ACES project is expected to address these issues.

### Historical perspective of Fund TA (1964–present)
- Origins and early expansion:
  - The Fund first provided TA in 1946 in response to a request from the Government of Ecuador for advice on its banking and monetary situation.
  - TA was limited in scale for almost 20 years and focused on policy advice and Fund-supported programs; the first formal training initiative was launched in 1950 but was limited.
  - Starting in the 1960s, TA and training increased to meet member needs; Central Banking and Fiscal Affairs Departments were created in 1964 and the IMF Institute was set up to centralize and expand training.
  - Institute early courses focused on policy, analytic techniques, and statistical methods and were offered in Washington, up to 20 weeks.
- Second expansion phase (early 1990s):
  - Triggered by emergence of new countries (mostly former Soviet Union).
  - Joint Vienna Institute (JVI) established in 1992 to train authorities from countries in transition.
  - IMF-Singapore Regional Training Institute (STI) established in 1998 for the Asia-Pacific region.
  - Expansion of TA to new central banks and ramp-up in all areas of Fund expertise; donor contributions made expansion possible, with Japan emerging as the largest bilateral supporter.
  - Localized delivery expanded with PFTAC (1993), CARTAC (2001), East AFRITAC (2002), West AFRITAC (2003).
- Third expansion phase (mid-2000s):
  - Marked expansion in donor participation and a revised TA funding model in 2008 emphasizing donor co-financing.
  - Multilateral partnerships expanded, including RTACs and Topical Trust Funds (TTFs).
  - New RTACs opened in Middle East (METAC, 2004), Central America (CAPTAC-DR, 2009), Central AFRITAC (2007), AFRITAC South (2011), and West AFRITAC 2 (to be opened later in 2013).
  - First TTF established in 2009 on Anti-Money Laundering and Combating Financial Terrorism; subsequent TTFs on Managing Natural Resources Wealth and Tax Policy and Administration introduced in 2011.
  - Training RTcs expanded with CEF in Kuwait (opening in 2011) and ATI in Mauritius (to be opened later in 2013).
  - Figure 6 (FY 2010–14 commitments) shows top five donors were Japan, the EU, the United Kingdom, Canada, and Kuwait; contributions included support for training, bilateral TA, TTFs, and RTACs.

### Review of prior prioritization efforts
- Numerous efforts since the 2000s tried to prioritize CD; a durable solution has been elusive.
- Past initiatives summarized (Box 1):
  - Resource Allocation Plan (RAP), introduced FY 1993: provides detailed TA delivery information by region/country; allocates TA resources on basis of priorities in RSNs but is not a strategic prioritization tool.
  - Prioritization filters (2001–2005): identified Main Program Areas and Key Policy Initiatives; discontinued in 2005 as "over determined."
  - Technical Assistance Consultations (TCs) and Technical Cooperation Action Plans (TCAPs): piloted in 2001, recommended discontinued or used selectively; found not helpful for identifying TA priorities.
  - Technical Assistance Country Strategy Notes (TACSNs): piloted in 2006, superseded by RSNs because they were country-specific and not helpful in prioritizing among countries and regions.
  - Charging for TA (introduced 2008): in 2011 management decided not to use TA charges as a prioritization tool.
  - Regional Strategy Notes (RSNs), introduced FY 2008: intended to allow better prioritization of TA by sector and integrate with medium-term budget; starting FY 2013, area departments added training prioritization to RSNs to better integrate TA and training; together with RAP, RSNs form current basis for prioritization.
- Ongoing work:
  - Health check of RSNs and further strengthening of RSNs suggested, including use of the Fund’s surveillance products for prioritization.

*Prepared by IMF staff as presented in the source document.*

### 37.      As discussed in Box 1, RSNs were introduced in FY 2008 and, together with the RAP,

### _052113a - 37.      As discussed in Box 1, RSNs were introduced in FY 2008 and, together with the RAP,

### Assessment of RSNs (FY 2008–13)
- Methodology
  - RSNs for each area department were assessed for FY 2008–FY 2013.
  - Each RSN was reviewed independently by two staff and assessed using five questions and a four-point rating system from 0 to 3, where 0 = “not at all” and 3 = “to a great extent.”
  - With only a few exceptions, there was a high degree of consensus between independent assessor ratings; disagreements were resolved by discussion.

- Summary findings (Table 2 scores reported exactly as in source)
  - Discussion of prioritization at regional level (TA) (Table 2, column 1):
    - FY 2008: 1.6
    - FY 2009: 1.2
    - FY 2010: 1.6
    - FY 2011: 1.6
    - FY 2012: 1.6
    - FY 2013: 2.0
    - Interpretation: Broadly constant over FY 2008–13, some improvement in FY 2013, slight dip in FY 2009; discussions often reflected demand rather than clearly stated priorities. FY 2013 included an RSN that discussed both top-down and bottom-up prioritization.
  - Discussion of prioritization at regional level (training) (Table 2, column 2):
    - FY 2008–FY 2012: NA
    - FY 2013: 1.6
    - Interpretation: No discussion of training priorities prior to FY 2013; training mentioned in all RSNs in FY 2013 but mostly in a cursory way.
  - Discussion of unresourced TA demand at regional level (Table 2, column 3):
    - FY 2008: 0.6
    - FY 2009: 0.4
    - FY 2010: 0.4
    - FY 2011: 0.4
    - FY 2012: 0.6
    - FY 2013: 0.6
    - Interpretation: A handful of references but not systematic; could provide “unfiltered” demand signals useful for prioritization and mid-year reallocations.
  - Identification of regional short vs. medium-term needs/priorities (Table 2, column 4):
    - FY 2008: 0.0
    - FY 2009: 0.0
    - FY 2010: 0.6
    - FY 2011: 0.6
    - FY 2012: 0.8
    - FY 2013: 1.6
    - Interpretation: Essentially no discussion of short-term needs prior to FY 2010; short-term needs discussion became more prominent over time, particularly FY 2013.
  - Integration of regional training needs/priorities with regional TA needs/priorities (Table 2, column 5):
    - FY 2008: 0.2
    - FY 2009: 0.2
    - FY 2010: 0.0
    - FY 2011: 0.4
    - FY 2012: 0.4
    - FY 2013: 1.0
    - Interpretation: Very few instances prior to FY 2013; significant improvement in FY 2013 following new guidance.
  - Discussion of regional TA delivery (e.g., RTACs) (Table 2, column 6):
    - FY 2008: 0.8
    - FY 2009: 0.8
    - FY 2010: 1.0
    - FY 2011: 1.0
    - FY 2012: 1.0
    - FY 2013: 2.0
    - Interpretation: Roughly constant and cursory over FY 2008–12; improvement in FY 2013 without new regional guidance to explain the change.

### Health check of RSN country pages (sample of intensive TA users, FY 2008–13)
- Methodology
  - Sample included countries identified as intensive TA users across six years, e.g., Afghanistan, Burundi, Cambodia, Democratic Republic of Congo, Indonesia, Kosovo, Kyrgyz Republic, Peru, Serbia, Sudan, Syria, Timor Leste, and Ukraine.
  - Each country page assessed using four questions and a four-point rating system from 0 to 3 (0 = “not at all”; 3 = “to a great extent”).

- Summary findings (Table 3 scores reported exactly as in source)
  - Identification of country short vs. medium-term needs/priorities (Table 3, column 1):
    - FY 2008: 1.7
    - FY 2009: 1.6
    - FY 2010: 1.5
    - FY 2011: 1.3
    - FY 2012: 1.8
    - FY 2013: 2.0
    - Interpretation: Country pages more frequently and more clearly identified short- vs. medium-term CD needs than RSNs.
  - Integration of country training needs/priorities with country TA needs/priorities (Table 3, column 2):
    - FY 2008: 0.3
    - FY 2009: 0.3
    - FY 2010: 0.2
    - FY 2011: 0.1
    - FY 2012: 0.0
    - FY 2013: 0.8
    - Interpretation: Infrequent and cursory discussions of integration even after FY 2013; training mentioned more frequently in FY 2013 but integration with TA still limited.
  - Discussion of progress in implementing past TA (Table 3, column 3):
    - FY 2008: 0.7
    - FY 2009: 0.5
    - FY 2010: 0.3
    - FY 2011: 0.3
    - FY 2012: 0.2
    - FY 2013: 2.5
    - Interpretation: Little discussion prior to FY 2013; significant improvement in FY 2013 after country pages were revamped and short guidance was circulated.
  - Discussion of unresourced TA demand (Table 3, column 4):
    - FY 2008: 0.0
    - FY 2009: 0.0
    - FY 2010: 0.0
    - FY 2011: 0.0
    - FY 2012: 0.0
    - FY 2013: 0.2
    - Interpretation: Essentially no discussion of unresourced TA demand at country level, slight improvement in FY 2013; two country pages described unresourced TA demand with one indicating it would be part of a future work plan.

### Fund surveillance products as information sources for prioritization
- Observations
  - Surveillance products (Article IV discussions, FSAPs, ROSCs, vulnerability exercises) could be used more proactively to prioritize CD.
  - Using Spring 2008 Vulnerability Exercise for Emerging Markets (VEE) data, staff examined TA before and after countries were rated as highly vulnerable.
- Findings from vulnerability analysis
  - Table 4 (Overall VEE rating): Very little difference in TA received before and after being classified as highly vulnerable; an increase two years after rating may reflect Fund-supported arrangements rather than deliberate prioritization or lack of TA requests by vulnerable countries.
  - Table 5 (Fiscal vulnerability): Amount of fiscal TA not much different two years before vs. two years after being rated highly vulnerable on fiscal indicators; further study needed.
  - Table 6 (Financial vulnerability): Some evidence of an increase in TA two years after such a rating; result may be due to intensification of financial crisis by 2010 and driven by two countries that had completed Fund programs at that time.
- Implication
  - Spring 2008 VEE results may not have been actively used for TA prioritization at the time.
  - There may be scope for country teams to use the Risk Assessment Matrix (RAM) in Article IV reports and vulnerability exercise results more actively when discussing CD needs with authorities.

### Use of Ex-Post Assessments (EPAs) and Ex-Post Evaluations (EPEs)
- Findings
  - Guidance notes for EPEs and EPAs do not explicitly require priorities for follow-up CD, but some recent EPAs include such information (examples cited in source).
  - Recent EPEs do not provide information on TA priorities, consistent with differing purposes: EPAs are forward-looking and can include follow-up CD strategy; EPEs focus on consistency with Fund policies and program performance.

### Proposals to improve the RSNs
- Proposed actions
  - Introduce a formal guidance note for RSN preparation to:
    - Explicitly discuss what factors determine prioritization.
    - Better integrate training and TA needs and priorities.
    - Clarify short- vs. medium-term priorities.
  - Update the guidance note annually to reflect institutional priorities.
  - Include in the note sources of information on country priorities, such as RAMs in Article IV reports, vulnerability exercises, and potentially the CAP (after experience).
  - Recommend future work to examine whether priorities identified in ROSCs, FSAPs, and Article IV reports align with those in RSNs and country pages.

### Box 2 — The Capability Assessment Program (CAP)
- Objective and scope
  - Objective: assess countries’ ability to effectively formulate and implement sound monetary and fiscal policies.
  - CAP will evaluate the extent to which core macroeconomic policy institutions (including the ministry of finance and the central bank) can fulfill mandates, deliver on objectives, and respond to pressing policy challenges.
  - CAP will start with a few pilot cases, including Tunisia, Libya, and Oman. Participation is voluntary.
- Purpose and audience
  - Provide a new way to assist country authorities in identifying urgent capacity needs across main macroeconomic policy institutions.
  - Main audience: country authorities; the assessment should help set CD priorities in consultation with multilateral and bilateral development partners.
- Nature of analysis
  - CAP is intended as a high-level assessment of relative strengths and weaknesses; it complements existing diagnostic tools by covering the main institutions engaged in fiscal and monetary policy design and implementation.
  - CAP analysis will be less detailed than traditional diagnostic tools and will be informed by findings from Article IV consultations and FSAPs.

*THE FUND’S CAPACITY DEVELOPMENT STRATEGY—BACKGROUND PAPER, INTERNATIONAL MONETARY FUND*

### 57.      Monitoring and evaluation are two areas where many past reviews have noted that a

### Monitoring and evaluation are two areas where many past reviews have noted that a more systematic approach is needed.

### Monitoring and evaluation — summary findings
- Monitoring of outputs (particularly quality) and evaluations are undertaken but not done consistently to track progress, explain shortfalls, or point to areas/circumstances where Fund TA and training could be improved.
- Most recent evaluations have been driven by donor requirements and often focus on specific projects or RTACs.
- With the expansion of externally financed CD projects, the number of project- or location-specific evaluations continues to grow, while higher-level assessments are less frequently undertaken owing to cost considerations.

### Quality control of TA delivery and training — processes and practices
- Backstopping
  - The role of backstopping has increased substantially with the scaling up of external funding and greater use of short-term experts.
  - The amount of backstopping is largely influenced by the expertise and experience of the experts; new experts require more backstopping.
  - Backstopping is defined as the core work carried out by Fund staff directly to support experts in the field to ensure consistency with international best practice and Fund standards and policies.
- Review of reports
  - Departments typically leave an aide-memoire or a copy of the first draft of the TA report with authorities at the end of the mission.
  - Each department has its own internal review process; FAD conducts ex-post quality reviews of a random selection of TA reports.
- Feedback
  - Some departments routinely solicit feedback from authorities on the usefulness of missions; others seek feedback during the Fund’s Annual and Spring Meetings.
  - STA sends questionnaires to authorities six or 12 months after the mission requesting the status of implementation of recommendations.
- Training content and review
  - Course development in ICD: content of new courses developed by a working group and reviewed by the Curriculum Development Committee (CDC); CDC reviews existing courses to strengthen structure and cover current policy issues.
  - Lecture/workshop review: course coordinator ensures quality/coherence; division chief reviews course program before front office clearance; new workshops are “tested” via department/divisional seminars; existing workshops continuously reviewed and refined based on participant and counselor feedback.
  - Ultimate quality of ICD training depends on capabilities and experience of trainers.

### Evaluation instruments for TA
- Fund-wide evaluations
  - Periodic reviews or task forces since the 1999 Review of Fund Technical Assistance, including an independent evaluation by the IEO.
- Department evaluations
  - In July 2002, the Executive Board established a TA Evaluation Program in which three to four evaluations a year are undertaken and presented to the Board.
  - The last report summarizing such evaluations was provided to the Board in 2010.
  - After 2010, departments have undertaken some evaluations not guided by donor financing (examples: STA selects one or two TA intensive countries each year; FAD internal evaluation in Mozambique; MCM evaluation on lessons learned in Belize, Costa Rica, Panama).
- RTAC/TTF evaluations
  - Evaluations of RTACs and TTFs are routinely conducted shortly after the mid-point and no later than 40 months into the funding cycle.
  - There has been only one completed evaluation of a TTF thus far given that the other two TTF have not yet reached their mid-point.

### Evaluation instruments for training and coverage
- Course participants
  - Reactions and suggestions collected via formal questionnaire and probing during closing sessions for every ICD-led course; results transmitted to department and CDC.
- Follow-up independent surveys
  - Questionnaires sent one year to 18 months after a course to participants and sponsoring managers; surveys conducted by an independent market research firm to ensure anonymity.
  - Starting in 2003, one survey of a donor-financed course was conducted each year.
  - Coverage expanded: in FY 2010 seven courses delivered in 2009 were surveyed; in FY 2011 eight courses delivered in 2010 were surveyed.
  - One RTC conducted an ad-hoc alumni survey to solicit feedback on training priorities.
- Triennial survey
  - Since 1995, INS/ICD has conducted a survey of sponsoring government agencies.
  - The most recent survey was sent to the 579 government agencies in 185 countries that had sent participants to the IMF Institute training program during 2009–11.

### Limitations of training evaluations and dissemination
- Focus of training evaluations is heavily tilted toward satisfaction rather than impact; dissemination of results is limited.
- Survey results typically highly favorable for satisfaction; usefulness for on-the-job performance is queried, but other impacts are not probed in depth.
- Until the 2012 triennial survey, questionnaires did not differentiate across regions or income groups, limiting usefulness for adapting programs.
- Detailed results widely disseminated within ICD and a summary reported to management and other Fund departments.

### Performance assessment improvements: accountability frameworks and RBM
- Accountability framework
  - New instrument aligning departments’ objectives with Fund-wide goals, holding departments accountable for delivering on related indicators.
- Results-based management (RBM)
  - Executive Board endorsed an RBM system to strengthen TA planning and results monitoring.
  - RBM, when broadly adopted, will capture information on whether outcomes are being achieved and standardize measurement across TA departments to capture performance beyond outputs.
  - Example: STA conducted a TA evaluation on Peru containing elements of RBM.
- RSNs
  - Updated guidance on preparing the RSNs introduced in FY 2013 to improve discussion of progress in implementing past TA.

### TA–Training synergies — findings and recommendations
- Rationale
  - Relevant training for country officials and Fund staff helps member countries implement TA advice by providing analytical skills and supporting institutional buy-in and dialogue with country teams.
- Existing synergies by design (training delivered by TA departments and RTACs)
  - FAD: training supports implementation of TA advice or assists policymakers to understand fiscal policy challenges; training may focus on topical policy issues or specific reform initiatives; end-of-mission workshops include practical exercises.
  - LEG: in AML/CFT, training strengthens capacities of groups supporting enforcement of legal frameworks.
  - MCM: training supports implementation of TA on financial stability, stress testing, risk-based supervision; TA missions often include hands-on workshops.
  - STA: training on statistical methodology and compilation normally in response to earlier or ongoing TA; hands-on workshops important.
- Need for more careful planning
  - Better alignment required between bilateral TA delivery and ICD’s regional training in applied macroeconomics and finance.
  - Complementary training for Fund staff could better support absorption of external TA and training by member countries.
- Successful models
  - Modernizing monetary policy frameworks in sub-Saharan Africa: coordinated technical assistance; training for mid- and higher-level officials; conferences; internal staff training aligned with TA and country training.
  - Jobs and growth agenda: coordinated CD activities aligned with surveillance objectives (see Box 3).
- Suggested elements for harnessing synergies
  - Training for staff should present both latest research and practical tools applicable to country analysis.
  - Training for country officials and staff should be well aligned.
  - Training should present cross-country experiences and best practices in implementing reform measures.
  - High-level seminars or short courses for policymakers can help ensure institutional buy-in.

### Revolution in online learning — MOOCs and implications for IMF training
- MOOCs characteristics and advantages
  - MOOCs leverage technology to reach large audiences at relatively low marginal cost; offered free to anyone with internet access.
  - MOOCs are interactive with graded exercises and active discussion forums; lectures broken into eight to 12 minute segments interspersed with exercises, usually coupled with longer weekly assignments or exams.
  - Assignments typically machine graded; some humanities courses use peer assessment of essays.
- Fund engagement
  - The Fund negotiated a partnership with edX to pilot courses on the edX platform.
  - MOOCs offer opportunities for blended learning and for research on how technology transforms learning since every answer and click is stored.
- Reach of MOOC providers (As of April 7, 2013) — Table 8 figures preserved as presented
  - edXCoursera
  - Partner institutions (number)1262
  - Courses (number)32313
  - Unique learners (number in millions)0.81.1
  - Course enrollments (number in millions)1.13.1
  - of which: international enrollments (share in percent)7070
- Notable enrolment examples
  - Stanford Summer 2011 free artificial intelligence course attracted 160,000 students worldwide.
  - At first anniversary of their launch, edX and Coursera had enrolled 1.1 million and 3.1 million students, respectively.
- Revenue models
  - Major MOOC providers (edX, Coursera, Udacity) committed to offering courses free and are seeking alternative revenue sources such as fees for optional verified certificates, job referrals for top students, and licensing courses to third parties.

*Source: THE FUND’S CAPACITY DEVELOPMENT STRATEGY—BACKGROUND PAPER (excerpts).*

---


_Source: https://www.imf.org/-/media/websites/imf/imported-full-text-pdf/external/np/pp/eng/2013/_052113a.pdf_