## Appraising Credit Ratings: Does the CAP Fit Better than the ROC?

_IMF Working Papers, May 1, 2012_

## Source details

**Canonical URL:** [Appraising Credit Ratings: Does the CAP Fit Better than the ROC?](https://www.imf.org/en/publications/wp/issues/2016/12/31/appraising-credit-ratings-does-the-cap-fit-better-than-the-roc-25910)

## Other formats

- [Markdown version](/en/publications/wp/issues/2016/12/31/appraising-credit-ratings-does-the-cap-fit-better-than-the-roc-25910/index.md)
- [Structured JSON version](/en/publications/wp/issues/2016/12/31/appraising-credit-ratings-does-the-cap-fit-better-than-the-roc-25910/index.json)
- [Bundle manifest](/en/publications/wp/issues/2016/12/31/appraising-credit-ratings-does-the-cap-fit-better-than-the-roc-25910/bundle-manifest.json)

## Bibliographic details
- Authors: R. John Irwin, Timothy C Irwin
- Published: May 1, 2012
- Series: IMF Working Papers
- DOI: https://doi.org/10.5089/9781475503517.001

---

### Summary
- ROC and CAP analysis are alternative methods for evaluating diagnostic systems, including assessments of credit risk.
- ROC analysis is widely used across many fields; CAP analysis is more common in finance.
- The paper compares the two methods using the OECD’s country risk ratings to predict whether a country will have a program with the IMF (an indicator of financial distress).
- Both ROC and CAP generate measures of accuracy that are independent of the choice of diagnostic threshold (such as risk rating).

### Comparison of ROC and CAP
- ROC analysis:
  - Widely used in many fields.
  - Has a natural interpretation of its measure of accuracy.
  - Produces curves that are independent of the probability of default.
  - Includes established theories for fitting models to data and for setting the optimal diagnostic threshold.
- CAP analysis:
  - More common in finance.
  - Also generates measures of accuracy independent of diagnostic threshold.
  - Lacks the independence from the probability of default that ROC curves provide.
  - Could potentially incorporate model-fitting and threshold-setting theories analogous to those used in ROC analysis.

### Illustration and Empirical Application
- Empirical illustration uses the OECD’s country risk ratings to assess prediction of whether a country will have an IMF program, treated as an indicator of financial distress.
- Both ROC and CAP are applied to this problem to compare their performance and interpretability in a finance-related diagnostic setting.

### Key Findings and Methodological Implications
- Both ROC and CAP analyses are useful because they provide threshold-independent accuracy measures.
- ROC’s advantages:
  - Clear interpretation of its accuracy metric.
  - Independence from the probability of default.
  - Existing theoretical tools for model fitting and optimal threshold selection.
- CAP could be enhanced by adopting ROC-like theoretical tools for fitting and threshold selection.
- Nonetheless, the interpretation and probability-independence features of ROC give it advantages unavailable to CAP analysis as currently used.

*By R. John Irwin and Timothy C Irwin — Appraising Credit Ratings: Does the CAP Fit Better than the ROC? (IMF Working Paper).*

---

## Content in this bundle

- **_wp12122**
  - [_wp12122 (Markdown version)](/-/media/websites/imf/imported-full-text-pdf/external/pubs/ft/wp/2012/_wp12122.pdf.md){rel="alternate" type="text/markdown"}
  - [_wp12122 (PDF)](/-/media/websites/imf/imported-full-text-pdf/external/pubs/ft/wp/2012/_wp12122.pdf){rel="external" type="application/pdf"}

---

_Source: https://www.imf.org/en/publications/wp/issues/2016/12/31/appraising-credit-ratings-does-the-cap-fit-better-than-the-roc-25910_