{
  "title": "Appraising Credit Ratings: Does the CAP Fit Better than the ROC?",
  "publication": "IMF Working Papers, May 1, 2012",
  "sourceUrl": "https://www.imf.org/en/publications/wp/issues/2016/12/31/appraising-credit-ratings-does-the-cap-fit-better-than-the-roc-25910",
  "canonical": "https://www.imf.org/en/publications/wp/issues/2016/12/31/appraising-credit-ratings-does-the-cap-fit-better-than-the-roc-25910",
  "overlayPath": "/en/publications/wp/issues/2016/12/31/appraising-credit-ratings-does-the-cap-fit-better-than-the-roc-25910/index.md",
  "summary": "ROC and CAP analysis are alternative methods for evaluating a wide range of diagnostic systems, including assessments of credit risk. ROC analysis is widely used in many fields, but in finance CAP analysis is more common.",
  "sections": [
    {
      "heading": "Summary",
      "content": "- ROC and CAP analysis are alternative methods for evaluating diagnostic systems, including assessments of credit risk.\n- ROC analysis is widely used across many fields; CAP analysis is more common in finance.\n- The paper compares the two methods using the OECD’s country risk ratings to predict whether a country will have a program with the IMF (an indicator of financial distress).\n- Both ROC and CAP generate measures of accuracy that are independent of the choice of diagnostic threshold (such as risk rating)."
    },
    {
      "heading": "Comparison of ROC and CAP",
      "content": "- ROC analysis:\n  - Widely used in many fields.\n  - Has a natural interpretation of its measure of accuracy.\n  - Produces curves that are independent of the probability of default.\n  - Includes established theories for fitting models to data and for setting the optimal diagnostic threshold.\n- CAP analysis:\n  - More common in finance.\n  - Also generates measures of accuracy independent of diagnostic threshold.\n  - Lacks the independence from the probability of default that ROC curves provide.\n  - Could potentially incorporate model-fitting and threshold-setting theories analogous to those used in ROC analysis."
    },
    {
      "heading": "Illustration and Empirical Application",
      "content": "- Empirical illustration uses the OECD’s country risk ratings to assess prediction of whether a country will have an IMF program, treated as an indicator of financial distress.\n- Both ROC and CAP are applied to this problem to compare their performance and interpretability in a finance-related diagnostic setting."
    },
    {
      "heading": "Key Findings and Methodological Implications",
      "content": "- Both ROC and CAP analyses are useful because they provide threshold-independent accuracy measures.\n- ROC’s advantages:\n  - Clear interpretation of its accuracy metric.\n  - Independence from the probability of default.\n  - Existing theoretical tools for model fitting and optimal threshold selection.\n- CAP could be enhanced by adopting ROC-like theoretical tools for fitting and threshold selection.\n- Nonetheless, the interpretation and probability-independence features of ROC give it advantages unavailable to CAP analysis as currently used.\n\nBy R. John Irwin and Timothy C Irwin — Appraising Credit Ratings: Does the CAP Fit Better than the ROC? (IMF Working Paper).\n\n---\n\n Content in this bundle\n\n- wp12122\n  - wp12122 (Markdown version){rel=\"alternate\" type=\"text/markdown\"}\n  - wp12122 (PDF){rel=\"external\" type=\"application/pdf\"}\n\n---\n\nSource: https://www.imf.org/en/publications/wp/issues/2016/12/31/appraising-credit-ratings-does-the-cap-fit-better-than-the-roc-25910"
    }
  ],
  "bullets": [
    "[Markdown version](/en/publications/wp/issues/2016/12/31/appraising-credit-ratings-does-the-cap-fit-better-than-the-roc-25910/index.md)",
    "[Structured JSON version](/en/publications/wp/issues/2016/12/31/appraising-credit-ratings-does-the-cap-fit-better-than-the-roc-25910/index.json)",
    "[Bundle manifest](/en/publications/wp/issues/2016/12/31/appraising-credit-ratings-does-the-cap-fit-better-than-the-roc-25910/bundle-manifest.json)",
    "Authors: R. John Irwin, Timothy C Irwin",
    "Published: May 1, 2012",
    "Series: IMF Working Papers",
    "DOI: https://doi.org/10.5089/9781475503517.001",
    "ROC and CAP analysis are alternative methods for evaluating diagnostic systems, including assessments of credit risk.",
    "ROC analysis is widely used across many fields; CAP analysis is more common in finance.",
    "The paper compares the two methods using the OECD’s country risk ratings to predict whether a country will have a program with the IMF (an indicator of financial distress).",
    "Both ROC and CAP generate measures of accuracy that are independent of the choice of diagnostic threshold (such as risk rating).",
    "ROC analysis:",
    "CAP analysis:",
    "Empirical illustration uses the OECD’s country risk ratings to assess prediction of whether a country will have an IMF program, treated as an indicator of financial distress.",
    "Both ROC and CAP are applied to this problem to compare their performance and interpretability in a finance-related diagnostic setting.",
    "Both ROC and CAP analyses are useful because they provide threshold-independent accuracy measures.",
    "ROC’s advantages:",
    "CAP could be enhanced by adopting ROC-like theoretical tools for fitting and threshold selection.",
    "Nonetheless, the interpretation and probability-independence features of ROC give it advantages unavailable to CAP analysis as currently used.",
    "**_wp12122**"
  ],
  "related": [
    {
      "title": "_wp12122",
      "role": "document",
      "sourceUrl": "https://www.imf.org/-/media/websites/imf/imported-full-text-pdf/external/pubs/ft/wp/2012/_wp12122.pdf",
      "summary": {
        "path": "/-/media/websites/imf/imported-full-text-pdf/external/pubs/ft/wp/2012/_wp12122.pdf.md",
        "mime": "text/markdown"
      },
      "binary": {
        "path": "/-/media/websites/imf/imported-full-text-pdf/external/pubs/ft/wp/2012/_wp12122.pdf",
        "mime": "application/pdf"
      }
    }
  ],
  "alternates": {
    "markdown": "/en/publications/wp/issues/2016/12/31/appraising-credit-ratings-does-the-cap-fit-better-than-the-roc-25910/index.md",
    "json": "/en/publications/wp/issues/2016/12/31/appraising-credit-ratings-does-the-cap-fit-better-than-the-roc-25910/index.json",
    "bundleManifest": "/en/publications/wp/issues/2016/12/31/appraising-credit-ratings-does-the-cap-fit-better-than-the-roc-25910/bundle-manifest.json"
  },
  "generatedAtUtc": "2026-09-17T09:12:08.800Z"
}