## How Effectively Can Current LLMs Analyze Macrofinancial Issues?

_IMF Working Papers, February 27, 2026_

## Source details

**Canonical URL:** [How Effectively Can Current LLMs Analyze Macrofinancial Issues?](https://www.imf.org/en/publications/wp/issues/2026/02/26/how-effectively-can-current-llms-analyze-macrofinancial-issues-574289)

## Other formats

- [Markdown version](/en/publications/wp/issues/2026/02/26/how-effectively-can-current-llms-analyze-macrofinancial-issues-574289/index.md)
- [Structured JSON version](/en/publications/wp/issues/2026/02/26/how-effectively-can-current-llms-analyze-macrofinancial-issues-574289/index.json)
- [Bundle manifest](/en/publications/wp/issues/2026/02/26/how-effectively-can-current-llms-analyze-macrofinancial-issues-574289/bundle-manifest.json)

## Bibliographic details
- Authors: Paola Ganum, Tohid Atashbar
- Published: February 27, 2026
- Series: IMF Working Papers
- DOI: https://doi.org/10.5089/9798229038935.001

---

### Summary
- Empirical evaluation of current Large Language Models (LLMs) on macrofinancial coverage in IMF Article IV staff reports using human economists' assessments as a benchmark.
- Reports tested span 2016-2024.
- Evaluations cover both qualitative ratings and binary questions.
- Key headline results for 2024 across advanced GPT models:
  - Average accuracy on ratings: 71-75 percent.
  - Average exact match rate on binary questions: 76-81 percent.
- Paper explores drivers of LLM performance and discusses key limitations such as optimistic bias.

### Methodology
- Benchmark: human economists' assessments of macrofinancial coverage in IMF Article IV staff reports.
- Models tested: several GPT models (advanced GPT models highlighted in results).
- Time coverage of reports assessed: 2016-2024.
- Outcome measures:
  - Qualitative ratings (accuracy reported as a percentage range).
  - Binary questions (exact match rates reported as a percentage range).
  - Open-ended questions evaluated for deeper contextual judgment.

### Main Findings
- LLMs can meaningfully assist economists in analyzing macrofinancial issues based on measured accuracy and exact match rates.
- Performance metrics (2024, advanced GPT models):
  - Ratings accuracy: 71-75 percent.
  - Exact match rate on binary questions: 76-81 percent.
- Behavioral tendencies of models:
  - LLMs tend to assign higher ratings than human experts.
  - LLM ratings are less dispersed than human expert ratings.
- Limitations in qualitative judgment:
  - Models struggle with open-ended questions requiring deep contextual judgment.

### Drivers of Performance (as discussed)
- Choice of GPT model affects outcomes (advanced GPT models show the reported high-end performance).
- Task type matters: binary questions yield higher exact match rates than open-ended qualitative assessments.

### Key Limitations and Risks
- Optimistic bias: models systematically assign higher ratings relative to human experts.
- Reduced dispersion in assigned ratings relative to human experts may mask uncertainty or nuance.
- Challenges remain in open-ended, context-rich judgment tasks, limiting standalone use for complex macrofinancial analysis.

### Policy and Practical Implications
- LLMs are a useful assistive tool for economists but are not a substitute for expert judgment in context-dependent assessments.
- Caution advised where open-ended analysis and deep contextual judgment are required.
- Awareness of optimistic bias and compressed rating dispersion is necessary when integrating LLM outputs into macrofinancial analysis workflows.

*Paola Ganum, and Tohid Atashbar. "How Effectively Can Current LLMs Analyze Macrofinancial Issues?", IMF Working Papers 2026, 035 (2026), DOI: https://doi.org/10.5089/9798229038935.001*

---

## Content in this bundle

- **Working Paper**
  - [Working Paper (Markdown version)](/-/media/files/publications/wp/2026/english/wpiea2026035-source-pdf.pdf.md){rel="alternate" type="text/markdown"}
  - [Working Paper (PDF)](/-/media/files/publications/wp/2026/english/wpiea2026035-source-pdf.pdf){rel="external" type="application/pdf"}

---

_Source: https://www.imf.org/en/publications/wp/issues/2026/02/26/how-effectively-can-current-llms-analyze-macrofinancial-issues-574289_