## Appendix 1  Random Sample Designs

## Source details

**Canonical URL:** [Appendix 1  Random Sample Designs](https://www.imf.org/-/media/files/publications/tnm/2021/english/tnmea2021009.pdf)

## Other formats

- [Markdown version](/-/media/files/publications/tnm/2021/english/tnmea2021009.pdf.md)
- [Structured JSON version](/-/media/files/publications/tnm/2021/english/tnmea2021009.pdf.json)

---

### Overview and rationale for random sampling in PIT gap estimation
- Random sampling of taxpayers for audit provides the most robust PIT gap estimation method (Figure 1).
- Bottom-up approach: extrapolates from non-compliance identified in audits of sampled taxpayers to the population.
- Top-down approaches using macroeconomic data are generally not appropriate for PIT because administrative data used for actual tax revenue is often the primary source for personal income statistics.
- Random Audit Program (RAP) is preferred from a statistical standpoint and provides additional operational benefits that can outweigh higher costs.
- Practical note: time needed for first estimate typically has a 1–2 years lag.

### Key findings on current practice and usefulness
- An increasing number of countries measure noncompliance in PIT; 20 countries measure their personal income tax (PIT) gap (International Survey on Revenue Administration).
- In 2017, 35 OECD countries reported conducting random audits (not limited to PIT), of which 14 used the resulting data for tax gap estimation.
- Tax gap analysis using the IMF’s RA-GAP approach provides comprehensive analysis of overall revenue losses and serves as strategic context for resource allocation and policy/administrative measures.
- Bottom-up approaches (especially RAPs) produce rich, detailed results that support evidence-led risk management and improvements in compliance risk profiling.

### Additional benefits of random audits
- Random selection:
  - tests all parts of the taxpayer population for risk, improving risk profiling and compliance results;
  - provides data to counteract bias inherent in risk-based case data;
  - can be published as part of an audit plan to increase deterrence.
- RAPs provide a counterfactual to evaluate how effective risk assessment is in selecting cases for risk-based audit.

### Costs and trade-offs
- Benefits:
  - Statistically robust methodology
  - Relatively simple analysis
  - Quantifiable error
  - Evidence-based risk and behavioral insights
  - Improved risk profiles and better targeting of risk-based audits
  - Additional deterrence boosts voluntary compliance
- Costs:
  - Requires compliance staff resources
  - Time needed for first estimate (1–2 years lag)
  - Requires adjustment for unregistered taxpayers and undetected risks
  - Opportunity costs of random audits replacing risk-based audits

### Definitions and conceptual framework
- Tax gap (RA-GAP definition): difference between potential tax revenue from the tax base and actual revenue.
- Tax gap decomposition:
  - Compliance gap: losses from non-compliance.
  - Policy gap: impact of policy decisions.
- RA-GAP recommends top-down or bottom-up approaches depending on the tax; for PIT, bottom-up is usually more appropriate.
- Bottom-up estimates rely on observations of individual taxpayers’ noncompliance and are typically required where there is no independent source of data for the PIT economic base.

### Practical considerations for using random audits
- Random sampling: every member of population has a known, non-zero probability of selection; sample is theoretically representative.
- Auditors perform a comprehensive audit and record adjustments (difference between taxpayer’s and auditor’s assessment of liabilities).
- Scaling: adjustments are scaled by the ratio of sample to population to quantify the gross tax gap, with further adjustments for non-detection, non-payment, and the hidden economy.
- Random audit designs allow quantification of sampling error (precision), but:
  - RAPs are resource intensive and may be unsuitable for very small populations (large proportion would need auditing for precision).
  - Revenue return on investment can be boosted by program design choices.

### Estimating from risk-based audits — limitations and bias
- Risk-based audits target high-risk taxpayers and are subject to selection bias; direct scaling to population likely over-estimates the tax gap.
- To extrapolate from risk-based samples to the population, selection bias must be corrected, which comes at a cost of either precision or certainty.
- Risk-based audit data might be used in principle with appropriate corrections, but RAP remains the preferred statistical method for PIT compliance gap estimation.

### RA-GAP program role and focus
- RA-GAP: IMF Fiscal Affairs Department’s Revenue Administration Division capacity development service; aims to estimate tax gaps and identify underlying causes.
- Prior RA-GAP technical notes have documented approaches for VAT, CIT, and excise taxes.
- For PIT, RA-GAP focuses on measuring the compliance gap using bottom-up methodology and does not provide a standard definition for the PIT policy gap due to lack of international consensus on a standard PIT base.

### BOX 2 — Estimating the PIT Gap in Low Income Countries: role, methods, and evidence
- Primary RAP benefit in low income countries: risk profiling and improving risk-based selection for audits rather than solely gap estimation.
- Where auditing capacity is insufficient, integrate random sampling into existing audit case selection and use results to estimate total compliance gap.
- Top-down approaches may appear attractive because:
  - The informal economy and associated compliance gap is typically larger in lower income countries.
  - National accounts in low income countries are less likely to rely on data from tax records.
- Limitations of top-down approaches:
  - Depend on how well national accounts capture unreported income and its distribution.
  - Household income surveys suffer from non-response and under-reporting; require calibration to national accounts or tax collections.
- Broad top-down estimates can be useful as an initial strategic risk indicator where RAPs are absent, especially when expected PIT gap is relatively high (example logic: easier to tolerate ± five percentage points margin for a 30 percent gap than for a 5 percent gap).

Methods to model the tax gap from risk-based audits
- Post-stratification:
  - Divide audited and non-audited taxpayers into strata; estimate stratum tax gap by scaling stratum audit results to stratum population size.
- Heckman two-stage procedure:
  - Step 1: estimate probability a case is selected for audit.
  - Step 2: model audit outcome using explanatory variables plus a regressor controlling for selection bias derived from Step 1.
- Extreme value method:
  - Assumes Pareto distribution where a small number of ‘extreme’ cases account for most revenue found; extrapolates to residual non-compliance.
- Caveats:
  - Each method requires high levels of data or strong assumptions.
  - Scale of error is largely unquantifiable.
  - Apart from Heckman, increases in hit-rate may be misinterpreted as increases in non-compliance rather than improved RA capability.

Indirect estimation approaches
- Income-expenditure approach:
  - Uses consumption (generally food expenditure) versus reported incomes to model under-reporting.
  - Laborious, requires multiple assumptions, yields non-comprehensive PIT gap estimates, may illuminate non-filer components.
- MIMIC and similar shadow-economy approaches:
  - Not recommended by RA-GAP for revenue administrations.
  - Estimate shadow economy from observable indicators; not synonymous with tax non-compliance and rest on strong assumptions.

Case study (Italy) — income-expenditure / bottom-up approach findings
- Method combined Household Budget Survey consumption data for 2013 matched with income tax declarations 2009–2016 and administrative data on property and financial wealth.
- Key quantitative findings:
  - Observed share of unreported income by self-employed taxpayers was more than 40 percent of their income on average.
  - Resulting gap for self-employment for the year 2018 was about €34 billion, comprised of about €26 billion of PIT gap and €8 billion of social contributions paid by the self-employed.
  - These estimates are in line with top-down PIT estimates available for the same year, about 67 percent of gross tax liabilities.
  - More than 80 percent of missing tax revenue is attributable to the taxpayers at the top of the income distribution.
  - Calculation used EUROMOD with policy rules for 2018 and income values from 2013 tax data uprated to 2018.

### Scope, adjustments, precision, and interpretation
- Scope and coverage:
  - RA-GAP defines the compliance gap as difference between current potential tax under existing policies and actual tax collected.
  - Final PIT gap estimates may be partial due to data limits and cost-effectiveness.
  - Time period should be specified (typically the fiscal year) and align with RA reporting.
  - Audits generally cover registered taxpayers or filed returns; unregistered or non-filers require additional methods.
- Adjustment for non-detection bias:
  - Audits should be adjusted for non-detection bias; multipliers can be developed but add unquantifiable error.
  - Without a multiplier, report the gap as a lower-bound or the detectable PIT gap.
- Net vs gross gap:
  - RA-GAP estimates the compliance gap net of voluntary adjustments and compliance interventions.
  - RAs may calculate gross gap first and then deduct late payments to obtain net gap.
- Precision and error:
  - All tax gap estimates are subject to error. Sampling error is measurable; non-sampling error (bias, analyst assumptions) often dominates and is generally not measurable.
  - Sensitivity analysis is useful; formal margins of error or statistical significance tests may be misleading.
  - Consistent methods/data over time enable robust trend analysis despite bias.

### Unintentional taxpayer errors (Box 4 summary)
- Unintentional errors are generally found to increase the observed compliance gap (not symmetric).
- Possible reasons:
  - Taxpayers take more care not to overpay than to underpay.
  - Auditors may treat unintentional errors differently from deliberate evasion, potentially biasing measured impact upward.
- Implication: non-sampling error and biases argue against over-reliance on formal precision measures.

### BOX 5 — Tax Gap Estimates as Published Statistics: governance and RAP operations
- Official status, governance, and revisions:
  - Published statistical governance frameworks and official status improve trust and credibility.
  - Clear revision policies are beneficial (example cited: HM Revenue and Customs revision policy application).
- RAP process overview (procedural steps):
  - Explore population data, scope for stratification, probable precision ranges.
  - Agree sample size balancing precision vs resources.
  - Prepare sample frame and segment by strata; select using random sampling.
  - Send sample to auditor teams for comprehensive audit.
  - Monitor and quality assure audits and data recording.
  - Extract program data for estimation.
- Recommended operational choices:
  - Begin with a pilot study to learn distribution of non-compliance and test procedures.
  - If insufficient sample in one year, consider multi-year pooling with trade-offs.
  - Stratify sample by risk or turnover/income to improve precision and yield.
  - Integrate random selection into general audit program if resources constrained.

A. Sample size and design
- Every taxpayer must have a chance of selection for validity.
- Sample size determined by acceptable precision and resources; margins of error set out for different sample sizes (see Appendix C in source).
- Stratification recommendations:
  - Use risk indicators where available; even high/low segmentation and over-selecting high-risk strata increases precision and yield.
  - Without risk indicators, segment by turnover or income and over-select highest earning segment.
  - Beware bias if registry excludes registered-but-not-yet-filed or non-filers.

B. Selecting the sample
- Sample frame should be comprehensive, up-to-date taxpayer registry.
- Advantages of selecting a larger initial sample: spare units for replacement, flexibility.
- Systematic sampling (every Nth filer) possible if no population list but has disadvantages (resource uncertainty, stratification difficulty, potential bias).

C. Data requirements and recording
- Record deselected cases and not-taken-up audits to adjust sample ratio and population size.
- Record cases already subject to risk-based audit or ineligible to avoid bias.
- Random audits focus on thorough investigation and high-quality data reporting, not hit-rate.
- Data to capture:
  - Positive and negative reassessments separately.
  - Disaggregate non-compliance by period.
  - Codes for non-compliance types and behaviors.
  - Representation by agent/accountant, business sector, specific income line items.
  - Standardized reporting scale to distinguish intentional vs unintentional behavior.
  - Post-audit coding by analysts if resources allow.
- Additional data for net PIT gap:
  - Enforced payments and debt data (aggregate acceptable).
  - Disaggregation by population, subgroups, and time period.
  - Debt generally added where considered unrecoverable.

D. Scaling results and calculation model
- Summary calculation workflow:
  - Calculate average audit adjustment (forecast open cases, treat outliers).
  - Total population = Ratio of sample in scope × Registered taxpayer population.
  - Gross gap = Average audit adjustment × Total population.
  - Net tax gap = Gross gap × Multiplier + Non-payment (+ hidden economy, where relevant) − Enforced payments.
  - Tax gap as proportion = Net gap / Total theoretical liabilities.
- PIT gap formula from a Random Audit Program (for a given period):
  - Net tax gap = ā –  × m̄ – × N_t + d + h − e
    - where:
      - Net tax gap is defined as gross non-compliance after enforced payments
      - ā = average audit adjustment
      - m̄ = average (or aggregate) non-detection multiplier (if used)
      - N_t = number of filing taxpayers
      - d = non-payment at time of calculation
      - h = estimate for hidden economy (non-filers), if available
      - e = forecast enforced payments (if used)
- Stratified sample expression (simplified notation from source):
  - net tax gap = Σ_s ( (N_S / n_S) × Σ_i (a_i × m_i) ) + d + h − e
    - where:
      - N_S = number of filing taxpayers in stratum S
      - n_S = number of audits sampled in stratum S
      - a_i = audit adjustment for taxpayer i
      - m_i = multiplier for taxpayer i
- Adjustments and caveats:
  - Adjust population number for deselected cases or incomplete frame.
  - Forecast open audits to avoid downward bias; open cases often larger.
  - Treatment of outliers must be consistent over time for trend analysis.
  - Additional adjustments may be needed for RAP coverage gaps (e.g., omitted low-risk segments, complex frauds).

Adjustments, imputation, and open cases
- Forecast or impute open audits using methods ranging from basic replacement to predictive or machine learning (e.g., ‘nearest neighbor’).
- Consistent treatment of outliers and open cases supports robust trend analysis.

Final remarks: strengths, limitations, best practice
- RAP is RA-GAP preferred approach for most countries to estimate PIT compliance gap.
- Benefits:
  - Robust strategic performance indicator.
  - Detailed empirical evidence to test risk profiles and improve compliance risk management.
  - Operational improvements can more than offset direct and opportunity costs.
- Limitations of alternatives:
  - Top-down usually inappropriate without individual income data.
  - Bottom-up from risk-based selections requires advanced analytics and strong assumptions.
  - ‘Big data’ and advanced analytics experimental and better suited to risk identification than unbiased gap estimation.
- Good practice recommendations:
  - Use stratified sampling based on risk profiles to increase yield and precision.
  - Apply systemic quality controls and rigorous data capture.
  - Use appropriate multipliers and adjustments cost-effectively.
  - Start with pilot RAPs to refine design.
  - Report PIT gap transparently and realistically; limited coverage (e.g., excluding non-filers) usually biases estimates downward unless adjusted.

### BOX 6 — Weighting data, sample design types, and nondetection multipliers
- Weighting data:
  - When cases have unequal inclusion probability, apply weights so sample results represent the population.
  - Average gap per stratum multiplied by proportion of population stratum represents.
- C. Single or multistage cluster samples:
  - Cluster design: divide population into clusters selected with equal probability, then sample within clusters.
  - Adopted when population list unavailable or to reduce sampling costs (e.g., sample fewer geographical areas).
  - Downsides: decreases precision; not recommended. Increased uncertainty presented as a ‘design effect’.
- D. Other sample designs:
  - RAP should use probabilistic sampling. If sample scope is limited to specific taxes or segments, estimate represents only that sampled population.

APPENDIX 2 — NONDETECTION MULTIPLIERS: overview and methods
- Purpose: adjust auditors’ assessments to account for non-detected non-compliance.
- Key methods:
  - a. Detection controlled estimation
    - Econometric approach scaling each auditor’s detections up to the “best” auditor.
    - Data requirements: each auditor must have completed at least 15 cases covering similar line items.
    - Requires auditor characteristics and non-compliance type variables.
    - Only scales up to observed capability of highest-performing auditor.
  - b. Comparing auditor detected liabilities to third party information
    - Conduct audits without third-party info, then compare retrospectively to third-party data.
    - Not meaningful if auditors already access that data; won’t capture items with no third-party records (e.g., cash income).
  - c. Secondary case review by expert auditors
    - Expert reviewers check cases to identify non-detection; limited by reviewers’ abilities.
  - d. Expert judgement
    - Panel of experienced auditors estimate undetected tax, possibly via Delphi technique; results contingent on panel composition.
  - e. Adopting multipliers calculated by others
    - Use external multipliers or international mid-points; may be unrepresentative but permit estimates where resources lack.
- General notes:
  - Appropriate multipliers reduce downward bias but add unknown error.
  - Validation of final multiplier generally almost impossible.
  - If no multiplier used, report gap as lower bound/detectable PIT gap.

Table 2. Sample Sizes by Margin of Error for Different Non-Compliance Rates
- Heading: Margin of error in percentage points / Rate of non-compliance
- Rows (sample sizes rounded to nearest five; assumes a large population size):
  - ‘Worst Case’ 50%: 95 38 51 06 5
  - High 40%: 115 37 01 02 5
  - Medium 25%: 70 29 08 00
  - Low 10%: 35 14 03 80
- Note: Sample sizes rounded to nearest five. Assumes a large population size.

APPENDIX 3 — sample size guidance and confidence concepts
- Confidence interval: range with known probability (generally 95 percent) to contain true population value.
- Margin of error: half the width of confidence interval in percentage points; table describes margin around rate of non-compliance, not value of the gap.
- Pilot study recommended to observe hit-rate and variance for sample-size decisions.
- In absence of pilot, assume worst-case incidence (50 percent) to estimate sample size.

APPENDIX 4 — project management checklist for a RAP (selected actions)
- Planning / Governance:
  - Establish working group of analysts, auditors, specialists.
  - Establish senior oversight (steering group).
- Design:
  - Analyze sample/sub-sample options, accuracy, costs; consider stratification and whether RAP aims to improve deterrence (requires larger sample).
  - Agree resources, selection/management/reporting requirements, templates, timeline, milestones.
- Scope and coverage:
  - Agree risks not reliably captured by RAP and taxpayers not cost-effective to sample.
- Implementation / sample selection:
  - Clean and up-to-date taxpayer registry as sample frame; include demographic/risk data for stratification.
- Audits and analysis:
  - Provide auditor support, monitor progress, quality assure data.
  - Extract data on pre-agreed date; check duplicates and open cases; treat outliers consistently.
  - Forecast open cases if required; apply non-detection multipliers and adjust for unaudited cases.
- Reporting:
  - Publish gap estimates and analysis per pre-agreed plan; report PIT gap as percent of potential collections with context and caveats.

Glossary excerpts (selected definitions)
- Bias: tendency to over- or under-estimate a population parameter.
- Bottom-up estimation: begins with taxpayer-level data and scales up non-compliance to population level.
- Confidence (statistical): typically set at 95%; probability the true value lies within confidence interval.
- Margin of error: half the width of confidence interval in percentage points.
- Probability or random sampling: each population unit has known, non-zero chance of inclusion; only general method providing measure of precision.
- Weight: an inflation factor applied to raw data.

*Source: tnmea2021009 - Appendix 1, BOX 2, BOX 5, BOX 6, and accompanying appendices (Technical Notes and Manuals 21/02 | 2021).*

### Appendix 1  Random Sample Designs                                                                             25

### Appendix 1  Random Sample Designs                                                                             25

### Overview and Rationale for Random Sampling in PIT Gap Estimation
- Random sampling of taxpayers for audit provides the most robust PIT gap estimation method (Figure 1).
- Bottom-up approach: extrapolates from non-compliance identified in audits of sampled taxpayers to the population.
- Top-down approaches using macroeconomic data are generally not appropriate for PIT because administrative data used for actual tax revenue is often the primary source for personal income statistics.
- Random Audit Program (RAP) is preferred from a statistical standpoint and provides additional operational benefits that can outweigh higher costs.

### Key Findings on Current Practice and Usefulness
- An increasing number of countries measure noncompliance in PIT; according to the International Survey on Revenue Administration, 20 countries measure their personal income tax (PIT) gap.
- In 2017, 35 OECD countries reported conducting random audits (not limited to PIT), of which 14 used the resulting data for tax gap estimation.
- Tax gap analysis using the IMF’s RA-GAP approach provides comprehensive analysis of overall revenue losses and serves as strategic context for resource allocation and policy/administrative measures.
- Bottom-up approaches (especially RAPs) produce rich, detailed results that support evidence-led risk management and improvements in compliance risk profiling.

### Additional Benefits of Random Audits
- Random selection:
  - tests all parts of the taxpayer population for risk, improving risk profiling and compliance results;
  - provides data to counteract bias inherent in risk-based case data;
  - can be published as part of an audit plan to increase deterrence.
- RAPs provide a counterfactual to evaluate how effective risk assessment is in selecting cases for risk-based audit.

### Costs and Trade-offs (from Table 1)
- Benefits:
  - Statistically robust methodology
  - Relatively simple analysis
  - Quantifiable error
  - Evidence-based risk and behavioral insights
  - Improved risk profiles and better targeting of risk-based audits
  - Additional deterrence boosts voluntary compliance
- Costs:
  - Requires compliance staff resources
  - Time needed for first estimate (1–2 years lag)
  - Requires adjustment for unregistered taxpayers and undetected risks
  - Opportunity costs of random audits replacing risk-based audits

### Definitions and Conceptual Framework
- Tax gap (RA-GAP definition): difference between potential tax revenue from the tax base and actual revenue.
- Tax gap decomposition:
  - Compliance gap: losses from non-compliance.
  - Policy gap: impact of policy decisions.
- RA-GAP recommends top-down or bottom-up approaches depending on the tax; for PIT, bottom-up is usually more appropriate.
- Bottom-up estimates rely on observations of individual taxpayers’ noncompliance and are typically required where there is no independent source of data for the PIT economic base.

### Practical Considerations for Using Random Audits
- Random sampling: every member of population has a known, non-zero probability of selection; sample is theoretically representative.
- Auditors perform a comprehensive audit and record adjustments (difference between taxpayer’s and auditor’s assessment of liabilities).
- Scaling: adjustments are scaled by the ratio of sample to population to quantify the gross tax gap, with further adjustments for non-detection, non-payment, and the hidden economy.
- Random audit designs allow quantification of sampling error (precision), but:
  - RAPs are resource intensive and may be unsuitable for very small populations (large proportion would need auditing for precision).
  - Revenue return on investment can be boosted by program design choices (discussed in other sections/appendices).

### Estimating from Risk-Based Audits — Limitations and Bias
- Risk-based audits target high-risk taxpayers and are subject to selection bias; direct scaling to population likely over-estimates the tax gap.
- To extrapolate from risk-based samples to the population, selection bias must be corrected, which comes at a cost of either precision or certainty.
- Risk-based audit data might be used in principle with appropriate corrections, but RAP remains the preferred statistical method for PIT compliance gap estimation.

### RA-GAP Program Role and Focus
- RA-GAP: IMF Fiscal Affairs Department’s Revenue Administration Divisions capacity development service; aims to estimate tax gaps and identify underlying causes.
- Prior RA-GAP technical notes have documented approaches for VAT, CIT, and excise taxes.
- For PIT, RA-GAP focuses on measuring the compliance gap using bottom-up methodology and does not provide a standard definition for the PIT policy gap due to lack of international consensus on a standard PIT base.

*Source: Appendix 1, "Random Sample Designs", Technical Notes and Manuals 21/02 | 2021*

### BOX 2. Estimating the PIT Gap in Low Income Countries

### BOX 2. Estimating the PIT Gap in Low Income Countries

### Role and benefits of a Random Audit Program (RAP) in low income countries
- The main benefit of RAPs is their use in risk profiling, to monitor and improve risk-based selection for audits, rather than solely as a way to estimate the PIT gap.
- RAPs are a recommended component of evidence-based audit programs.
- Where auditing capacity is insufficient for a separate RAP, random sampling can be integrated into case selection for the existing audit program: some audit cases are selected randomly from the total taxpayer population and their results used to estimate the total compliance gap.
- The costs of an RAP can be off-putting and difficult to justify solely for gap estimation in low income countries; the improvements to compliance risk management and risk-based audits better justify the costs.

### Top-down approaches and their limitations in low income countries
- Top-down approaches to PIT gap estimation may appear attractive in low income countries because:
  - The informal economy, and associated compliance gap, is typically larger in lower income countries.
  - National accounts in low income countries are less likely to rely on data from tax records.
- Accuracy of top-down estimates depends critically on:
  - How well national accounts capture unreported income and its distribution among taxpayers.
  - The detail of other factors potentially affecting individual taxpayers’ liability.
- Household income surveys can offer a top-down approach but:
  - Survey totals are often distorted by non-response (particularly for households at the high and very low end of the income distribution) and under-reporting by respondents.
  - They need to be calibrated to national accounts or other aggregates (e.g., tax collections) to adjust for such biases, exposing them to similar limitations in gap estimation.

### Usefulness of broad top-down estimates when RAPs are absent
- In the absence of RAP-based estimates, an initial top-down assessment of the PIT compliance gap can be valuable.
- Where the expected PIT gap is relatively high, a broad estimate provides a useful strategic risk indicator and can inform the design of a RAP.
- The levels of uncertainty associated with national accounts aggregates are less important for larger gaps (example given: easier to tolerate a margin of error of plus or minus five percentage points for a compliance gap of 30 percent of liabilities than for a 5 percent gap).

### Methods to model the tax gap from risk-based audits
- Post-stratification:
  - Taxpayers (audited and non-audited) are divided into strata using variables relevant to audit selection; the tax gap for each stratum is estimated by scaling stratum audit results to total population size for each stratum.
- Heckman two-stage procedure:
  - Step 1 estimates the probability that a case is included in the sample (selected for audit).
  - Step 2 models the outcome of an audit (determinants of yield) using explanatory variables and a regressor that controls for selection bias, calculated from parameters estimated in the first step.
- Extreme value method:
  - Assumes majority of non-compliance is identified by existing audits of higher-risk taxpayers and extrapolates to residual non-compliance in lower-risk, unaudited taxpayers.
  - Applicable where data follow a Pareto distribution and a small number of ‘extreme’ cases account for the majority of total revenue found in risk-based audits.
- Caveats on these approaches:
  - Each requires either high levels of data on taxpayer characteristics and selection criteria, or strong assumptions.
  - Scale of error in estimates is largely unquantifiable.
  - Apart from the Heckman two-stage approach, an increase in hit-rate will be interpreted as an increase in non-compliance in the population; trend analysis may be misleading if improved hit-rate reflects improved RA capability.

### Indirect estimation approaches
- Income-expenditure approach:
  - Models under-reporting of income through observation of the relationship between consumption (generally food expenditure) and reported incomes.
  - Preparing and manipulating the data can be very laborious; estimating the gap requires multiple assumptions and yields non-comprehensive PIT gap estimates but can illuminate components of the gap (e.g., tax not paid by non-filers).
  - Given high analytical requirements, these approaches may be best undertaken by external experts (e.g., academics) commissioned by the RA.
- MIMIC and similar shadow-economy approaches:
  - Not recommended for revenue administrations by RA-GAP.
  - MIMIC produces an estimate of the size of the shadow economy based on observable indicators assumed to measure hidden economic activity; shadow economy activity is not synonymous with tax non-compliance.
  - These approaches rest on strong assumptions, have been criticized for measuring compliance losses, and their results risk misreporting or misuse; extreme caution advised.

### Case Study (Italy) — income-expenditure / bottom-up approach findings
- The Italian Ministry of Economy and Finance (2019) produced bottom-up PIT gap estimates to describe self-employed income under-reporting and distributional analyses.
- Method features:
  - Extension of the Pissarides-Weber approach allowing detailed breakdowns by household categories (e.g., single vs married, location, presence of children).
  - Household consumption data from the Household Budget Survey for 2013 were matched with income tax declarations for 2009–2016 via anonymized personal IDs; data on property and financial wealth from administrative sources were also used.
- Key quantitative findings:
  - Observed share of unreported income by self-employed taxpayers was more than 40 percent of their income on average.
  - The resulting gap for self-employment for the year 2018 was about €34 billion, comprised of about €26 billion of PIT gap and €8 billion of social contributions paid by the self-employed.
  - These estimates are in line with top-down PIT estimates available for the same year, about 67 percent of gross tax liabilities.
  - More than 80 percent of missing tax revenue is attributable to the taxpayers at the top of the income distribution.
  - The calculation used EUROMOD with policy rules for 2018 and income values from 2013 tax data uprated to 2018.

### Scope of the PIT gap estimates
- RA-GAP defines the compliance gap as the difference between current potential tax under existing policies and actual tax collected.
- Final scope of a PIT gap estimate may be partial due to:
  - Practical limits to data collectible from a random audit program or otherwise available.
  - Cost-effectiveness considerations.
- Time period:
  - The tax gap should be for a specified time period, typically the fiscal year, and align with the period the RA uses for reporting performance.
- Coverage limitations:
  - Audits (random or risk-based) generally cover registered taxpayers or filed returns; liabilities from unregistered or non-filing taxpayers will not be directly covered.
  - Additional methods (research surveys into hidden economy activities or data-matching to third-party information) may be needed to account for PIT due from non-filers.
- Adjustment for non-detection bias:
  - Results of audits should be adjusted for non-detection bias, ideally accounting for differences in auditor capacity and scope of audits.
  - Multipliers can be developed to allow for non-detected risk to reach a theoretically comprehensive PIT compliance gap, but this is methodologically challenging and adds unquantifiable error.
  - In the absence of a multiplier, the gap can be described as a lower-bound estimate or the detectable PIT gap.
- Supplementary estimates:
  - Low frequency but high value risks may be unlikely to be captured reliably by random audit programs; supplementary estimates or adjustments can be developed to cover these components.
- Net vs gross gap:
  - Using the RA-GAP approach, the compliance gap is estimated net of revenue from voluntary adjustments and compliance interventions (i.e., the amount of tax that will never be recovered).
  - RAs may also calculate the gross gap (including PIT not paid on time) and then deduct late payments to calculate the net gap.
- Scope of behaviors included:
  - PIT compliance gap estimates from audits include under-payments due to unintended taxpayer error and offsetting effects of over-payments; they may include amounts attributable to avoidance or aggressive tax planning that the RA seeks to challenge.
  - Large and complex avoidance schemes are likely too infrequent or difficult to identify to be captured reliably in random audit programs.

### Precision, error, and interpretation
- All tax gap estimates are subject to error; PIT gap results should be interpreted as broad estimates.
- If estimation approach and data are consistent over time, the scale and direction of error should be roughly constant, enabling robust trend analysis.
- Mathematical measures of precision generally capture sampling error, whereas tax gap estimates are often more affected by non-sampling error (bias in the final sample, analysts’ assumptions), which is generally not possible to measure.
- Sensitivity analysis of assumptions can provide insights into potential impacts on error; presenting formal margins of error or statistical significance tests may be misleading.
- Trend analysis:
  - Good tax gap analysis provides robust estimates of trend over time, enabling detection of changes in taxpayer behavior via consistent comparisons.

### Unintentional taxpayer errors (Box 4 summary)
- Intuition might suggest genuine errors are unbiased (over- and under-declaration balance), but in practice unintentional errors are generally found to increase the observed compliance gap.
- Possible reasons:
  - Taxpayers may take more care not to overpay than to underpay.
  - Finding an unintentional error generates less paperwork and a quicker audit than finding deliberate evasion, which may create a perverse incentive for auditors to under-report taxpayers’ motivation and bias observed impact of unintentional errors upwards.
- Implication:
  - Non-sampling error and potential biases argue against over-reliance on formal precision measures in final reports.

*Source: BOX 2. Estimating the PIT Gap in Low Income Countries (Technical Notes and Manuals 21/02 | 2021).*

### BOX 5. Tax Gap Estimates as Published Statistics

### BOX 5. Tax Gap Estimates as Published Statistics

### Official status, governance, and revisions
- Following published statistical governance frameworks and gaining an official status can enhance trust and credibility of published tax gap estimates.
- Public scrutiny from media and elected representatives increases when revenue agencies (RAs) release estimates; official status helps manage that scrutiny.
- Having a clear policy on how and when statistics are revised is beneficial. Example cited: HM Revenue and Customs publishes its policy on revisions to statistics; this policy applies to tax gap estimates as they have Official Statistics status and aligns with guidance issued by the UK Statistics Authority.

### Random Audit Program (RAP) process overview
- Key procedural steps (summary of Figure 3 and text):
  - Analysts explore population data, scope for stratification, and probable ranges of precision.
  - Agree sample size, balancing precision of estimates against resources.
  - Prepare sample frame (population list), checking it represents population of interest.
  - Segment sample frame by strata and select using a random sampling method.
  - Sample sent to auditor teams for comprehensive audit.
  - Regular monitoring and quality assurance of audits and data recording.
  - Extract data from the program for estimation.
- Recommended operational choices:
  - Begin with a pilot study of a smaller size sample to understand distribution of non-compliance, test procedures, and better estimate full-program requirements.
  - If sufficient sample size cannot be resourced in a single year, consider spreading the program over multiple years and pooling results; trade-offs include delayed full estimate production and broader time-period estimates that may be disadvantaged if tax policy, operations, or the economy change part-way through the period.
  - Stratify the sample to gain analytical and revenue benefits; stratification by risk or by turnover/income is recommended to improve precision and yield.
  - Integrate some random case selection into general audit program where resources are constrained.

### A. Sample size and design
- Every taxpayer in the population of interest must have a chance of selection for validity.
- Sample size is calculated based on acceptable precision and available resources; margins of error should be set out for different sample sizes (see Appendix C in source).
- Balance precision for whole-population and subgroup estimates against audit revenue and resource costs (e.g., staff costs).
- Stratification recommendations:
  - Use risk indicators for stratification where available; even simple high/low risk segmentation and over-selecting high-risk strata can increase precision and revenue yield.
  - Without risk indicators, segment by turnover or income and over-select highest earning segment to increase precision.
  - Beware potential bias if taxpayer registry joined only to returns data excludes registered but not-yet-filed or non-filers.

### B. Selecting the sample
- The sample frame should be the comprehensive, up-to-date list of taxpayers in the RA’s registry.
- Advantages of initially selecting a larger sample than required:
  - Spare units for replacing deselected cases.
  - Flexibility for project expansion.
- In absence of a population list, systematic sampling (every Nth filer from a random start) is possible but has disadvantages:
  - Uncertainty about resource requirements.
  - Difficulty with stratification.
  - Potential bias if program stops before the entire population has filed or if filing order correlates with compliance.
- Select samples quickly and progress to fieldwork to avoid legal limitations and bias from case loss (e.g., late filers associated with non-compliance or selected cases becoming risk-based audits).

### C. Data requirements and recording
- Record cases not taken up for audit and number deselected from sample to determine relevant sample ratio and adjust population size.
- Record cases already subject to risk-based audit or not eligible for audit to enable substitution or imputation and avoid bias.
- Random audits focus on thorough investigation and high-quality data reporting rather than hit-rate and revenue.
- Guidance and standardized reporting are important; data should capture:
  - Positive and negative reassessments recorded separately.
  - Disaggregation of non-compliance of the period of interest from other time periods.
  - Codes for specific non-compliance types and behaviors.
  - Whether the taxpayer is represented by an agent or accountant.
  - Business sector.
  - Specific types of income or line items where non-compliance was identified.
- Record type of behavior leading to reassessment; use a standardized reporting scale to distinguish between intentional (e.g., evasion) and unintentional (e.g., error) non-compliance, aligned where possible to legal definitions.
- Where resources allow, consider post-audit data coding by analysts or specialist teams to improve reliability of behavior classification.
- Additional data required to estimate the net PIT gap:
  - Enforced payments from compliance activity and debt data from RA records (payments and debt can generally be taken at aggregate rather than taxpayer level).
  - Disaggregate data to the level of target population, subgroups, and time period under consideration.
  - Debt generally added to the tax gap estimate (as ‘non-payment’) where considered unrecoverable.

### D. Scaling results to the total taxpayer population and calculation model
- Summary tax gap calculation (Figure 4 workflow):
  - Calculate average audit adjustment (forecast open cases, treat outliers).
  - Total population = Ratio of sample in scope × Registered taxpayer population.
  - Gross gap = Average audit adjustment × Total population.
  - Net tax gap = Gross gap × Multiplier + Non-payment (+ hidden economy, where relevant) − Enforced payments.
  - Tax gap as proportion = Net gap / Total theoretical liabilities.
- PIT gap formula from a Random Audit Program (for a given period):
  - Net tax gap = ā –  × m̄ – × N_t + d + h − e
    - where:
      - Net tax gap is defined as gross non-compliance after enforced payments
      - ā = average audit adjustment
      - m̄ = average (or aggregate) non-detection multiplier (if used)
      - N_t = number of filing taxpayers
      - d = non-payment at time of calculation
      - h = estimate for hidden economy (non-filers), if available
      - e = forecast enforced payments (if used)
- Stratified sample expression (simplified notation from source):
  - net tax gap = Σ_s ( (N_S / n_S) × Σ_i (a_i × m_i) ) + d + h − e
    - where:
      - N_S = number of filing taxpayers in stratum S
      - n_S = number of audits sampled in stratum S
      - a_i = audit adjustment for taxpayer i
      - m_i = multiplier for taxpayer i
- Adjustments and caveats:
  - Adjust total population number based on ratio of deselected cases or estimate population sizes from other sources if sample frame incomplete.
  - Allowance may be required for audits still open at estimation time; forecast open audit values to avoid downward bias.
  - Open cases should generally be forecast based on auditor’s latest estimate or imputed using statistical methods; initial gap estimates can be re-estimated based on final data where necessary.
  - Outlier cases can be retained unadjusted, deleted, capped, or imputed; consistency in treatment over time is important for trend analysis.
  - Note: additional adjustments may be appropriate for gaps in RAP coverage (e.g., low-risk segments omitted, complex frauds).

### Adjustments, imputation, and treatment of open cases
- Open audits often larger and harder to calculate than routine cases and can disproportionately impact final estimates.
- Forecasting open audits or using imputation methods (basic replacement to sophisticated predictive or machine learning techniques, e.g., ‘nearest neighbor’) are options for substituting missing values.
- Consistent treatment of outliers and open cases supports robust trend analysis.

### Final remarks: strengths, limitations, and best practice
- A well-designed and executed RAP is the RA-GAP preferred approach to estimating the PIT compliance gap for most countries.
- Benefits:
  - Provides a robust strategic performance indicator.
  - Supplies detailed empirical evidence to test risk profiles and improve case selection and compliance risk management.
  - Operational improvements from RAPs can be expected to more than offset direct and opportunity costs.
- Limitations of alternative approaches:
  - Top-down approaches usually inappropriate without independent individual income data.
  - Bottom-up methods using risk-based audit selections require advanced analytics and high-quality administrative data and still rely on strong assumptions.
  - ‘Big data’ and advanced analytics (e.g., machine learning, large-scale data matching) are often experimental and better suited to risk identification and profiling than unbiased gap estimation; may be beyond some administrations’ capabilities.
- Good practice recommendations:
  - Use stratified sampling based on existing risk profiles to increase audit yield and precision for a given sample size.
  - Apply systemic quality controls and manage data capture rigorously.
  - Use appropriate multipliers and adjustments to minimize distortions cost-effectively.
  - Start with smaller pilot RAPs to refine design and implementation for larger exercises.
  - Report PIT gap estimates transparently and realistically, recognizing limits and strengths; note that limited coverage (e.g., excluding non-filers) usually biases estimates downward unless adjustments are made.
- Conclusion: A comprehensively analyzed and reported RAP provides an unbiased, detailed analysis of taxpayer compliance behavior that aids compliance risk management, increases revenue yield, and improves transparency and governance.

*Source: BOX 5. Tax Gap Estimates as Published Statistics, TNM EA 21/02 (2021).*

### BOX 6. Weighting Data

### BOX 6. Weighting Data

### Weighting data: purpose and application
- When cases are selected with an unequal chance of inclusion (for example, stratified samples where a particular stratum is over-selected), the sample is not representative of the population and adjustments must be made at the point of analysis.
- A weight is a variable added to sample data and is used to adjust how much influence a specific case or stratum of cases has in analysis so that final results are representative of the population.
- In tax gap analysis, the average value of the gap for each stratum can be multiplied by the proportion of the population that the stratum represents.

### C. Single or Multistage Cluster Samples
- Cluster sample design: population divided into clusters selected with equal selection probability; then sample units from within selected clusters are selected with equal selection probability.
- Typical adoption reasons:
  - Practical when a whole population list is unavailable but groupings are known.
  - Way to reduce sampling costs (for example, by sampling in just a few geographical areas).
- Downsides and cautions:
  - A cluster sample design decreases the precision of estimates and is not recommended.
  - The increased uncertainty is presented in results as a ‘design effect’, a factor that increases standard errors.
  - RAs will typically have a full population list and adequate coverage to access taxpayers in all regions.
  - Where budgets are restricted, a cluster sample design could limit the number of regions in the RAP, but compliance rates can vary by geographical location, so good prior knowledge of the population is required to avoid selection bias.

### D. Other Sample Designs
- The RAP should employ a probabilistic sampling strategy to produce sound tax gap estimates.
- If only specific taxes, taxpayer segments, or risks are in scope of the RAP rather than the whole population (for example, international or ‘offshore’ risks, or certain sectors), the gap estimate will only represent the population from which the random sample was taken.

### APPENDIX 2. NONDETECTION MULTIPLIERS — overview and methods
- Purpose: adjust auditors’ assessments of undeclared liabilities to take account of non-compliance that auditors were unable to detect.
- Key methods for deriving a non-detection multiplier:
  - a. Detection controlled estimation
    - Econometric approach addressing varying abilities of individual auditors to detect liabilities.
    - Derives an uplift to RAP results by scaling up what was detected by each auditor to what the “best” auditor would have found.
    - Can provide a multiplier specific to each line item and for each auditor based on detection rates.
    - Data requirements are high: each auditor must have completed at least 15 cases covering broadly the same line items.
    - Requires characteristics of auditors (e.g., grade and experience) and specific types of non-compliance as explanatory variables.
    - Only scales up to the observed capability of the highest performing auditor, which still may not equate with full detection.
  - b. Comparing auditor detected liabilities to third party information
    - Audits must first be conducted without access to third party information then compared retrospectively to third party data on actual income sources.
    - Not meaningful if auditors already have access to all or most available data.
    - Will not indicate magnitude of non-detection for items with no additional data to cross reference (e.g., income received in cash).
  - c. Secondary case review by expert auditors
    - Cases are reviewed by a separate group of experts (for example, most experienced auditors) to identify non-detection.
    - If cases are passed while still open, additional checks can be conducted and differences compared.
    - Requires additional resource investment and is limited by the abilities of the expert reviewers.
  - d. Expert judgement
    - A panel of experts (for example, experienced auditors) estimate how much tax goes undetected in an audit.
    - Can be structured via detailed scenarios and use a Delphi technique whereby respondents provide independent assessments through rounds and responses are pooled to reach consensus.
    - Drawback: unlikely to be a way to validate the final outcome; results contingent on the specific experts on the panel.
  - e. Adopting multipliers calculated by others
    - Some RAs adopt multipliers calculated by other RAs or experts, or the mid-point of international ranges.
    - Limitation: external multipliers may be unrepresentative of country-specific PIT and audit regimes and the specific population under consideration.
    - Advantage: permits an RA with limited resources to estimate a theoretically comprehensive tax gap in the absence of an alternative.
- General notes:
  - Appropriate multipliers can reduce downward bias from non-detection but add an unknown degree of error to estimates.
  - It is generally almost impossible to validate the accuracy of the final multiplier.
  - If an RA chooses not to employ a non-detection multiplier, the gap can be reported as subject to this limitation and therefore interpreted as a lower bound or conservative estimate, or defined as the detectable PIT gap.

### Table 2. Sample Sizes by Margin of Error for Different Non-Compliance Rates
- Heading: Margin of error in percentage points / Rate of non-compliance
- Rows (sample sizes rounded to nearest five; assumes a large population size):
  - ‘Worst Case’ 50%: 95 38 51 06 5
  - High 40%: 115 37 01 02 5
  - Medium 25%: 70 29 08 00
  - Low 10%: 35 14 03 80
- Note: Sample sizes rounded to nearest five. Assumes a large population size.

### APPENDIX 3. SAMPLE SIZE EXAMPLE TABLE — key explanations
- A confidence interval sets out a range which has a known and controlled probability (generally 95 percent) to contain the true population value.
- The margin of error describes half the width of the confidence interval in percentage points.
- The table describes the margin of error around the estimate of the rate of non-compliance, not the value of the gap.
- When considering sample size, analysts should consider the possible number of yielding cases from which to estimate the gap based on possible rates of non-compliance and whether the sample size would provide sufficient accuracy.
- A pilot study based on a smaller initial sample can help indicate hit-rate and variance to guide decisions.
- In absence of a pilot study, an appropriate sample size can be estimated assuming the worst-case scenario of incidence (50 percent) and making assumptions regarding the distribution of values for estimating the confidence interval around the gap. This could be informed by known values and variance from risk-based audits.

### APPENDIX 4. PROJECT MANAGEMENT CHECKLIST FOR A RANDOM AUDIT PROGRAM (RAP)
- Planning / Project Management
  - Establish working group of analysts, auditors, and other specialists to implement the RAP.
  - Establish senior-level oversight of the RAP via a steering group or other means.
- Design
  - Analyze sample and sub-sample sizes options, their expected accuracy and costs, taking into account stratification options, e.g., taxpayer segments or current risk profiles.
    - Remember:
      - Simple compliance gap estimates should balance costs vs expected accuracy.
      - Risk analysis and profiling need larger samples and more detailed analysis.
      - If the RAP is to increase deterrence effects, sample size needs to be significant.
  - Agree audit and analytical resources required (time, staff, other costs).
  - Agree case selection, management and reporting requirements, with guidance for auditors.
  - Prepare reporting and data capture templates for auditors and agree quality assurance.
  - Agree timeline and project plan for RAP and reporting estimates, including milestones.
- Scope and Coverage
  - Agree non-compliance risks that will not be reliably captured in RAP (e.g., complex frauds).
  - Identify taxpayers that are either not in the sample frame or not cost-effective to sample.
  - Clarify coverage of expected PIT gap estimate, and how to cover missing areas.
- Implementation / Sample Selection
  - Sample frame is usually the taxpayer registry, cleaned and up to date.
  - For stratified samples, the registry data must include demographic and/or risk profiling data.
  - Select cases and share with audit teams, including instructions and templates for RAP.
- Audits
  - Auditors conduct comprehensive audits of sample cases.
  - Provide help desk or other support for auditors to resolve any queries.
  - Monitor progress of audits and record any exceptions.
  - Quality assure audits, and especially data recording.
- Analysis
  - Extract data on pre-agreed date and conduct analysis of results.
  - Check data for any recording issues e.g., duplicates and open cases. Identify and treat outliers.
  - If required, forecast the value of open cases, yield and unrecoverable debt.
  - Apply non-detection multiplier and adjust results for unaudited cases.
  - Quality assure the analysis and interpret the results.
- Reporting
  - Follow pre-agreed publication plan to publish gap estimates and associated analysis.
  - Report PIT gap as a percent of potential collections, with appropriate context and caveats.
  - Follow good practice on transparency, e.g., published methodology and data protection.

### Glossary excerpts (selected definitions)
- Bias: The tendency of a measurement process to over- or under-estimate the value of a population parameter.
- Bottom-up estimation: A bottom-up approach begins with taxpayer-level data and scales up non-compliance identified to the population level to identify the total gap.
- Confidence (statistical): The degree of certainty of an estimate. In sample statistics, confidence is typically set at 95%, which means there is a 95% chance that the actual population parameter or true value lies within the confidence interval.
- Margin of error: A measure of precision; it describes half the width of confidence interval in percentage points.
- Probability or random sampling: Selection methodology whereby each population unit has known, non-zero chance of inclusion in the sample; it is the only general method known which can provide a measure of precision of the estimate.
- Weight: An inflation factor applied against raw data.

*Source: tnmea2021009 - BOX 6. Weighting Data*

---


_Source: https://www.imf.org/-/media/files/publications/tnm/2021/english/tnmea2021009.pdf_