## _wp08183 — References (selected findings and appendix summary)

## Source details

**Canonical URL:** [_wp08183 — References (selected findings and appendix summary)](https://www.imf.org/-/media/websites/imf/imported-full-text-pdf/external/pubs/ft/wp/2008/_wp08183.pdf)

## Other formats

- [Markdown version](/-/media/websites/imf/imported-full-text-pdf/external/pubs/ft/wp/2008/_wp08183.pdf.md)
- [Structured JSON version](/-/media/websites/imf/imported-full-text-pdf/external/pubs/ft/wp/2008/_wp08183.pdf.json)

---

### I. Motivation and objectives
- Recent studies have used kernel density estimation (KDE) on grouped data (income averages for population quantiles) to estimate poverty (Sala-i-Martin 2002a, 2002b, 2004, 2006; Ackland, Dowrick, and Freyens, forthcoming; Fuentes, 2005).
- Two study goals:
  - Assess appropriateness of KDE on grouped data for poverty analysis across poverty indicators, poverty lines, KDE parameters (bandwidths and kernels), and income distributions.
  - Analyze robustness of KDE-based global poverty estimates to bandwidth choice and provide a framework for interpreting these estimates.
- Relevance: PSIA and global/regional poverty assessment when unit household surveys are unavailable; grouped data are often the only published source.

### Context, critique of prior work, and methods for world distribution construction
- The study responds to Sala-i-Martin (2002a–2006) who used KDE on grouped data and reported large reductions in world income poverty between 1970 and 2000.
- The authors show Sala-i-Martin’s figures lack robustness to bandwidth choice:
  - Multiple "optimal" bandwidths imply an estimated share of ‘$1/day poor’ in 2000 higher by a factor of 1.8 compared with some choices.
  - Alternative bandwidths produce $2/day poverty headcounts in 2000 higher by 287 million people relative to some baseline.
  - Sala-i-Martin’s $2/day headcount may have been underestimated by as much as 50 percent due to bandwidth choice.
- Two methods for constructing world income distribution:
  - Kernel of quintiles: assign each individual the average income of their national population quintile, then apply KDE.
  - Kernel of kernels: estimate country income density from quintile means, then integrate country densities population-weighted.

### Data structure, grouped-data KDE bias, and estimation setup
- Data considered: quantile means (trimmed means; usually five quintile means).
- Analytical evaluation of grouped-data KDE bias is prohibitively difficult; Monte Carlo simulations used.
- Distributions simulated: Log-normal, Dagum, Generalized Beta II (GB2), and a notional multimodal distribution (population-weighted 2004 world distribution).
- Simulation design highlights:
  - 200 samples per distribution, each with 1000 observations.
  - Default results reported based on KDE from five datapoints (quintile means).
  - Densities estimated at 100 equidistant points; 5000-observation samples simulated from fitted densities for summary comparisons.

### Bandwidth and kernel specifications examined
- Bandwidths:
  - Silverman 1 (S1): 1.06 × σ̂ × J^{−1/5} — tends to oversmooth; poor on heavily skewed distributions.
  - Silverman 2 (S2): 0.79 × IQR × J^{−1/5} — superior for long-tailed/heavily skewed; poor on bimodal.
  - Silverman 3 (S3): 0.9 × A × J^{−1/5}, A = min(σ̂, IQR/1.34) — balanced smoothing.
  - Sheather-Jones (S-J): data-driven plug-in estimator; recommended benchmark.
  - Hybrid bandwidth (Silverman S3 with σ̂ instead of A), computed assuming σ̂ = 0.6 exogenously:
    - Hybrid bandwidth = 0.39 for quintile data
    - Hybrid bandwidth = 0.34 for decile data
    - Hybrid bandwidth = 0.296 for ventile data
  - Oversmoothed bandwidth used as an upper-bound benchmark.
- Kernels: Gaussian, Epanechnikov, Quartic, Triweight, Triangular, Uniform.

### Key Monte Carlo and empirical findings (biases and distributional effects)
- Information content:
  - Five quantile means provide more information than a small raw sample but generally insufficient for reliable nonparametric recovery of distributional features.
- Summary statistic distortions (Table 1, Quartic kernel, S3 bandwidth, quintile inputs):
  - Mean: overestimated (e.g., Log-normal ratio 1.12; Dagum 1.11; GB2 1.13; Multimodal 1.17).
  - Median: fairly well estimated (ratios around 0.98–1.03 except Multimodal 0.91).
  - St. Dev.: substantially underestimated for unimodal distributions (e.g., Log-normal 0.89) and overestimated for multimodal (1.24).
  - Quintile mean ratios: poorest quintiles typically sub-unity; richest quintiles typically >1 (e.g., Log-normal bottom 0.93 top 1.25).
- Density bias pattern:
  - Two crossing points where bias = 0; density biased upward in tails (left and right) and downward in the middle.
  - Left-tail upward bias consistent with underestimation of poorest-group average incomes; right-tail upward bias consistent with overestimation of richest-group averages.
- Bandwidth effects:
  - Hybrid bandwidth often smaller than optimal and leads to undersmoothing; can produce zero density below a threshold income and spurious modes when finite-support kernels are used.
  - Bandwidth strongly influences the lowest income level at which the estimator can produce nonzero density; poverty lines below that threshold yield zero estimated poverty.
- Poverty headcount ratio biases (examples):
  - For low poverty lines (poverty line = median × 0.25): systematic overestimation.
    - Log-normal low-line true headcount 12.10 — quintiles ratio 1.17, deciles 1.28, ventiles 1.23 (Table 2).
  - For high poverty lines (poverty line = median × 1.75): systematic underestimation.
    - Multimodal high-line true headcount 81.83 — quintiles ratio 0.91, deciles 0.91, ventiles 0.94 (Table 2).
  - Bias magnitudes:
    - Poverty headcount ratio biases reach 6–7 percentage points for unimodal distributions.
    - Biases reach 10–11 percentage points for the multimodal distribution considered.
- FGT and distribution-sensitive measures:
  - FGT indices of depth and severity are more substantially underestimated by quintile data than by decile or ventile data; bias magnitude increases with index sensitivity (e.g., FGT(3), FGT(4) ratios vary widely in Tables 2–4).

### Country case studies (Tanzania, Nicaragua, Vietnam)
- International poverty lines: $1/day and $2/day; capability poverty line also analyzed.
- $1/day line:
  - Headcount ratio overstated by up to factor 1.6 and understated by as low as 0.94 depending on quantile aggregation (Table 5).
- $2/day line:
  - Headcount understated by at most 8 percent in examples (e.g., Nicaragua $2/day survey 79.03 percent — quintile input understates by ~6 percentage points).
- Poverty gap and FGT:
  - Poverty gap ratio overestimated by up to factor 1.75 in Vietnam (least poor in sample); occasionally underestimated in Tanzania (poorest country in sample).
  - Distribution-sensitive FGT(3) overestimated by factor 2 using S1 and by one fifth using S3 (Vietnam).
- Bandwidth sensitivity:
  - S1 can overestimate headcount by 12 percent (Nicaragua, capability line).
  - S3 and hybrid can overestimate by 5 percent (Nicaragua).
  - S3 often the best-performing "optimal" bandwidth in these country comparisons.
- Visual density artifacts (Figure 6):
  - S1 oversmoothing causes left-tail bias; S3 may reveal spurious modes with quintile means; fixing bandwidth across kernels can produce extremely distorted graphics.

### Global assessment: sensitivity of world poverty estimates to bandwidth choice
- Data: income shares for 94 developing countries (1990) covering 94 percent of world population; also 134 countries for year 2000 to compare with Sala-i-Martin.
- Global headcount sensitivity ($1/day, $1.5/day, $2/day):
  - $1/day headcount ratios most sensitive because $1/day falls in left tail.
  - For $1/day headcount in 1990 and 2000, ratios vary by factor 1.8 when oversmoothed bandwidth included; factor 1.6 when excluded.
  - In 2000, $1/day, $1.5/day and $2/day headcount ratios vary by factor between 1.4 and 1.8 across optimal bandwidths.
- Impact on counts of poor (Table 8):
  - Variation in numbers of poor translates into differences between 162 and 278 million people across bandwidth choices and poverty lines.
  - For 2000 $2/day poor: variation of 278 million — an error of 50 percent relative to Sala-i-Martin’s $2/day global headcount for 2000 (Sala-i-Martin, 2006) per authors’ comparison.
  - For 1990 $1.5/day poor: variation of 180 million — an error of 36 percent relative to Sala-i-Martin’s 1990 $1.5/day headcount.
- Trend sensitivity 1990–2000:
  - Fall in $1.5/day and $2/day poverty rates ranges between 7 percent and 18 percent across bandwidths (Minoiu, 2007, Table 11).
  - Number of people lifted from $1/day poverty between 1990 and 2000 ranges between 19 million and 38 million.
  - Reduction in $1.5/day poverty ranges between 45 and 92 million.
  - A reduction of 45 million in $1.5/day poor is one half of Sala-i-Martin’s documented reduction; a reduction of 25 million in $1/day poor is one fifth of Chen and Ravallion’s documented fall of 129.5 million (1990–2001).
- Implication: Sala-i-Martin’s use of a hybrid bandwidth likely biased global poverty estimates downward given the bandwidths examined.

### Conclusions and policy recommendations
- Main conclusions:
  - KDE applied to grouped (quantile-mean) data produces substantial and variable biases; sign and magnitude depend on data-generating distribution, bandwidth, kernel, and number of quantile means.
  - Average incomes of poorer groups tend to be overstated and of richer groups understated in the unimodal cases studied.
  - KDE-based global headcount ratios vary by up to a factor of 1.8 across recommended bandwidths; number of ‘$2/day poor’ varies by 136 million in 1990 and by 278 million in 2000 across bandwidth choices.
  - KDE on grouped data can yield misleading visual representations and spurious modes.
- Policy and methodological recommendations:
  - Exercise caution in applying standard KDE to grouped quantile-mean data for poverty measurement.
  - Prefer data-driven bandwidth selectors (e.g., Sheather-Jones S-J) over fixed hybrid bandwidths for robustness.
  - Perform Monte Carlo or simulation-based sensitivity analyses to characterize small-sample bias when only grouped data are available.
  - Apply boundary-bias corrections and log-transformation to mitigate left-boundary bias, but recognize these do not fully resolve problems.
  - Use alternative methods (e.g., parametric Lorenz curve interpolation) whenever feasible, especially given poor performance of KDE on multimodal distributions.

### Key appendix numeric highlights (selected exact figures)
- Multimodal distribution (2004 population-weighted world distribution): Mean: $8295 St Dev 9571 Min $515.6 Max $64298.
- Table 7 (global poverty rates, 1990 examples):
  - $1/day: S3 7.2 Overs-smoothed 9.5 Variant 6.4 Sheather-Jones 7.5 Direct plug-in 8.4 Hybrid 5.3 — Ratio highest/lowest 1.8 — Percentage-point diff. 4.2.
- Table 8 (global poverty counts, 2000 examples, millions):
  - $1/day: S3 256 Overs-smoothed 362 Variant 232 Sheather-Jones 269 Direct plug-in 300 Hybrid 200 — Difference highest-lowest 162.
  - $2/day: S3 830 Overs-smoothed 998 Variant 796 Sheather-Jones 850 Direct plug-in 899 Hybrid 720 — Difference 278.

*Source: _wp08183 - References (selected content and appendix figures/tables).*

### References..............................................................................................................

### _wp08183 - References....................................................................................................................

### I. MOTIVATION
- Recent studies have employed nonparametric smoothing techniques, particularly kernel density estimation (KDE) on grouped data, to obtain poverty estimates (Sala-i-Martin 2002a, 2002b, 2004, 2006; Ackland, Dowrick, and Freyens, forthcoming; Fuentes, 2005).
- Grouped data (usually income averages for a small number of population quantiles) are necessary for long-horizon world poverty and inequality assessments because representative household surveys are not available, or are difficult to obtain or analyze (Shorrocks and Wan, 2008).
- The study has two goals:
  - Assess the appropriateness of KDE methods on grouped data for poverty analysis, identifying biases in poverty estimates across a wide range of poverty indicators, poverty lines, KDE parameters (e.g., bandwidths and kernels), and income distributions.
  - Analyze the robustness of KDE-based global poverty estimates to the choice of bandwidth and provide a framework for interpreting these estimates.
- Relevance: The analysis is relevant to researchers conducting poverty and social impact analyses (PSIA), especially when full household surveys are unavailable and grouped data are the only published source (e.g., low-income or post-conflict countries, PRGF program assessments).

### CONTEXT AND PRIOR WORK
- The study responds to Sala-i-Martin (2002a, 2002b, 2004, 2006), who used KDE on grouped data to estimate national, regional, and global poverty and concluded substantial reductions in world income poverty between 1970 and 2000.
- The authors demonstrate that Sala-i-Martin’s figures lack robustness to the choice of underlying parameters:
  - Multiple "optimal" bandwidth values are consistent with an estimated share of ‘$1/day poor’ in 2000 higher than Sala-i-Martin’s estimate by a factor of 1.8.
  - Alternative bandwidths give rise to $2/day poverty headcounts in 2000 higher than Sala-i-Martin’s estimate by 287 million people.
  - The authors suggest Sala-i-Martin may have underestimated the $2/day headcount by as much as 50 percent due to methodological choice of bandwidth.
- The term "optimal" is used to describe smoothing parameter values that maximize the approximate integrated mean square error (a global goodness-of-fit criterion). Alternative optimality criteria (e.g., goodness-of-fit up to a poverty line, or accuracy of the poverty headcount ratio) may yield different bandwidths.

### DATA STRUCTURE AND APPROACH
- Grouped data considered: income averages for a small number of population quantiles (usually five).
- Because analytical derivations of KDE properties in small samples are prohibitively difficult or impossible, the authors undertake Monte Carlo simulations for a range of plausible income distributions:
  - Distributions considered: Log-normal (two parameters), Dagum (three parameters), Generalized Beta II (four parameters), and a notional multimodal distribution.
- Empirical comparison: Three nationally representative household surveys (Nicaragua, Tanzania and Vietnam) are used to compare KDE-based poverty estimates from grouped data with estimates from unit data.
- Global assessment: The performance of KDE in global poverty analysis is assessed using grouped data for a large number of developing countries.

### FINDINGS ON BIASES AND LIMITATIONS
- The informational poverty of five income averages for five population quantiles (per country-year) renders nonparametric density estimation methods inappropriate in many cases, although five averages provide more information than a small random sample.
- Poverty is misestimated in a majority of cases, but in some instances biases at different income levels cancel, yielding good estimates.
- Magnitude of biases identified:
  - Biases in poverty headcount ratio estimates reach 6-7 percentage points for the unimodal distributions considered.
  - Biases reach 10-11 percentage points for the multimodal distribution considered.
- Biases in estimates of Foster-Greer-Thorbecke (FGT) poverty measures generally increase with the degree of distributional sensitivity.
- Reasons researchers use KDE on grouped data:
  - KDE does not require prior beliefs about the functional form of the underlying distribution (nonparametric).
  - KDE reproduces the entire income distribution from a manageable amount of data, useful for regional or global analyses when household surveys are unavailable or costly to process.
  - Unit data for many nationally representative household surveys (including for China and India) are not readily publicly available; only grouped data may be published.

### COMPARISON WITH ALTERNATIVE METHODS
- An alternative to KDE is parametric estimation of the Lorenz curve; many parametric Lorenz forms exist and are implemented in tools such as POVCAL and SimSIP (World Bank).
- While not explicitly compared here, the Lorenz curve parameterizations embodied in POVCAL and SimSIP:
  - Perform well on grouped data for unimodal distributions.
  - Perform less well for multimodal distributions (Minoiu and Reddy, forthcoming).

### MONTE CARLO AND EMPIRICAL MATERIAL (APPENDIX ITEMS LISTED)
- Appendix Figures (listed):
  - Figure 1. Distributions used in Monte Carlo analysis
  - Figure 2. Bias of KDE-based density (log-normal distribution)
  - Figure 3. Bias of estimated density (multimodal distribution)
  - Figure 4. Bias of estimated density (Dagum distribution)
  - Figure 5. Bias in the poverty headcount ratio versus location of poverty line
  - Figure 6. Survey-based and grouped data KDE-based density estimates
- Appendix Tables (listed):
  - Table 1. Summary statistics from KDE-based sample
  - Table 2. Bias of poverty measures (Low and High Poverty Lines)
  - Table 3. Bias of poverty measures (Triweight kernel, Poverty line: 0.25 x median)
  - Table 4. Bias of poverty measures (Hybrid bandwidth, Poverty line: 0.5 x median)
  - Table 5. Bias of poverty measures (Epanechnikov kernel, Silverman bandwidth)
  - Table 6. Bias of poverty measures (Gaussian kernel, Poverty line: Capability)
  - Table 7. Global poverty rates (% poor)
  - Table 8. Global poverty counts (millions)

*Source: _wp08183 - References...............................................................................................................................*

### 20.2 percent to 7 percent between 1970 and 2000. The author proposes two methods for

### _wp08183 - 20.2 percent to 7 percent between 1970 and 2000. The author proposes two methods for

### Methods for constructing a world income distribution
- Two methods described:
  - Kernel of quintiles method: construct dataset where each person’s income level is the average income of the national population quintile to which that person belongs; apply kernel density estimation (KDE) to that dataset.
  - Kernel of kernels method: estimate each country’s income density from quintile means, then integrate the individual country densities into a population-weighted world income density.
- The analysis is conducted in a manner directly comparable to the kernel of kernels method proposed by Sala-i-Martin; the two approaches yield similar results.

### Data structure and bias of the estimator
- Data available to the researcher: average incomes of several population quantiles (quantile means), i.e., linear functions of order statistics (“systematic statistics”).
- Quantile means:
  - Are trimmed means (four quintile means are asymmetrically trimmed; the central quintile mean is symmetrically trimmed).
  - Are asymptotically normally distributed under mild conditions (Stigler (1973, 1974); Mason (1981)).
  - Carry more precise information about the location of underlying order statistics than a small raw sample but less information than a large sample of unit data.
- Nonparametric KDE limitations:
  - Nonparametric estimators rely on a wealth of nearby observations; with a small number of quantile means, KDE is inappropriate for reliably uncovering distributional features.
  - Boundary (left) bias at zero can underestimate mass near zero by as much as 50 percent (Marron and Ruppert, 1994). Log-transformation of quantile means is used to partially circumvent boundary bias.
- Derived grouped-data KDE bias (compact form shown in text):
  - Grouped-data KDE bias depends on density probability functions of quantile means, kernel k(.), bandwidth h, and the number of grouped datapoints J.
  - Let v(x) = sum_{j=1}^J g_j(x)/J; the grouped-data estimator has the same bias as the survey-based estimator if v(x) = f(x).
  - Analytical evaluation of v(x) is prohibitively difficult due to correlation structure among trimmed means; Monte Carlo simulations are used to assess small-sample bias.

### Bandwidths and kernels considered
- Bandwidths used (three Silverman rule-of-thumb bandwidths S1, S2, S3 and others including Sheather-Jones (S-J) and variants):
  - Silverman 1 (S1): 1.06 × σ̂ × J^{−1/5}. Properties: tends to oversmooth; performs poorly on heavily skewed distributions.
  - Silverman 2 (S2): 0.79 × IQR × J^{−1/5}. Properties: superior for long-tailed and heavily skewed distributions; performs poorly on bimodal distributions.
  - Silverman 3 (S3): 0.9 × A × J^{−1/5}, where A = min(σ̂, IQR/1.34). Properties: balanced smoothing; works reasonably on skewed and multimodal distributions.
  - Sheather-Jones (S-J): data-driven plug-in estimator; recommended benchmark; outperforms rule-of-thumb bandwidths theoretically and in simulations.
  - Hybrid bandwidth (Silverman’s S3 with σ̂ instead of A): fixed across kernels and datasets; computed assuming a standard deviation of 0.6 (regardless of country), yielding hybrid bandwidth values exogenous to data:
    - Hybrid bandwidth = 0.39 for quintile data
    - Hybrid bandwidth = 0.34 for decile data
    - Hybrid bandwidth = 0.296 for ventile data
  - Oversmoothed bandwidth (largest bandwidth consistent with a ‘reasonable’ amount of smoothing) included as an upper-bound benchmark.
- Kernels considered: Gaussian, Epanechnikov, Quartic, Triweight, Triangular, Uniform.
  - Asymptotically choice of kernel is inconsequential for minimizing mean integrated squared error, but with small numbers of quantile means choice may matter in practice.
- Estimation implementation details:
  - Each density estimated at 100 equidistant points.
  - Large samples of 5000 observations simulated from the fitted density to compare summary statistics and poverty measures.

### Monte Carlo study design
- Distributions used in simulations:
  - Log-normal (parameters fit to Russian 1995 income data).
  - Dagum (parameters fit to Mexican 1996 income data).
  - Generalized Beta II (parameters fit to Mexican 1996 income data).
  - Notional multimodal distribution: the population-weighted 2004 world distribution of income where individuals of each country are assigned per capita GDP of that country (modes correspond to India mean, China mean, and richest nations mean).
- Simulation setup:
  - From each distribution, 200 samples with 1000 observations each are simulated.
  - Results reported are based on KDE from five datapoints unless otherwise specified.
  - For density-fitting evaluation: 200 draws, averaging densities at fixed evaluation points to compute bias along the support.

### Key findings from Monte Carlo analysis
- Summary statistics and quantile means:
  - Mean: systematically overestimated (by at most one fifth).
  - Median: estimated fairly well across distributions.
  - Standard deviation: substantially underestimated for the three unimodal distributions; overestimated for the multimodal distribution.
  - Quantile mean estimation pattern: ratios of estimated to true quintile means are
    - Sub-unity for the poorest two population groups (systematic underestimation for poorest 40 percent).
    - Greater than 1 for the richest two population groups (systematic overestimation for richest 40 percent).
    - The middle 20 percent’s average income is precisely estimated for unimodal distributions (except the multimodal case where it is understated).
- Density bias along support:
  - KDE on grouped data produces distortions at every point along income support with two crossing points where bias is zero.
  - Estimated density biased upwards in tails (left and right) and biased downwards in the middle.
  - Upward bias in left tail consistent with underestimation of poorest group average incomes; upward bias in right tail consistent with overestimation of richest group average incomes.
  - Choice of kernel is not visually consequential when canonical bandwidths are used (canonical bandwidths ensure comparable smoothing across kernels), but kernel choice becomes consequential when using noncanonical/hybrid bandwidths.
- Bandwidth effects and hybrid bandwidth pitfalls:
  - Hybrid bandwidth (fixed, exogenous values 0.39, 0.34, 0.296 for quintile/decile/ventile) can be close to optimal by chance but generally leads to nontrivial bias and is inappropriate for poverty analysis from grouped data.
  - Hybrid bandwidth often smaller than optimal bandwidths for datasets considered and thus leads to undersmoothing; undersmoothing can concentrate density mass at the quantile means and produce areas of zero density between modes when kernels have finite support.
  - Bandwidth strongly influences the lowest income level at which the estimator can produce nonzero density; poverty lines falling below that threshold lead to zero estimated poverty.
- Poverty estimate biases:
  - Poverty headcount ratio:
    - Systematically overestimated for low poverty lines (poverty line = population median × 0.25).
    - Systematically underestimated for high poverty lines (poverty line = population median × 1.75).
  - Quantitative examples (Log-normal distribution, quintile means):
    - Share of poor overestimated by a factor of 1.17 (or 2 percentage points) for quintile means at the low poverty line.
    - Bias rises to a factor of 1.28 (or 3.4 percentage points) for decile means.
    - Biases slightly lower for ventile means.
  - FGT poverty indices:
    - FGT indicators of depth of poverty are more substantially underestimated by quintile data than by decile or ventile data.
    - Bias magnitude increases with distributional sensitivity of the FGT index.
  - For high poverty line:
    - Poverty headcount ratio underestimated by almost 9 percent (or 7 percentage points) for multimodal data.
    - Underestimated by between 5 and 7 percent (approximately 5 percentage points) for other distributions.
  - Bias across range of poverty lines:
    - Extent of poverty broadly overestimated for lower poverty lines (positive bias up to 9 percentage points reported for lower lines in multimodal case).
    - Estimates relatively accurate for poverty lines near the population median (biases cancel where density over- and underestimation offset).
    - Underestimation for higher poverty lines (up to 11 percentage points at the median observed for multimodal distribution where the density estimate “misses” the first mode).

### Implications for poverty analysis from grouped data
- KDE on grouped (quantile-mean) data:
  - Produces systematic distortions in summary statistics, quantile means, densities along the support, and poverty measures.
  - Leads to overestimation of poverty for low poverty lines and underestimation for high poverty lines; magnitude of biases depends on number of quantile means, bandwidth choice, kernel choice, and true underlying distribution shape (notably multimodality).
- Practical consequences:
  - Sparse grouped data (quintile, decile, ventile means) are insufficient to reliably recover tails of the income distribution via standard KDE methods.
  - Boundary bias and bandwidth selection critically affect poverty estimates; log-transformation alleviates but does not eliminate these problems.
  - Use of non-data-driven hybrid bandwidths (as in some prior work) is likely to produce misleading density diagrams and biased poverty estimates unless optimality occurs by chance.
- Recommended analytical approach (based on findings in the text):
  - Avoid relying on KDE applied to a small number of quantile means for poverty measurement without careful assessment of bandwidth sensitivity and bias.
  - Use Monte Carlo or other simulation-based sensitivity analysis to characterize small-sample bias when only grouped data are available.
  - Prefer data-driven bandwidth selectors (e.g., Sheather-Jones S-J) over fixed hybrid bandwidths for robustness, and consider boundary-bias correction measures and log-transformation to mitigate left-boundary bias.
  - Recognize that increasing the number of quantile means (towards 25–30) improves estimator performance, but such granularity is rarely available in practice.

*Italic source: content unit _wp08183 - 20.2 percent to 7 percent between 1970 and 2000. The author proposes two methods for (source PDF provided).*

### 0.5 x the median). The size of the bias and the way in which it varies with the data generating

### _wp08183 - 0.5 x the median). The size of the bias and the way in which it varies with the data generating

### Biases from kernel density estimation (KDE) on grouped data
- The size and sign of biases in poverty indicators depend on the data generating process, the bandwidth, the weighting function, the kernel, and the number of quantile means available.
- For multimodal distributions:
  - S1 bandwidth leads to an upward bias of 70 percent (poverty headcount ratio).
  - Hybrid bandwidth leads to a downward bias of 50 percent (poverty headcount ratio).
  - Substantial downward biases are associated with the hybrid bandwidth across poverty indicators (Table 3).
- For the Dagum distribution (examples of bandwidth-kernel variation):
  - Poverty headcount ratio biased upwards by 11 percent (Gaussian kernel).
  - Poverty headcount ratio biased downwards by 9 percent (Triweight kernel).
- For unimodal distributions:
  - Poverty headcount ratio is overestimated for lower poverty lines.
  - Poverty headcount ratio is accurately estimated at poverty lines close to the population median.
  - Poverty headcount ratio is underestimated for higher poverty lines.
- Magnitude of biases:
  - For the poverty rate, biases can reach 6-7 percentage points in unimodal distributions.
  - For the poverty rate, biases can reach 10-11 percentage points in the multimodal distribution considered.

### Country studies: Tanzania, Nicaragua, Vietnam (grouped-data KDE vs survey-based)
- Data and poverty lines considered:
  - Countries: Tanzania, Nicaragua, Vietnam (nationally representative household surveys).
  - International poverty lines: $1/day and $2/day.
  - A capability, nutritionally anchored poverty line (Reddy, Visaria and Asali).
- $1/day poverty line:
  - Headcount ratio is overstated by a factor of at most 1.6 and understated by a factor of at most 0.94 regardless of number of quantile means (Table 5).
- $2/day poverty line:
  - Headcount ratio understated by at most 8 percent (example: Nicaraguan $2/day headcount ratio of 79.03 percent is understated by approximately 6 percentage points when input data are quintile means).
- Poverty gap ratio:
  - Overestimated by a factor of maximum 1.75 for the least poor country (Vietnam).
  - Less misestimated for Nicaragua.
  - Occasionally underestimated for the poorest country (Tanzania).
- Bandwidth sensitivity (using capability poverty line; Table 6):
  - Poverty headcount ratio overestimated by 12 percent (S1 bandwidth, Nicaragua).
  - Poverty headcount ratio overestimated by 5 percent (S3 and hybrid bandwidth, Nicaragua).
  - Distributionally-sensitive FGT (3) overestimated by a factor of 2 using S1 and by one fifth using S3 (Vietnam).
  - Best performing optimal bandwidth appears to be S3 in the majority of cases.
- Visual density representation issues (Figure 6):
  - S1 bandwidth can cause oversmoothing and left-tail density biases.
  - S3 bandwidth may reveal spurious modes when using quintile means (artifact, not present in unit data).
  - Keeping bandwidth fixed across kernels (canonical bandwidths) can lead to extremely distorted visual representations.
  - Quartic kernel with S1 bandwidth yields positive density biases in left and right tails and negative biases in center.
  - KDE on decile means: more biased locally in the left tail than on quintile means but less biased globally.

### Global poverty sensitivity to bandwidth choice
- Data and approach:
  - Income shares for 94 developing countries covering 94 percent of the world’s population in 1990 from UNU/WIDER World Inequality database V. 2.0a (2005), scaled using per capita GDP (PPP) from WDI (2006) to obtain quintile income averages; KDE undertaken for each country and fitted distributions aggregated to world distribution.
  - Also analyzed year 2000 for a larger sample of 134 countries to compare with Sala-i-Martin.
- Bandwidths considered:
  - Silverman’s rule-of-thumb bandwidth (S3), oversmoothed bandwidth (variant), Sheather-Jones plug-in estimator, direct plug-in estimator, hybrid bandwidth (to compare with Sala-i-Martin).
  - Gaussian kernel focus (results similar with Epanechnikov kernel and canonical bandwidth).
- Sensitivity of global headcount ratios:
  - For the $1/day poverty line in 1990 and 2000, headcount ratios are most sensitive to bandwidth choice because the $1/day line falls in the left tail.
  - For the $1/day poverty line, headcount ratio varies by a factor of 1.8 when the oversmoothed bandwidth is considered and by a factor of 1.6 when it is excluded.
  - Headcount ratios vary less for higher poverty lines; almost equal across bandwidths for $3/day and $4/day in 1990.
  - In 2000, $1/day, $1.5/day and $2/day headcount ratios vary by a factor between 1.4 and 1.8 across optimal bandwidths.
- Impact on numbers of poor:
  - Variation in numbers of poor translates into a difference between 162 and 278 million people (across bandwidths and lines).
  - Under- or overestimating the ‘$2/day poor’ by 278 million individuals (in 2000) would represent an error of 50 percent (based on the $2/day global headcount for 2000 by Sala-i-Martin, 2006).
  - Under- or overestimating the number of ‘1.5/day poor’ by 180 million individuals (in 1990) would represent an error of 36 percent (based on the $1.5/day global headcount in the same year of Sala-i-Martin, 2006).
- Trend sensitivity (1990–2000):
  - Fall in $1.5/day and $2/day poverty rates ranges between 7 percent and 18 percent across bandwidths (Minoiu, 2007, Table 11).
  - Number of people lifted from $1/day poverty between 1990 and 2000 ranges between 19 million and 38 million.
  - Reduction in $1.5/day poverty ranges between 45 and 92 million.
  - A reduction of 45 million in ‘$1.5/day poor’ is one half of Sala-i-Martin’s documented reduction; a reduction of 25 million in ‘$1/day poor’ is one fifth of Chen and Ravallion’s documented fall of 129.5 million (between 1990 and 2001).
  - All KDE-based estimates are nevertheless consistent with a reduction in world poverty, but the magnitude of that reduction is sensitive to bandwidth choice.
- Implication for Sala-i-Martin’s results:
  - Sala-i-Martin’s exclusive focus on the hybrid bandwidth likely led in the direction of underestimating global poverty.

### Conclusions and policy implications
- Main findings:
  - Biases from applying KDE to grouped data are substantial and variable; direction and magnitude cannot be predicted without distribution-specific analysis.
  - Average income of poorer groups is overstated; average income of richer groups is understated for the unimodal distributions considered.
  - KDE on grouped data can lead to misleading visual representations of income distributions.
  - Global KDE-based headcount ratios vary by a factor of 1.8 across recommended bandwidths; number of ‘$2/day poor’ varies by 136 million in 1990 and by 278 million in 2000 depending on bandwidth.
  - Recent global poverty estimates that rely on KDE with grouped data (e.g., Sala-i-Martin) are highly sensitive to bandwidth choice and likely biased downward given the bandwidth choices examined.
- Recommendations:
  - Applied researchers should exercise caution in employing standard KDE methods on grouped data.
  - Alternative estimation methods such as parametric Lorenz curve interpolation should be applied whenever possible.

*Source: IMF working paper content unit provided.*

### REFERENCES

### _wp08183 - REFERENCES

### References (bibliographic list)
- Comprehensive bibliography of sources on global income distribution, poverty measurement, kernel density estimation, and parametric and nonparametric methods, including (but not limited to) works by Ackland, Aziz and Duenwald, Babones, Bandourian et al., Bhalla, Berry et al., Bianchi, Bourguignon and Morrisson, Chen and Ravallion, Coady, Deaton, Dhongde, Essama-Nssah et al., Fuentes, Grosh and Nafziger, Jann, Jones et al., Kakwani and Son, Marron and Nolan, Mason, Milanovic, Minoiu, Moore, Mosteller, Nye et al., Pittau, Pritchett, Quah, Reddy and Minoiu, Sala-i-Martin, Sheather and Jones, Siddiqui and Butler, Silverman, Shorrocks and Wan, Stigler, UNU/WIDER (World Inequality Database V. 2.0a (June)), Yatchew, Yotopoulos, Wand and Jones, World Development Indicators Online Database (2006), World Bank (1994), Wu and Perloff.
- Topics covered by the references include: Measuring Global Poverty; China’s Provincial Growth Dynamics; World Distribution of Income; Kernel Density Estimation; Bandwidth Selection methods (Sheather-Jones, Silverman variants, S1–S3, Hybrid (Sala-i-Martin)); Dagum, Log-normal, Generalized Beta II, and multimodal distributions; Poverty measures (headcount, poverty gap ratio, squared poverty gap, FGT(3), FGT(4)); simulations linking household surveys with macro-models (PAMS); and ungrouping income distributions.

### Appendix — Monte Carlo and KDE-based analysis (figures and tables)
- Figure 1: Distributions used in Monte Carlo analysis
  - Panel 1. Dagum: Mean: 289488.1 St Dev 639427.3 [right tail clipped at 95th percentile]
  - Panel 2. Log-normal: Mean: 72253.87 St Dev 127140.1 [right tail clipped at 90th percentile] — Parameters from fitted 1995 Russian income distribution
  - Panel 3. Generalized Beta II: Mean: 30054.52 St Dev 91279.17 [right tail clipped at 90th percentile] — Parameters from fitted 1996 Mexican income distribution
  - Panel 4. Multimodal distribution representing the 2004 population-weighted world distribution of income: Mean: $8295 St Dev 9571 Min $515.6 Max $64298

- Table 1: Summary statistics from KDE-based sample (ratios between estimated quantity and true value; Quartic kernel, S3 bandwidth; Input data: Quintile means)
  - Log-normal: Mean 1.12 Median 1.03 St. Dev. 0.89; Quintile means: Bottom 0.93 Second 0.94 Third 1.03 Fourth 1.20 Top 1.25
  - Dagum: Mean 1.11 Median 0.98 St. Dev. 0.59; Quintile means: Bottom 0.98 Second 0.92 Third 1.01 Fourth 1.17 Top 1.14
  - GB2: Mean 1.13 Median 1.02 St. Dev. 0.45; Quintile means: Bottom 0.99 Second 0.92 Third 1.01 Fourth 1.19 Top 1.12
  - Multimodal: Mean 1.17 Median 0.91 St. Dev. 1.24; Quintile means: Bottom 0.68 Second 0.87 Third 0.92 Fourth 1.11 Top 1.04

- Figures 2–4: Bias of KDE-based density estimates (examples)
  - Figure 2: Log-normal distribution — Input data: Quintile means - Silverman 3 bandwidth; shows True vs Estimated Density and Bias (Epanechnikov and Gaussian kernels)
  - Figure 3: Multimodal distribution — Input data: Quintile means; S3-GaussianHybrid-Epan comparisons and bias plots
  - Figure 4: Dagum distribution — comparisons across S1 bandwidth, S3 bandwidth, Hybrid bandwidth, Gaussian and Epanechnikov kernels

- Table 2: Bias of poverty measures (ratios between estimated quantity and its true counterpart; S3 bandwidth, Epanechnikov kernel)
  - Poverty line Æ LOW
    - Poverty headcount ratio (%) — True quantities and estimated ratios:
      - Log-normal True 12.10 — Quintiles 1.17 Deciles 1.28 Ventiles 1.23
      - Dagum True 9.43 — Quintiles 1.09 Deciles 1.26 Ventiles 1.21
      - Gen. Beta II True 9.45 — Quintiles 1.07 Deciles 1.24 Ventiles 1.18
      - Multimodal True 8.14 — Quintiles 1.00 Deciles 1.18 Ventiles 1.15
    - Poverty gap ratio — Log-normal True 4.57 — Quintiles 1.14 Deciles 1.40 Ventiles 1.34; Multimodal True 2.93 — Quintiles 0.63 Deciles 1.04 Ventiles 1.08
    - Squared poverty gap — Log-normal True 2.49 — Quintiles 1.02 Deciles 1.40 Ventiles 1.35; Multimodal True 1.22 — Quintiles 0.48 Deciles 1.09 Ventiles 1.18
    - FGT(3) — Log-normal True 1.54 — Quintiles 0.92 Deciles 1.42 Ventiles 1.37; Multimodal True 0.56 — Quintiles 0.38 Deciles 1.17 Ventiles 1.34
    - FGT(4) — Log-normal True 1.03 — Quintiles 0.82 Deciles 1.41 Ventiles 1.37; Multimodal True 0.27 — Quintiles 0.30 Deciles 1.29 Ventiles 1.54
  - Poverty line Æ HIGH
    - Poverty headcount ratio (%) — Log-normal True 68.02 — Quintiles 0.95 Deciles 0.96 Ventiles 0.97; Multimodal True 81.83 — Quintiles 0.91 Deciles 0.91 Ventiles 0.94
    - Poverty gap ratio — Log-normal True 40.77 — Quintiles 0.99 Deciles 1.01 Ventiles 1.01; Multimodal True 45.79 — Quintiles 0.96 Deciles 0.97 Ventiles 0.98
    - Squared poverty gap, FGT(3), FGT(4) — reported with exact true values and estimated ratios for Log-normal, Dagum, Gen. Beta II, Multimodal (see table for all entries)

- Figure 5: Bias in the poverty headcount ratio versus location of poverty line (expressed in percentage points)
  - Horizontal axis: "Poverty Line = true median multiplied by a factor of..." (factors include 0.25, 0.5, 0.75, 1, 1.25, 1.5, 1.75, 2)
  - Shows estimated - true headcount ratio for Log-normal, Multimodal, Dagum, Gen. Beta II

- Table 3: Bias of poverty measures (Triweight kernel, Poverty line: 0.25 x median; ratios between estimated and true)
  - Example entries:
    - Poverty headcount ratio (%) True quantities:
      - Log-normal 12.10 — S1 1.45 S2 1.24 S3 1.17 Hybrid (Sala-i-Martin) 1.40
      - Dagum 9.43 — S1 1.48 S2 1.17 S3 1.10 Hybrid 1.04
      - Multimodal 8.14 — S1 1.70 S2 1.10 S3 1.00 Hybrid 0.50
    - Poverty gap ratio, Squared poverty gap — similar S1–Hybrid entries for each distribution

- Table 4: Bias of poverty measures (Hybrid bandwidth, Poverty line: 0.5 x median; comparisons across kernels)
  - Poverty headcount ratio (%) True values and kernel ratios:
    - Log-normal True 27.91 — Gaussian 0.98 Uniform 1.01 Epan. 0.97 Quartic 0.95 Triweight 0.93 Tri-angular 0.95
    - Dagum True 23.57 — Gaussian 1.04 Uniform 1.05 Epan. 0.98 Quartic 0.93 Triweight 0.91 Tri-angular 0.95
    - Multimodal True 19.19 — Gaussian 1.39 Uniform 1.43 Epan. 1.40 Quartic 1.36 Triweight 1.33 Tri-angular 1.36
  - Poverty gap ratio and Squared poverty gap reported similarly with exact numeric entries

- Table 5: Bias of poverty measures (Epanechnikov kernel, S3 bandwidth) — Survey-based country examples, ratios between estimated quantity and survey counterpart
  - Poverty line: $1/day and $2/day; Indicator: Poverty headcount ratio (%), Poverty gap ratio, Squared poverty gap, FGT(3), FGT(4)
  - Country survey estimates and KDE ratios (Quintiles, Deciles, Ventiles):
    - Vietnam: Poverty headcount ratio (%) Survey estimate $1/day 5.20 — Quintiles 1.34 Deciles 1.59 Ventiles 1.47; $2/day 35.69 — Quintiles 1.04 Deciles 1.04 Ventiles 1.03
    - Nicaragua: $1/day 44.62 — Quintiles 1.02 Deciles 1.02 Ventiles 1.02; $2/day 79.03 — Quintiles 0.92 Deciles 0.95 Ventiles 0.96
    - Tanzania: $1/day 75.39 — Quintiles 0.94 Deciles 0.96 Ventiles 0.96; $2/day 94.75 — Quintiles 0.97 Deciles 0.97 Ventiles 0.98
    - Additional poverty gap, squared gap, FGT(3), FGT(4) entries for Vietnam, Nicaragua, Tanzania with exact ratios (see table)

- Table 6: Bias of poverty measures (Gaussian kernel, Poverty line: Capability) — Bandwidth variants S1, S2, S3, S-J, Hybrid; ratios between estimated and survey
  - Example entries:
    - Poverty headcount ratio (%) Survey estimates: Vietnam 41.98 — S1 1.00 S2 1.00 S3 1.00 S-J 1.00 Hybrid 1.00
    - Nicaragua 30.61 — S1 1.12 S2 1.06 S3 1.05 S-J 1.07 Hybrid 1.05
    - Tanzania 40.13 — S1 1.04 S2 1.03 S3 1.02 S-J 1.03 Hybrid 1.03
    - Poverty gap ratio, Squared poverty gap, FGT(3), FGT(4): exact survey estimates and S1–Hybrid ratios for Vietnam, Nicaragua, Tanzania (see table for all entries)

- Figure 6: Survey-based and grouped data KDE-based density estimates
  - Panel comparisons for Tanzania, Nicaragua, Vietnam illustrating:
    - Effect of Bandwidth (S1 vs S3) — Gaussian kernel - Input data: Quintile means
    - Effect of Kernel (Gaussian vs Epanechnikov) — Hybrid bandwidth - Input data: Quintile means
    - Effect of Higher Number of Quantile Means — Quartic kernel - Input data: Quintile and Decile means - S3 bandwidth
    - Poverty lines in ascending order across panels: capability - $1/day - $2/day

- Table 7: Global poverty rates (% poor) — by bandwidth method and year (ratios and differences reported)
  - Bandwidths compared: S3, Overs-smoothed variant of S3, Sheather-Jones, Direct plug-in, Hybrid (Sala-i-Martin)
  - Year 1990 (examples):
    - $1/day rates: S3 7.2 Overs-smoothed 9.5 Variant of S3 6.4 Sheather-Jones 7.5 Direct plug-in 8.4 Hybrid 5.3 — Ratio b/w highest and lowest estimate 1.8 — Percentage point diff. b/w highest and lowest estimate 4.2
    - $1.5/day: S3 13.4 Overs-smoothed 16.2 Variant 12.8 Sheather-Jones 13.9 Direct plug-in 14.9 Hybrid 11.7 — Ratio 1.4 — Percentage point diff. 4.5
    - $2/day: S3 24.5 Overs-smoothed 26.8 Variant 24.2 Sheather-Jones 25.2 Direct plug-in 25.8 Hybrid 23.4 — Ratio 1.1 — Percentage point diff. 3.4
    - $3/day and $4/day: reported likewise with exact values and ratio/diff summaries
  - Year 2000 (examples):
    - $1/day: S3 5.3 Overs-smoothed 7.5 Variant 4.8 Sheather-Jones 5.6 Direct plug-in 6.2 Hybrid 4.2 — Ratio 1.8 — Percentage point diff. 3.3
    - $1.5/day: S3 9.4 Overs-smoothed 12.6 Variant 8.9 Sheather-Jones 10.0 Direct plug-in 10.7 Hybrid 6.9 — Ratio 1.8 — Percentage point diff. 5.7
    - $2/day: S3 17.2 Overs-smoothed 20.7 Variant 16.5 Sheather-Jones 17.7 Direct plug-in 18.7 Hybrid 15.0 — Ratio 1.4 — Percentage point diff. 5.7
    - $3/day and $4/day: reported with exact values and ratio/diff summaries

- Table 8: Global poverty counts (millions) — by bandwidth method and year; difference between highest and lowest estimate
  - Year 1990 (millions):
    - $1/day: S3 289 Overs-smoothed 381 Variant of S3 257 Sheather-Jones 303 Direct plug-in 338 Hybrid 213 — Difference b/w highest and lowest estimate 168
    - $1.5/day: S3 540 Overs-smoothed 651 Variant 518 Sheather-Jones 559 Direct plug-in 599 Hybrid 471 — Difference 180
    - $2/day: S3 987 Overs-smoothed 1079 Variant 975 Sheather-Jones 1016 Direct plug-in 1040 Hybrid 943 — Difference 136
    - $3/day: S3 1536 Overs-smoothed 1560 Variant 1524 Sheather-Jones 1533 Direct plug-in 1544 Hybrid 1496 — Difference 64
    - $4/day: S3 2008 Overs-smoothed 1989 Variant 2026 Sheather-Jones 2012 Direct plug-in 1998 Hybrid 2001 — Difference 37
  - Year 2000 (millions):
    - $1/day: S3 256 Overs-smoothed 362 Variant 232 Sheather-Jones 269 Direct plug-in 300 Hybrid 200 — Difference 162
    - $1.5/day: S3 452 Overs-smoothed 606 Variant 426 Sheather-Jones 481 Direct plug-in 517 Hybrid 333 — Difference 273
    - $2/day: S3 830 Overs-smoothed 998 Variant 796 Sheather-Jones 850 Direct plug-in 899 Hybrid 720 — Difference 278
    - $3/day: S3 1331 Overs-smoothed 1445 Variant 1319 Sheather-Jones 1341 Direct plug-in 1384 Hybrid 1235 — Difference 210
    - $4/day: S3 1833 Overs-smoothed 1893 Variant 1843 Sheather-Jones 1866 Direct plug-in 1870 Hybrid 1784 — Difference 109

*Referenced content and appendix figures/tables are reproduced from the provided content unit _wp08183 - REFERENCES.*

---


_Source: https://www.imf.org/-/media/websites/imf/imported-full-text-pdf/external/pubs/ft/wp/2008/_wp08183.pdf_