## IMF Working Paper — Harness Satellite Data to Improve Social Assistance Targeting in Eastern Caribbean

## Source details

**Canonical URL:** [IMF Working Paper — Harness Satellite Data to Improve Social Assistance Targeting in Eastern Caribbean](https://www.imf.org/-/media/files/publications/wp/2024/english/wpiea2024084-print-pdf.pdf)

## Other formats

- [Markdown version](/-/media/files/publications/wp/2024/english/wpiea2024084-print-pdf.pdf.md)
- [Structured JSON version](/-/media/files/publications/wp/2024/english/wpiea2024084-print-pdf.pdf.json)

---

### Introduction: role, challenges, and objectives
- Social assistance protects vulnerable populations but resources remain limited; prioritizing beneficiaries (targeting) is central due to:
  - Limited centralized data on income or household information.
  - Substantial informal sector making income difficult to verify (Jensen (2022) estimate: tax system includes only 1–10 percent of the economically active population in low-income countries and 30–50 percent in middle-income countries, compared to 90–95 percent in high-income countries).
  - High costs of regular surveys in small economies and limited administrative capacities.
- Region is highly exposed to hurricanes, earthquakes, and volcanic eruptions; rapid, accurate targeting is needed during shocks (COVID-19, cost-of-living crisis).
- Hypothesis and two research questions:
  - Hypothesis: Granular geospatial data from high-resolution satellite imagery analyzed with machine learning can improve targeting where traditional data are lacking.
  - Questions:
    1. Can satellite imagery accurately measure local well-being in the Eastern Caribbean?
    2. Can these measures improve targeting, and through which channels?
- HDI chosen as the outcome because it summarizes health, human capital, and standard of living; subnational HDI measures enable use in national social assistance policies.
- Methodological extension: provide parish- and 1 km x 1 km grid-level HDI predictions for Eastern Caribbean, validate against ground-truth microdata, and quantify improvements in social assistance targeting under geographic and hybrid designs.

### Data sources and coverage
- Countries/territories covered: Anguilla, Antigua and Barbuda, Dominica, Grenada, Monserrat, St. Kitts and Nevis, St. Lucia, St. Vincent and the Grenadines, and Trinidad and Tobago.
- Socio-economic data:
  - National-level outcomes (HDI, per capita income, health, education) from United Nations Development Program.
  - HDI components: health = life expectancy at birth; human capital = average years of schooling for adults aged 25+ and expected years of schooling for children; standard of living = log of GNI per capita; aggregation via geometric mean.
  - Subnational income microdata availability: St. Lucia (Population and Housing Census) and Grenada (Labor Force Survey).
  - Sherman et al. (2023) provide first-level administrative HDI estimates for many countries but excluded Anguilla, Monserrat, St. Kitts and Nevis, and St. Lucia.
- Satellite data:
  - MOSAIKS (daytime): global 0.01 x 0.01 equal-angle grid (~1 km x 1 km near equator); each tile has a 4,000-feature vector from random convolutions.
  - Nighttime lights: DMSP.
  - Typical parish aggregates two or three dozen MOSAIKS tiles; finest unit is 1 km x 1 km.

### Modeling approach and validation
- Core model: ridge regression of form Y = β0 + β1 X_MOSAIKS + β2 X_NL + ε, where Y is outcome (e.g., population density, income), X_MOSAIKS daytime features, X_NL nighttime lights.
- Aggregation/disaggregation:
  - Upscaling: population-weighted averages from fine to coarse units.
  - Downscaling: apply coefficients estimated at coarser resolutions to predict finer-resolution outcomes.
- Evaluation metrics: mean squared error (MSE), mean absolute error (MAE), and R-square; training/testing split with in-sample and out-of-sample evaluation.
- Eleven model variants trained (selection includes global-scale, by-continent, Barbados-based, Guadeloupe-based, Martinique-based, neighbors-based, national-level, subnational-level, Barbados EB, St. Lucia settlement). Model 9 (subnational-level model) selected as baseline for subnational and grid-level predictions.

### General model performance (summary)
- MOSAIKS (daytime) generally outperforms nighttime lights alone; combined MOSAIKS + nightlights outperforms either alone.
- Combined predictions explain between 53 percent and 92 percent of the in-sample variation across models and between 42 percent and 78 percent of the out-of-sample variation across models.
- Exceptions: Barbados enumeration block model and St. Lucia settlement model explain less than 10 percent of in-sample or out-of-sample variation.

### Key prediction results (population density and HDI)
- Population density:
  - National-level predictions (model 8, combined): in-sample R-square 0.80 and out-of-sample R-square 0.48.
  - Subnational-level predictions (model 9, combined): in-sample R-square 0.74 and out-of-sample R-square 0.70.
  - In Eastern Caribbean validation sample, models 8 and 9 each explain 27 percent of variation in population density.
  - Subnational-level model (model 9) outperforms model 8 for out-of-sample accuracy and stability across aggregation levels.
  - Appendix table highlights specific MSE and R-square by model (examples preserved exactly):
    - Global-scale, in-sample: MSE 1.092, R-square 0.624 (MOSAIKS); Both MSE 0.984, R-square 0.661.
    - By-continent, in-sample: MSE 0.324, R-square 0.726 (MOSAIKS); Both MSE 0.248, R-square 0.790.
    - Subnational-level, out-of-sample: MSE 1.017, R-square 0.673 (MOSAIKS); Both MSE 0.943, R-square 0.697.
    - Barbados EB and St. Lucia settlement models: in-sample R-square 0.088 and 0.034 respectively; out-of-sample R-square 0.020 and -0.022 respectively.
- HDI and components:
  - National-level HDI predictions (combined MOSAIKS + nightlights) explain over 80 percent of in-sample and out-of-sample variation across countries in many specifications (Table examples preserved exactly):
    - HDI, National-level, in-sample: Both MSE 0.003, R-square 0.867.
    - HDI, National-level, out-of-sample: Both MSE 0.004, R-square 0.811.
    - HDI, Subnational-level, out-of-sample: Both MSE 0.003, R-square 0.858.
  - For the Eastern Caribbean regional validation:
    - Model 9 explains 45 percent of HDI variation in the region; model 8 explains 12 percent.
    - Validation against ground truth:
      - St. Lucia (district level, 2010 Census): correlation coefficients between predicted and census income: 0.57 (predicted HDI), 0.62 (predicted GNI), 0.62 (predicted income).
      - Grenada (parish level, 2019Q4 Labor Force Survey): correlation coefficient 0.31.
  - Grid-level: model 9 applied to generate 1 km x 1 km HDI predictions for simulations.

### Policy simulation design and principal results
- Objectives: use grid-level predicted HDI as treatment "ground truth" to evaluate targeting under alternative information and eligibility rules through three experiments:
  1. Reduce aggregation bias in pure geographic targeting (Policies 1 vs. 2).
  2. Improve allocation of resources across areas under budget constraint (Policies 3 vs. 4).
  3. Proxy hard-to-verify information in hybrid targeting (Policies 5 vs. 6).
- Targeting accuracy measured by exclusion and inclusion errors:
  - Exclusion error = FN / (FN + TP)
  - Inclusion error = FP / (FP + TN)
  - Use exclusion‒inclusion curve (EIC) mapping all eligibility thresholds.
- Key simulation findings (preserve exact outcomes where provided):
  - Reducing aggregation bias:
    - For St. Lucia, Policy 2 (subnational information) EIC lies strictly below Policy 1 (national information) EIC for all eligibility thresholds—parish-level information outperforms national-level information.
    - Granular information reduces aggregation bias, especially where low-HDI localities are adjacent to high-HDI localities.
  - Improving resource allocation:
    - Policy 4 (parish-specific percentile rules with aggregate caseload parity and equal absolute HDI threshold) outperforms Policy 3 (nationally defined percentile rule) in St. Lucia; Policy 4’s EIC lies strictly below Policy 3’s EIC for all eligibility thresholds.
    - Locality-specific eligibility enables proportional allocation of caseloads to need and better administrative resource targeting.
  - Proxying hard-to-verify information:
    - Proxy variable calibrated to have a 0.25 correlation with ground truth.
    - Policy 6 (proxy + parish-level HDI combined score with equal weights) reduces targeting error relative to Policy 5 (proxy alone); Policy 6’s EIC lies below Policy 5’s EIC for almost all percentile cutoffs in St. Lucia.
    - Across sampled Eastern Caribbean countries, Policy 6 outperforms Policy 5, with the improvement magnitude depending on the parish-level HDI correlation with ground truth.
- Calibration and context:
  - Informal sector estimated on average 35 percent of the economy in sample countries (Peters 2017; IMF 2022).

### Policy implications and operational recommendations
- Satellite- and ML-predicted local well-being can:
  - Reduce aggregation bias in geographic targeting by providing fine-resolution information (parish- and 1 km x 1 km grid-level).
  - Improve allocation of program and administrative resources via locality-specific eligibility rules and proportional caseload assignment.
  - Complement existing proxies and proxy hard-to-verify variables (e.g., informal income) in hybrid targeting schemes.
- Operational recommendations:
  - Use parish-level or other subnational predictions when within-country heterogeneity is substantial.
  - Under budget constraints, consider parish-specific percentile rules while preserving equity via absolute thresholds and aggregate caseload parity.
  - Combine local HDI predictions with existing proxy indicators to improve ranking accuracy; weighting schemes can be adjusted beyond equal weights.
  - Collect periodic ground-truth survey data on household welfare to validate and monitor satellite-informed mappings.
- Extension suggested: apply satellite-based predictive approaches to emergent post-disaster social assistance by predicting vulnerability to specific hazards.

### Appendix highlights (selected exact metrics)
- Combined MOSAIKS + nightlights explain between 53 percent and 92 percent of in-sample variation and between 42 percent and 78 percent of out-of-sample variation across models.
- Selected table entries preserved exactly:
  - Population density, Global-scale, in-sample (Both): MSE 0.984, R-square 0.661.
  - Population density, By-continent, in-sample (Both): MSE 0.248, R-square 0.790.
  - Population density, Subnational-level, out-of-sample (Both): MSE 0.943, R-square 0.697.
  - HDI, National-level, in-sample (Both): MSE 0.003, R-square 0.867.
  - HDI, National-level, out-of-sample (Both): MSE 0.004, R-square 0.811.
  - HDI, Subnational-level, out-of-sample (Both): MSE 0.003, R-square 0.858.
- Table A1: Correlation (information value of subnational data) by country:
  - ATG: 0.45
  - DMA: 0.24
  - GRD: 0.16
  - KNA: 0.38
  - LCA: 0.48
  - TTO: 0.26
  - VCT: 0.19

*Source: IMF Working Paper — Harness Satellite Data to Improve Social Assistance Targeting in Eastern Caribbean (wpiea2024084-print-pdf).*

### 1. Introduction ........................................................................................................

### 1. Introduction

### Role and challenges of social assistance in the Eastern Caribbean
- Social assistance programs protect vulnerable populations and mitigate poverty, but resources for social assistance remain limited.
- Prioritizing beneficiaries (targeting) is a central policy challenge in the Eastern Caribbean due to:
  - Limited availability of centralized data on income or household information.
  - A substantial informal sector that makes income difficult to verify (Jensen (2022) estimate that the tax system includes only 1–10 percent of the economically active population in low-income countries and 30–50 percent in middle-income countries, compared to 90–95 percent in high-income countries).
  - High costs of conducting regular surveys in small economies and limited administrative capacities.
- The need for rapid, accurate targeting intensifies during large economic and natural disaster shocks, including the COVID-19 pandemic and cost-of-living crisis; the region is highly exposed to hurricanes, earthquakes, and volcanic eruptions.

### Common targeting approaches and limitations
- Targeting commonly relies on:
  - Location characteristics (e.g., areas hit by natural disasters).
  - Demographic characteristics (e.g., older people, households with young children).
  - Individual and household characteristics (e.g., education and household assets).
  - Tacit local knowledge (e.g., authorities knowing specific informal settlements are vulnerable).
- In countries with extensive, periodic, representative household surveys, econometric models can be used to estimate poverty and vulnerability. Where such surveys are lacking, estimates are often extrapolated from past or peer-country experience, producing potentially large targeting errors.
- Conventional targeting systems are estimated to incur more than 25 percent exclusion errors (under-coverage) and inclusion errors (leakage) when compared to ground truth survey data (McBride et al. 2016); errors for countries lacking survey data are unknown and could be larger.

### Use of satellite imagery and machine learning: objectives of this paper
- Hypothesis: Granular geospatial data from high-resolution satellite imagery, analyzed with machine learning, can improve targeting in the Eastern Caribbean by providing local information where traditional data are lacking.
- Two specific questions:
  1. Can satellite imagery be used to accurately measure local well-being in the Eastern Caribbean?
  2. Can these measures improve targeting, and through which channels?

### Methodological extension and scope
- Builds on Sherman et al. (2023), which provides global estimates of the United Nations Human Development Index (HDI) at second-level administrative units.
- Extensions in this paper:
  - Provide comprehensive parish- and 1 square kilometer grid-level HDI predictions for Eastern Caribbean countries and territories, filling coverage gaps in Sherman et al. (2023).
  - Validate model performance against ground-truth microdata from the region.
  - Quantify how these estimates can improve social assistance targeting under various geographic and hybrid targeting designs.
- Notes on coverage and terminology:
  - Sherman et al. (2023) do not include Anguilla, Monserrat, St. Kitts and Nevis, and St. Lucia in their estimation; they provide country-level estimates for the rest of the sample.
  - Level 1 administrative boundaries are called “parishes” in Antigua and Barbuda, Barbados, Dominica, Grenada, St Kitts and Nevis, St. Vincent and the Grenadines, and “districts” in St. Lucia; the paper uses parish and district interchangeably.

### Human Development Index (HDI) rationale
- HDI summarizes three dimensions of well-being: health, human capital, and standard of living.
- HDI is more comprehensive than income or wealth alone and is widely used for development assistance and resource allocation.
- Subnational HDI measures are necessary to facilitate use of HDI in national social assistance policies.
- Approach uses a downscaling technique (train on province-level data, predict at finer geographic levels) and evaluates a large set of prediction models, including the downscaling model.

### Key empirical findings (summary)
- National-level HDI estimations are well-correlated with ground-truth HDI data at the country level in the Eastern Caribbean.
- Parish- and district-level HDI predictions are strongly and positively correlated with income and other well-being indicators in countries where disaggregated data exist, suggesting parish/district-level HDI can be a good proxy for local well-being.
- One kilometer by one kilometer grid-level estimations provide substantially more detailed understanding of local well-being.

### Policy simulation exercises and principal results
- Overall approach: simulate policymaker scenarios distributing social assistance benefits to those most in need; consider geographic targeting, allocation of resources across areas, and hybrid targeting that combines verifiable and imputed information.
- Simulation 1 — Reducing aggregation bias in pure geographic targeting:
  - Using subnational-level well-being measures to select program beneficiaries significantly reduces targeting errors compared to using national-level measures alone.
  - Improvement arises because granular geospatial information provides more accurate proxies for social assistance need than aggregated data.
- Simulation 2 — Improving allocation of resources across geographic areas:
  - Programs with eligibility rules informed by local well-being estimates have smaller targeting errors than programs with uniform eligibility rules.
  - Local well-being estimates enable proportional allocation of caseloads to need and can also inform allocation of administrative resources.
- Simulation 3 — Proxying for hard-to-verify information (hybrid targeting):
  - In scenarios where eligibility is based on verifiable proxies (e.g., demographics, formal income), adding information on average local well-being reduces targeting errors.
  - Local well-being can proxy hard-to-verify information such as informal income and complement other socioeconomic indicators and tacit local knowledge, or be used to verify and complement such knowledge.

### Organization of the paper
- Section 2: Related literature.
- Section 3: Data and methodology.
- Section 4: Model performance comparison and baseline estimations.
- Section 5: Simulation results and policy implications.
- Section 6: Conclusion.

*IMF Working Paper — Harness Satellite Data to Improve Social Assistance Targeting in Eastern Caribbean*

### 3. Data and Methodology

### 3. Data and Methodology

### Overview
- Objective: Predict well-being in the Eastern Caribbean at granular levels by combining microdata and high-resolution satellite data (SIML).
- Countries/territories covered: Anguilla, Antigua and Barbuda, Dominica, Grenada, Monserrat, St. Kitts and Nevis, St. Lucia, St. Vincent and the Grenadines, and Trinidad and Tobago.

### Data description

H3: Socio-economic Data
- National-level socio-economic outcomes (HDI, per capita income, health, education) sourced from the United Nations Development Program.
- HDI composition:
  - Health: life expectancy at birth (proxy).
  - Human capital: average years of schooling for adults aged 25+ and expected years of schooling for children of school entering age.
  - Standard of living: log of gross national income (GNI) per capita.
  - Aggregation method: geometric mean of three sub-indexes.
- Subnational data availability:
  - Income microdata available for St. Lucia (Population and Housing Census) and Grenada (Labor Force Survey).
  - Subnational-level HDI not available for Anguilla, Monserrat, St. Kitts and Nevis, and St. Lucia.
  - Sherman et al. (2023) provide first-level administrative unit HDI estimates for other countries/territories in region.

H3: Satellite data
- Two satellite datasets combined:
  - MOSAIKS (Multi-task Observation using Satellite Imagery & Kitchen Sinks).
  - Nighttime light data from the U.S. Air Force Defense Meteorological Satellite Program (DMSP).
- MOSAIKS specifics:
  - Organized on a global 0.01 x 0.01 equal-angle grid (roughly 1 kilometer by 1 kilometer near the equator).
  - Each tile associated with a vector of 4,000 task-agnostic features computed using random convolutions.
  - Typical parish aggregates two or three dozen MOSAIKS tiles.
  - Finest geographic unit: 1km x 1km grid-level MOSAIKS features.
- Nighttime lights used to complement MOSAIKS daytime features.

### Methodology

H3: Modeling approach
- Core strategy: Train ridge regression models on countries with available outcome data using MOSAIKS and nighttime light features, then apply estimated feature weights to predict outcomes in countries/geographies lacking data.
- Spatial matching: Align outcome data, MOSAIKS features, and nighttime light features so units match before training.
- Model equation (baseline):
  - Y = β0 + β1 X_MOSAIKS + β2 X_NL + ε
  - Where Y is the outcome variable (e.g., population density, income), X_MOSAIKS is the matrix of daytime MOSAIKS features, and X_NL is the matrix of nighttime light features.
  - Coefficients β0, β1, β2 estimated via ridge regression.
- Aggregation/disaggregation:
  - Upscaling: apply finer-resolution estimates to generate coarser-resolution predictions (e.g., grid → national) using population-weighted averages.
  - Downscaling: apply coefficients estimated at coarser resolutions to predict finer-resolution outcomes (e.g., national-level outcome → subnational or grid-level predictions).

H3: Evaluation metrics and validation procedure
- Metrics used: mean squared error (MSE), mean absolute error (MAE), and R-square.
  - MSE measures variance of residuals.
  - MAE measures average of residuals.
  - MSE, MAE, and R-square combined provide assessment of prediction accuracy vs. actual values.
- Training/testing split:
  - Dataset split into training and testing samples; evaluate in-sample and out-of-sample performance using MSE and R-square.
- Validation:
  - Compare predicted aggregated national-level outcomes to actual national values where available.
  - Validate subnational predictions against ground truth where available (e.g., St. Lucia, Grenada).
  - Use additional HDI components (health, education, income per capita, log GNI) to assess which well-being dimensions are captured.

### Model estimation and selection

H3: Models trained
- Eleven models with variation in aggregation level and countries included:
  1. Global scale model: training sample includes a random sample of 1 million grid points across the world.
  2. By-continent model: same as model 1 but split into 6 continents and trained by continent.
  3. By-continent fixed model: same as 2 but classifies all Eastern Caribbean countries on the North American continent instead of the South American continent.
  4. Barbados-based model: training sample includes all grid points in Barbados.
  5. Guadeloupe-based model: same as 4 but using Guadeloupe data.
  6. Martinique-based model: same as 4 but using Martinique data.
  7. Neighbors-based model: training sample combines Barbados, Guadeloupe, and Martinique.
  8. National-level model: training sample is the area-weighted national average for 179 countries.
  9. Subnational-level model: training sample is the area-weighted average at the subnational level (first-level administrative units) for 179 countries.
  10. Barbados enumeration block (EB) model: training sample consists of enumeration blocks from the Barbados Survey of Living Conditions.
  11. St. Lucia settlement model: training sample consists of settlements from the St. Lucia Population and Housing Census.
- Units of observation and Voronoi polygon matching used for specific enumeration/settlement models where noted.

H3: General findings from model comparisons
- MOSAIKS (daytime) predictions generally outperform nighttime lights alone.
- Combined MOSAIKS + nighttime lights outperform either dataset alone.
- Combined predictions explain:
  - Between 53 percent and 92 percent of the in-sample variation across models.
  - Between 42 percent and 78 percent of the out-of-sample variation across models.
- Exceptions:
  - Barbados enumeration block model and St. Lucia settlement model explain less than 10 percent of in-sample or out-of-sample variation, possibly due to matching errors when polygons are irregular or small.

### Prediction results

H3: Population density predictions
- National-level validation:
  - Upscaling used to generate national-level predictions for models except model 8 (same-scale for model 8).
  - National-level and subnational-level models (models 8 and 9) perform very well compared to other models.
    - Using combined MOSAIKS and nightlights:
      - National-level predictions (model 8) explain 80 percent (in-sample) and 48 percent (out-of-sample) of variation.
      - Subnational-level predictions (model 9) explain 74 percent (in-sample) and 70 percent (out-of-sample) of variation.
  - In Eastern Caribbean validation sample (Antigua and Barbuda, Dominica, Grenada, St. Kitts and Nevis, St. Lucia, St. Vincent and the Grenadines, Trinidad and Tobago):
    - Models 8 and 9 each explain 27 percent of the variation (population density).
- Subnational-level validation:
  - Global-scale models (models 1-3) explain over 70 percent of variation at subnational level versus 30 to 35 percent at national level.
  - Subnational-level model (model 9) explains 47 percent of subnational variation (an improvement from 27 percent at national level).
  - Overall: model 9 outperforms model 8 for out-of-sample accuracy and stability across aggregation levels.

H3: HDI predictions
- National-level HDI prediction:
  - Variants of models 8 and 9 trained and evaluated; training sample for model 8 is area-weighted national averages, for model 9 a mix of subnational and national averages where subnational HDI unavailable.
  - Combined MOSAIKS + nightlights:
    - Predictions from both models explain over 80 percent of in-sample and out-of-sample variations across countries.
    - For other well-being measures, predictions explain over 75 percent of in-sample variation and over 60 percent of out-of-sample variation.
    - Exception: model 8 explaining health out-of-sample is 46 percent.
  - Regional validation (Eastern Caribbean):
    - Model 9 explains 45 percent of HDI variation in the region.
    - Model 8 explains 12 percent of HDI variation in the region.
    - Model 9 explains between 30 to 42 percent of variation for other measures; education is an exception where model 9 explains only 9 percent of variation.
- Subnational-level HDI prediction:
  - Based on prior results, model 9 selected as baseline for subnational HDI prediction and grid-level downscaling.
  - Validation against ground truth:
    - St. Lucia (district level, 2010 Population and Housing Census):
      - Correlation coefficients between predicted and census income: 0.57 (predicted HDI), 0.62 (predicted GNI), 0.62 (predicted income).
    - Grenada (parish level, 2019Q4 Labor Force Survey):
      - Correlation coefficient: 0.31.
  - Grid-level predictions:
    - Model 9 applied with a downscaling exercise to generate 1km x 1km grid-level HDI predictions for policy simulations.

*Source: IMF Working Paper — 3. Data and Methodology*

### 5. Policy Simulations

### 5. Policy Simulations

### Objectives and approach
- Examine how predicted local HDI (grid-level) can improve targeting of social assistance in the Eastern Caribbean.
- Conduct three simulation experiments to illustrate channels through which granular predictions help:
  - Reduce aggregation bias in geographic targeting.
  - Improve allocation of resources across geographic areas.
  - Proxy hard-to-verify information in a hybrid targeting setting.
- Use grid-level predicted values as treatment "ground truth" and compare simulations under different information granularity and rules.

### Reducing aggregation bias in geographic targeting
- Context:
  - Geographic targeting assigns benefits to neediest areas when household-level data are unavailable.
  - Aggregation bias arises when small geographic units are assigned a larger unit’s information despite within-unit heterogeneity (e.g., poor rural interior vs. well-off coastal strip).
- Simulation design:
  - Eligibility rule based on absolute HDI values: program selects localities with HDI below a threshold (relevant when budget is flexible).
  - Two policy scenarios differing by available information granularity:
    - Policy 1 (national information): simulate grid-level HDI in each parish assuming a log-normal distribution with mean and standard deviation matched to national-level statistics.
    - Policy 2 (subnational information): simulate grid-level HDI in each parish assuming a log-normal distribution with mean and standard deviation matched to parish-level statistics.
- Targeting accuracy metrics:
  - Exclusion and inclusion errors defined as:
    - 퐸푥푐푙푢푠푖표푛 푒푟푟표푟 = 퐹푁 / (퐹푁 + 푇푃)
    - 퐼푛푐푙푢푠푖표푛 푒푟푟표푟 = 퐹푃 / (퐹푃 + 푇푁)
    - where 푇푃, 푇푁, 퐹푃, 퐹푁 denote true positives, true negatives, false positives, and false negatives respectively.
  - Use exclusion‒inclusion curve (EIC) mapping full space of eligibility thresholds (from no beneficiaries to universal coverage).
- Key findings:
  - For St. Lucia, Policy 2’s EIC lies strictly below Policy 1’s EIC for all eligibility thresholds—parish-level information outperforms national-level information.
  - Granular information reduces aggregation bias, especially for localities with low HDI adjacent to high-HDI localities that would be missed using aggregate data.
  - Cross-country variation exists: countries with greater within-country variation in local well-being gain more from subnational data (see correlation comparisons referenced).

### Improving resource allocation (relative eligibility under budget constraint)
- Context:
  - Eligibility rule based on relative HDI values (percentiles) appropriate when beneficiary count is constrained by budget.
- Simulation design (both use parish-level mean and standard deviation information):
  - Policy 3 (nationally defined eligibility rule): In each parish, localities in the lowest national 푃 percentile are eligible (equal coverage across parishes).
  - Policy 4 (parish-specific eligibility rule): Localities in the lowest parish-specific percentile are eligible; two constraints imposed:
    - Policy 4 selects the same total number of beneficiaries as Policy 3.
    - Selected beneficiaries satisfy the same absolute HDI threshold (equity consideration: a locality chosen in one parish implies a locality with same HDI in another parish would also be chosen).
- Key findings:
  - For St. Lucia, Policy 4’s EIC lies strictly below Policy 3’s EIC for all eligibility thresholds—parish-specific eligibility outperforms uniformly applied national-percentile rules.
  - Implication: allocating caseloads and administrative resources proportional to local needs (using local well-being predictions) improves targeting.
  - Mechanism: Policy 3 enforces equal coverage leading to higher absolute HDI thresholds in better-off districts and lower thresholds in worse-off districts; Policy 4 adjusts coverage to reflect cross-district HDI differences, eliminating high-HDI beneficiaries in better-off districts and admitting low-HDI beneficiaries in worse-off districts.
  - Appendix indicates qualitatively similar results across other countries.

### Proxying for hard-to-verify information in hybrid targeting
- Context:
  - Policymakers often use imperfect proxies (demographic indicators, employment indicators, partial income) to rank need; proxies are imperfect relative to true need.
  - Investigate whether adding local HDI predictions to existing proxies reduces targeting error.
- Simulation design:
  - Simulated a proxy variable partially correlated with the ground truth to capture existing proxies.
  - Two policies compared:
    - Policy 5 (proxy targeting): select beneficiaries using proxy variable; those below a percentile are eligible.
    - Policy 6 (proxy + parish-level information): rank population by a combined score of parish-level predicted HDI and the proxy variable; those below a percentile are eligible. (Equal weights used in simulation; results generalize to other weighting schemes.)
  - Calibration: proxy variable set to have a 0.25 correlation with the ground truth (value in range of targeting errors in literature).
- Key findings:
  - Combining local well-being with proxies (Policy 6) reduces targeting error relative to using proxy alone (Policy 5): Policy 6’s EIC lies below Policy 5’s EIC for almost all percentile cutoffs in St. Lucia.
  - Cross-country: Policy 6 outperforms Policy 5 in all sampled Eastern Caribbean countries, with extent of improvement varying by how well parish-level HDI correlates with ground truth.
  - Interpretation: local well-being measures can proxy for unobserved/hard-to-verify information (e.g., informal income), and when combined with formal income or proxies they improve beneficiary selection.

### Policy implications and recommendations (derived from simulations)
- Local well-being measures predicted from satellite and ML methods can:
  - Reduce aggregation bias in geographic targeting by providing fine-resolution information.
  - Improve allocation of program and administrative resources by informing locality-specific eligibility and caseload distribution.
  - Complement existing proxies and proxy hard-to-verify variables (e.g., informal income) in hybrid targeting schemes.
- Operational recommendations:
  - Use parish-level or other subnational predictions when within-country heterogeneity is substantial.
  - When budgets constrain coverage, consider parish-specific percentile rules while preserving equity via absolute thresholds and aggregate caseload parity.
  - Combine local HDI predictions with existing proxy indicators to improve ranking accuracy; weighting schemes can be adjusted beyond equal weights.
  - Collect periodic ground-truth survey data with directly measured household welfare to validate and assess satellite-informed local well-being mappings.
- Extension:
  - Apply similar satellite-based predictive approaches to emergent social assistance after natural disasters by predicting vulnerability to specific hazards.

### Key numeric and calibration details preserved
- Proxy variable calibrated to have a 0.25 correlation with the ground truth.
- Informal sector estimated on average 35 percent of the economy in sample countries (Peters 2017; IMF 2022).

*Source: IMF Working Paper — 5. Policy Simulations (Harness Satellite Data to Improve Social Assistance Targeting in Eastern Caribbean).*

### Appendix 1: Figures and Tables

### Appendix 1: Figures and Tables

### Population density prediction models — in-sample and out-of-sample performance
- Table: Population density predictions across models: in-sample performance metrics (MSE and R-square reported exactly)
  - Global-scale, Random coordinates: MSE 1.092, R-square 0.624 (MOSAIKS); MSE 2.180, R-square 0.249 (Nightlights); MSE 0.984, R-square 0.661 (Both)
  - By-continent, Random coordinates: MSE 0.324, R-square 0.726 (MOSAIKS); MSE 0.523, R-square 0.557 (Nightlights); MSE 0.248, R-square 0.790 (Both)
  - By-continent fixed, Random coordinates: MSE 0.888, R-square 0.470; MSE 1.227, R-square 0.267; MSE 0.793, R-square 0.526
  - Barbados-based MOSAIKS, coordinates: MSE 0.224, R-square 0.880; MSE 1.472, R-square 0.210; MSE 0.149, R-square 0.920
  - Guadeloupe-based MOSAIKS, coordinates: MSE 0.640, R-square 0.588; MSE 1.311, R-square 0.157; MSE 0.670, R-square 0.570
  - Martinique-based MOSAIKS, coordinates: MSE 0.905, R-square 0.620; MSE 1.962, R-square 0.175; MSE 0.831, R-square 0.651
  - Neighbors-based MOSAIKS, coordinates: MSE 0.810, R-square 0.616; MSE 1.704, R-square 0.191; MSE 0.725, R-square 0.656
  - National-level, National polygons: MSE 0.312, R-square 0.783; MSE 1.287, R-square 0.106; MSE 0.284, R-square 0.803
  - Subnational-level, Subnational polygons: MSE 0.844, R-square 0.698; MSE 2.158, R-square 0.229; MSE 0.728, R-square 0.740
  - Barbados Enumeration blocks: MSE 7.408, R-square 0.088; MSE 6.592, R-square 0.185; MSE 7.294, R-square 0.102
  - St. Lucia settlement, Settlements: MSE 8.397, R-square 0.034; MSE 7.615, R-square 0.146; MSE 8.300, R-square 0.046

- Table: Population density predictions across models: out-of-sample performance metrics (MSE and R-square reported exactly)
  - Global-scale, Random coordinates: MSE 1.105, R-square 0.621; MSE 2.186, R-square 0.250; MSE 0.995, R-square 0.659
  - By-continent, Random coordinates: MSE 0.343, R-square 0.713; MSE 0.536, R-square 0.551; MSE 0.262, R-square 0.781
  - By-continent fixed, Random coordinates: MSE 0.953, R-square 0.421; MSE 1.248, R-square 0.242; MSE 0.853, R-square 0.482
  - Barbados-based MOSAIKS, coordinates: MSE 0.992, R-square 0.548; MSE 1.889, R-square 0.138; MSE 0.757, R-square 0.655
  - Guadeloupe-based MOSAIKS, coordinates: MSE 1.023, R-square 0.490; MSE 1.613, R-square 0.196; MSE 0.917, R-square 0.543
  - Martinique-based MOSAIKS, coordinates: MSE 1.551, R-square 0.467; MSE 2.536, R-square 0.128; MSE 1.513, R-square 0.480
  - Neighbors-based MOSAIKS, coordinates: MSE 1.169, R-square 0.393; MSE 1.699, R-square 0.118; MSE 1.115, R-square 0.421
  - National-level, National polygons: MSE 1.092, R-square 0.527; MSE 1.946, R-square 0.158; MSE 1.202, R-square 0.480
  - Subnational-level, Subnational polygons: MSE 1.017, R-square 0.673; MSE 2.266, R-square 0.271; MSE 0.943, R-square 0.697
  - Barbados EB, Enumeration blocks: MSE 7.450, R-square 0.020; MSE 7.041, R-square 0.076; MSE 7.449, R-square 0.020
  - St. Lucia settlement, Settlements: MSE 8.685, R-square -0.022; MSE 8.021, R-square 0.026; MSE 8.705, R-square -0.024

### Validation of national- and subnational-level population density predictions (Eastern Caribbean)
- National-level validation (summary statistics and performance metrics across models)
  - Ground-truth: Mean 5.346, Variance 0.146
  - Global-scale: Mean 4.705, Variance 1.132, MSE 1.092, R-square 0.353
  - By-continent: Mean 4.772, Variance 1.433, MSE 1.266, R-square 0.283
  - By-continent fixed: Mean 4.228, Variance 0.831, MSE 1.764, R-square 0.294
  - Barbados-based: Mean 6.193, Variance 0.752, MSE 1.994, R-square 0.793
  - Guadeloupe-based: Mean 4.937, Variance 0.020, MSE 0.285, R-square 0.078
  - Martinique-based: Mean 5.424, Variance 0.123, MSE 0.115, R-square 0.321
  - Neighbors-based: Mean 5.185, Variance 0.129, MSE 0.187, R-square 0.102
  - National-level model: Mean 5.451, Variance 0.339, MSE 0.226, R-square 0.277
  - Subnational-level model: Mean 5.053, Variance 0.436, MSE 0.360, R-square 0.271
  - Barbados EB: Mean 4.983, Variance 0.044, MSE 0.345, R-square 0.131
  - St. Lucia settlement: Mean 3.512, Variance 0.079, MSE 3.575, R-square 0.009

- Subnational-level validation (summary statistics and performance metrics across models)
  - Ground-truth: Mean 5.357, Variance 0.884
  - Global-scale: Mean 4.328, Variance 2.723, MSE 1.946, R-square 0.761
  - By-continent: Mean 4.329, Variance 2.421, MSE 1.895, R-square 0.704
  - By-continent fixed: Mean 4.031, Variance 1.996, MSE 2.390, R-square 0.710
  - Barbados-based: Mean 6.156, Variance 1.075, MSE 3.101, R-square 0.076
  - Guadeloupe-based: Mean 4.945, Variance 0.224, MSE 0.711, R-square 0.394
  - Martinique-based: Mean 5.124, Variance 0.182, MSE 0.716, R-square 0.242
  - Neighbors-based: Mean 5.136, Variance 0.421, MSE 0.508, R-square 0.473
  - National-level model: Mean 5.578, Variance 1.151, MSE 2.074, R-square 0.000
  - Subnational-level model: Mean 4.833, Variance 1.171, MSE 0.929, R-square 0.467
  - Barbados EB: Mean 4.947, Variance 0.395, MSE 1.211, R-square 0.035
  - St. Lucia settlement: Mean 3.476, Variance 0.307, MSE 4.263, R-square 0.191

### HDI and component predictions — model performance
- Table 5: HDI predictions at the national level: in-sample performance metrics (MSE and R-square preserved)
  - HDI, National-level: MOSAIKS MSE 0.003, R-square 0.894; Nightlights MSE 0.008, R-square 0.657; Both MSE 0.003, R-square 0.867
  - HDI, Subnational-level: MOSAIKS MSE 0.004, R-square 0.860; Nightlights MSE 0.010, R-square 0.618; Both MSE 0.003, R-square 0.888
  - GNI, National-level: MOSAIKS MSE 92.373, R-square 0.768; Nightlights MSE 238.488, R-square 0.402; Both MSE 100.321, R-square 0.748
  - GNI, Subnational-level: MOSAIKS MSE 44.633, R-square 0.817; Nightlights MSE 128.099, R-square 0.474; Both MSE 39.572, R-square 0.838
  - Health, National-level: MOSAIKS MSE 0.003, R-square 0.763; Nightlights MSE 0.006, R-square 0.576; Both MSE 0.003, R-square 0.805
  - Health, Subnational-level: MOSAIKS MSE 0.003, R-square 0.807; Nightlights MSE 0.008, R-square 0.483; Both MSE 0.003, R-square 0.816
  - Income, National-level: MOSAIKS MSE 0.009, R-square 0.734; Nightlights MSE 0.012, R-square 0.618; Both MSE 0.006, R-square 0.824
  - Income, Subnational-level: MOSAIKS MSE 0.006, R-square 0.812; Nightlights MSE 0.013, R-square 0.583; Both MSE 0.005, R-square 0.850
  - Ed, National-level: MOSAIKS MSE 0.004, R-square 0.883; Nightlights MSE 0.013, R-square 0.578; Both MSE 0.005, R-square 0.844
  - Ed, Subnational-level: MOSAIKS MSE 0.006, R-square 0.833; Nightlights MSE 0.016, R-square 0.576; Both MSE 0.005, R-square 0.859

- Table 6: HDI predictions at the national level: out-of-sample performance metrics
  - HDI, National-level: MOSAIKS MSE 0.005, R-square 0.778; Nightlights MSE 0.006, R-square 0.718; Both MSE 0.004, R-square 0.811
  - HDI, Subnational-level: MOSAIKS MSE 0.005, R-square 0.813; Nightlights MSE 0.010, R-square 0.590; Both MSE 0.003, R-square 0.858
  - GNI, National-level: MOSAIKS MSE 170.990, R-square 0.534; Nightlights MSE 161.413, R-square 0.561; Both MSE 140.825, R-square 0.617
  - GNI, Subnational-level: MOSAIKS MSE 108.781, R-square 0.656; Nightlights MSE 206.053, R-square 0.348; Both MSE 105.854, R-square 0.665
  - Health, National-level: MOSAIKS MSE 0.006, R-square 0.390; Nightlights MSE 0.004, R-square 0.546; Both MSE 0.005, R-square 0.457
  - Health, Subnational-level: MOSAIKS MSE 0.004, R-square 0.754; Nightlights MSE 0.008, R-square 0.454; Both MSE 0.003, R-square 0.767
  - Income, National-level: MOSAIKS MSE 0.013, R-square 0.586; Nightlights MSE 0.008, R-square 0.730; Both MSE 0.008, R-square 0.733
  - Income, Subnational-level: MOSAIKS MSE 0.008, R-square 0.738; Nightlights MSE 0.013, R-square 0.545; Both MSE 0.006, R-square 0.792
  - Ed, National-level: MOSAIKS MSE 0.007, R-square 0.765; Nightlights MSE 0.012, R-square 0.594; Both MSE 0.006, R-square 0.813
  - Ed, Subnational-level: MOSAIKS MSE 0.008, R-square 0.780; Nightlights MSE 0.016, R-square 0.533; Both MSE 0.006, R-square 0.825

- Table 7: Validating national-level HDI predictions in the Eastern Caribbean (MSE and R-square)
  - HDI, National-level: MSE 0.002, R-square 0.117
  - HDI, Subnational-level: MSE 0.001, R-square 0.452
  - GNI, National-level: MSE 33.964, R-square 0.445
  - GNI, Subnational-level: MSE 36.740, R-square 0.362
  - Health, National-level: MSE 0.003, R-square 0.005
  - Health, Subnational-level: MSE 0.001, R-square 0.416
  - Income, National-level: MSE 0.003, R-square 0.312
  - Income, Subnational-level: MSE 0.004, R-square 0.293
  - Education, National-level: MSE 0.005, R-square 0.153
  - Education, Subnational-level: MSE 0.005, R-square 0.085

### Policy experiments — Exclusion-Inclusion Curves (EIC) and illustrative outcomes
- Figures document EICs and simulated HDI distributions for multiple policy experiments:
  - Policies 1 and 2: Figure 7 presents EIC for Policies 1 and 2; Figure 8 shows illustrative outcomes (Panel A: Policy 1; Panel B: Policy 2) plotting distribution of simulated St. Lucia HDI.
  - Policies 3 and 4: Figure 1 (page 35) presents EIC for Policies 3 and 4; Figure 10 shows illustrative outcomes (Panel A: Policy 3; Panel B: Policy 4) plotting distribution of simulated St. Lucia HDI.
  - Policies 5 and 6: Figure 11 presents EIC for Policies 5 and 6.
- Note: All policy EICs and illustrative outcome distributions are denoted as "Source: Authors’ calculations" in the figures.

### Appendix 2 — Additional results and country-level heterogeneity
- Geographical heterogeneity and MOSAIKS resolution
  - Figure A1: shows cross-district variation in population density in Dominica (Source: Gridded Population of the World and authors’ calculations).
  - Figure A2: illustrates the resolution of MOSAIKS grid for Barbados; red crosses denote MOSAIKS grid corners (Source: Sherman et al. (2023) and authors’ calculations).

- EICs across other Caribbean countries (ATG, DMA, GRD, KNA, TTO, VCT)
  - Figure A3: EIC for policies 1 and 2 in other Caribbean countries (ATG: Antigua and Barbuda, DMA: Dominica, GRD: Grenada, KNA: St. Kitts and Nevis, TTO: Trinidad and Tobago, VCT: St. Vincent and the Grenadines).
  - Figure A4: EIC for policies 3 and 4 for other Caribbean countries.
  - Additional figure: EIC for policy experiments 5 and 6 for other Caribbean countries.

- Table A1: Information value of subnational data (Correlation by country)
  - ATG: 0.45
  - DMA: 0.24
  - GRD: 0.16
  - KNA: 0.38
  - LCA: 0.48
  - TTO: 0.26
  - VCT: 0.19

*Source: Appendix 1: Figures and Tables, wpiea2024084-print-pdf.*

---


_Source: https://www.imf.org/-/media/files/publications/wp/2024/english/wpiea2024084-print-pdf.pdf_