## 1perea2022005 - Section 1

## Source details

**Canonical URL:** [1perea2022005 - Section 1](https://www.imf.org/-/media/files/publications/cr/2022/english/1perea2022005.pdf)

## Other formats

- [Markdown version](/-/media/files/publications/cr/2022/english/1perea2022005.pdf.md)
- [Structured JSON version](/-/media/files/publications/cr/2022/english/1perea2022005.pdf.json)

---

### Summary of mission outcomes and priority recommendations
- Mission timing and scope
  - Remote technical assistance (TA) mission to INEI during June 21, 2021–March 23, 2022 to update CPI weights and modernize CPI data collection by introducing web scraping and scanner data.
  - Mission spread over 10 months to align assistance with INEI work pace.
- CPI rebasing and weights
  - INEI launched the rebased and updated CPI from January 2022 with national coverage.
  - Number of COICOP classes covered increased from 8 to 12.
  - New weights based on data collected for nine months during 2019/2020; household expenditures for the remaining three months were estimated by extrapolating observed values for the missing months.
- Scanner data status
  - INEI initiated contacts with retailers but had not yet obtained scanner data.
  - Guidance provided on key features to request and on memorandum of understanding with data providers.
- Web scraping status and role
  - INEI aims to use web scraped data for products where quality adjustments are complex.
  - Data on televisions, mobile phone, laptops, housing, and shirts has been web scraped.
  - Televisions was the first product with web data available since May 2021 and used as the example to implement methodology and R code.
- Data source and legal considerations
  - Confirm mandate to web scrape given possible legal restrictions; always contact website owner and document data transmission via memorandum of understanding.
- Aggregation approach
  - Sub-index compiled for each website and each product; web-scraped sub-indices aggregated (Laspeyres-type) using turnover weights.
  - CPI sub-index from in-person price collection compiled as usual (Jevons); web-scraped and in-person sub-indices aggregated using turnover weights of companies/shops included in each.
- Data collection guidance
  - If online and physical shop prices or varieties differ, collect both online and in-person and estimate turnover volumes for each; otherwise in-person collection can be replaced by web scraping.
- Priority recommendations (from Table 1)
  - March 2022: Use the time dummy hedonic method with 12-months rolling window. — Responsible: INEI
  - June 2022: Investigate potential data sources for turnover with national accounts, tax authorities, business statistics, etc. — Responsible: INEI
  - September 2022: The CPI using big data (web scrapping) are released. — Responsible: INEI

### Detailed technical assessment and recommended actions (high-level milestones)
- Priority action / milestone summary (selected items and target completion dates)
  - Perform web scraping twice per week. — Continuous
  - Retrieve all available data: all varieties and all characteristics of each variety. — Continuous
  - Web scrape more than one website for each product. — Continuous
  - Use the time dummy hedonic method with 12-months rolling window. — April 2022
  - Use turnover weights to aggregate product sub-indices by website and to aggregate web-scraped sub-indices with in-person sub-indices by product. — April 2022
  - Investigate potential data sources for turnover with national accounts, tax authorities, business statistics, etc. — June 2022
  - Keep data collection in both online and physical shops if prices or varieties are different. — Continuous
  - Release CPI using big data (web scraping). — September 2022
  - Establish a formal agreement between retailer and INEI for scanner data transmission. — December 2022

### A. Consumer Price Index — findings and actions
- Implementation and coverage
  - Rebased CPI launched from January 2022 with national coverage.
  - COICOP classes covered increased from 8 to 12.
- Weight construction
  - New weights based on nine months of 2019/2020 data; three missing months estimated by extrapolating observed monthly values.
- Supplementary data sources for adjustments and validation
  - Household Budget Survey 2009, Survey on living conditions, and national accounts recommended to calculate rates of change and identify seasonal patterns at detailed levels.

### B. Scanner data — findings and recommendations
- Characteristics and benefits of scanner data
  - Scanner data are transaction-level data including turnover and quantities sold.
  - Benefits: actual transaction prices (including discounts), turnover information for representativeness, immediate inclusion of new products, longer temporal coverage, much larger sample size, reduced measurement errors, lower response burden and reduced cost for price collection.
- Practical and legal considerations
  - Obtaining scanner data can take from a couple months to a couple years; strong cooperation with retailers is essential.
  - Initial meetings should explain the purpose, use in other countries, importance of CPI, guarantee data confidentiality and security, and arrange a written agreement.
- Recommended agreement topics (to include in memorandum)
  - which data should be delivered
  - level of granularity of the data
  - confidentiality of the data
  - when and how the data is transmitted
  - temporal and spatial coverage
  - outlet type dimension
  - data is delivered free of charge
  - duration of the contract
- Recommended action
  - Write a formal agreement between retailer and INEI.

### C. Web scraping for the Consumer Price Index — methodology, cleaning, and integration
- Rationale and scope
  - Web scraping transforms unstructured website data into structured data for analysis; useful for products with complex quality adjustments.
  - Advantages: wide geographic coverage, excellent timeliness, cost-efficiency, low administrative burden, and detailed product characteristics.
  - Limitations: web-scraped data are not necessarily transaction prices (advertised products may not have been sold), online and in-store prices can differ, and coverage may not be exhaustive.
- Operational recommendations for web scraping
  - Frequency: perform web scraping once per week during the first three weeks of the month.
  - Data retrieval: retrieve all available data — all varieties and all characteristics of each variety.
  - Coverage: web scrape more than one website for each product.
  - Website selection: prioritize most important websites or more general aggregators; prepare a script for each website; adapt code when a website changes structure.
  - Legal: confirm mandate to web scrape; contact website owner and write a memorandum of understanding for data transmission even if legal framework permits scraping.
- Data cleaning steps (applied to INEI data and implemented in R)
  - Removing duplicates
  - Removing observations with missing values in variables used in the model
  - Identification and removal of outliers (experiment with interquartile multiplier)
  - Creating categories where appropriate (e.g., grouping screen sizes when many distinct values)
- Hedonic modeling and compilation
  - Sub-indices are calculated using the time-dummy hedonic method with a 12-month rolling window.
  - Method rationale: pools one year of data to produce more stable, less volatile indices; recommended when monthly observations are few.
  - Regression model (log-linear) and index derivation exactly as specified:
    - ln p_n^t = β_0 + Σ_t δ_t D_n^t + Σ_k β_k Z_nk^t + ε_n^t
    - t - period; n – number of observations in period t; k – variety specifications; lnp_n^t – price logarithm; β_0 – intercept; δ_t – coefficient of the time dummy variable that will generate the index; D_n^t – time dummy variables; β_k^t – “shadow” price of variety specifications k in period t; Z_nk^t – quantity of variety specifications k in period t and variety n; ε_n^t – error term
    - Index for current period (t): I_t = exp(δ̂_t) * 100
  - Chaining to avoid revisions: indices from overlapping 12-month windows are chained using the last overlap period between windows (example provided in Table 2).
- Aggregation into CPI
  - Compile a sub-index for each website and product.
  - Aggregate web-scraped sub-indices by website using turnover weights (Laspeyres-type).
  - Compile in-person CPI sub-index as usual (Jevons).
  - Aggregate web-scraped and in-person sub-indices using turnover weights of companies/shops included in each, ensuring the turnover concept/coverage is consistent across firms (e.g., technology-only turnover vs. all turnover).
  - Obtain turnover data from business register, tax offices, or national accounts as available.
- Recommendations (explicit action items)
  - Perform web scraping once per week during the first three weeks of the month.
  - Retrieve all available data, i.e., all varieties and all characteristics of each variety.
  - Web scrape more than one web site for each product.

### Section 2 — PERU
- Methodological recommendations and data collection guidance
  - Use the time dummy hedonic method with 12-months rolling window.
  - Use turnover weights to aggregate product sub-indices by web site and further to aggregate the web scraped sub-indices with the in-person price collection subindices, by product.
  - Investigate potential data sources for turnover with national accounts, tax authorities, business statistics, etc.
  - Keep data collection in both online and physical shops if the prices or the varieties are different.
- Officials Met During the Mission
  - Guillermo Gomez – Prices Executive Director, INEI
  - Lilia Montoya Sánchez – Technical Director, INEI
  - Jesús Córdova – Area Director, INEI
  - Mónica Saavedra – Methodologist, INEI
  - Esther Ponce – Analist, INEI
  - William Chambi – Analist, INEI
  - Richard Huiman – Analist, INEI
  - Diego Guevara – Analist, INEI
  - Milagros Sánchez – Analist, INEI

*IMF Country Report No. 22/332 — Section 1*

### Section 1

### 1perea2022005 - Section 1

### Summary of mission outcomes and priority recommendations
- Mission timing and scope
  - Remote technical assistance (TA) mission to INEI during June 21, 2021–March 23, 2022 to update CPI weights and modernize CPI data collection by introducing web scraping and scanner data.
  - Mission spread over 10 months to align assistance with INEI work pace.
- CPI rebasing and weights
  - INEI launched the rebased and updated CPI from January 2022 with national coverage.
  - Number of COICOP classes covered increased from 8 to 12.
  - New weights based on data collected for nine months during 2019/2020; household expenditures for the remaining three months were estimated by extrapolating observed values for the missing months.
- Scanner data status
  - INEI initiated contacts with retailers but had not yet obtained scanner data.
  - Guidance provided on key features to request and on memorandum of understanding with data providers.
- Web scraping status and role
  - INEI aims to use web scraped data for products where quality adjustments are complex.
  - Data on televisions, mobile phone, laptops, housing, and shirts has been web scraped.
  - Televisions was the first product with web data available since May 2021 and used as the example to implement methodology and R code.
- Data source and legal considerations
  - Confirm mandate to web scrape given possible legal restrictions; always contact website owner and document data transmission via memorandum of understanding.
- Aggregation approach
  - Sub-index compiled for each website and each product; web-scraped sub-indices aggregated (Laspeyres-type) using turnover weights.
  - CPI sub-index from in-person price collection compiled as usual (Jevons); web-scraped and in-person sub-indices aggregated using turnover weights of companies/shops included in each.
- Data collection guidance
  - If online and physical shop prices or varieties differ, collect both online and in-person and estimate turnover volumes for each; otherwise in-person collection can be replaced by web scraping.
- Priority recommendations (from Table 1)
  - March 2022: Use the time dummy hedonic method with 12-months rolling window. — Responsible: INEI
  - June 2022: Investigate potential data sources for turnover with national accounts, tax authorities, business statistics, etc. — Responsible: INEI
  - September 2022: The CPI using big data (web scrapping) are released. — Responsible: INEI

### Detailed technical assessment and recommended actions (high-level milestones)
- Priority action / milestone summary (selected items and target completion dates)
  - Perform web scraping twice per week. — Continuous
  - Retrieve all available data: all varieties and all characteristics of each variety. — Continuous
  - Web scrape more than one website for each product. — Continuous
  - Use the time dummy hedonic method with 12-months rolling window. — April 2022
  - Use turnover weights to aggregate product sub-indices by website and to aggregate web-scraped sub-indices with in-person sub-indices by product. — April 2022
  - Investigate potential data sources for turnover with national accounts, tax authorities, business statistics, etc. — June 2022
  - Keep data collection in both online and physical shops if prices or varieties are different. — Continuous
  - Release CPI using big data (web scraping). — September 2022
  - Establish a formal agreement between retailer and INEI for scanner data transmission. — December 2022

### A. Consumer Price Index — findings and actions
- Implementation and coverage
  - Rebased CPI launched from January 2022 with national coverage.
  - COICOP classes covered increased from 8 to 12.
- Weight construction
  - New weights based on nine months of 2019/2020 data; three missing months estimated by extrapolating observed monthly values.
- Supplementary data sources for adjustments and validation
  - Household Budget Survey 2009, Survey on living conditions, and national accounts recommended to calculate rates of change and identify seasonal patterns at detailed levels.

### B. Scanner data — findings and recommendations
- Characteristics and benefits of scanner data
  - Scanner data are transaction-level data including turnover and quantities sold.
  - Benefits: actual transaction prices (including discounts), turnover information for representativeness, immediate inclusion of new products, longer temporal coverage, much larger sample size, reduced measurement errors, lower response burden and reduced cost for price collection.
- Practical and legal considerations
  - Obtaining scanner data can take from a couple months to a couple years; strong cooperation with retailers is essential.
  - Initial meetings should explain the purpose, use in other countries, importance of CPI, guarantee data confidentiality and security, and arrange a written agreement.
- Recommended agreement topics (to include in memorandum)
  - which data should be delivered
  - level of granularity of the data
  - confidentiality of the data
  - when and how the data is transmitted
  - temporal and spatial coverage
  - outlet type dimension
  - data is delivered free of charge
  - duration of the contract
- Recommended action
  - Write a formal agreement between retailer and INEI.

### C. Web scraping for the Consumer Price Index — methodology, cleaning, and integration
- Rationale and scope
  - Web scraping transforms unstructured website data into structured data for analysis; useful for products with complex quality adjustments.
  - Advantages: wide geographic coverage, excellent timeliness, cost-efficiency, low administrative burden, and detailed product characteristics.
  - Limitations: web-scraped data are not necessarily transaction prices (advertised products may not have been sold), online and in-store prices can differ, and coverage may not be exhaustive.
- Operational recommendations for web scraping
  - Frequency: perform web scraping once per week during the first three weeks of the month.
  - Data retrieval: retrieve all available data — all varieties and all characteristics of each variety.
  - Coverage: web scrape more than one website for each product.
  - Website selection: prioritize most important websites or more general aggregators; prepare a script for each website; adapt code when a website changes structure.
  - Legal: confirm mandate to web scrape; contact website owner and write a memorandum of understanding for data transmission even if legal framework permits scraping.
- Data cleaning steps (applied to INEI data and implemented in R)
  - Removing duplicates
  - Removing observations with missing values in variables used in the model
  - Identification and removal of outliers (experiment with interquartile multiplier)
  - Creating categories where appropriate (e.g., grouping screen sizes when many distinct values)
- Hedonic modeling and compilation
  - Sub-indices are calculated using the time-dummy hedonic method with a 12-month rolling window.
  - Method rationale: pools one year of data to produce more stable, less volatile indices; recommended when monthly observations are few.
  - Regression model (log-linear) and index derivation exactly as specified:
    - ln p_n^t = β_0 + Σ_t δ_t D_n^t + Σ_k β_k Z_nk^t + ε_n^t
    - t - period; n – number of observations in period t; k – variety specifications; lnp_n^t – price logarithm; β_0 – intercept; δ_t – coefficient of the time dummy variable that will generate the index; D_n^t – time dummy variables; β_k^t – “shadow” price of variety specifications k in period t; Z_nk^t – quantity of variety specifications k in period t and variety n; ε_n^t – error term
    - Index for current period (t): I_t = exp(δ̂_t) * 100
  - Chaining to avoid revisions: indices from overlapping 12-month windows are chained using the last overlap period between windows (example provided in Table 2).
- Aggregation into CPI
  - Compile a sub-index for each website and product.
  - Aggregate web-scraped sub-indices by website using turnover weights (Laspeyres-type).
  - Compile in-person CPI sub-index as usual (Jevons).
  - Aggregate web-scraped and in-person sub-indices using turnover weights of companies/shops included in each, ensuring the turnover concept/coverage is consistent across firms (e.g., technology-only turnover vs. all turnover).
  - Obtain turnover data from business register, tax offices, or national accounts as available.
- Recommendations (explicit action items)
  - Perform web scraping once per week during the first three weeks of the month.
  - Retrieve all available data, i.e., all varieties and all characteristics of each variety.
  - Web scrape more than one web site for each product.

*IMF Country Report No. 22/332 — Section 1*

### Section 2

### PERU

### Methodological recommendations and data collection guidance
- Use the time dummy hedonic method with 12-months rolling window.
- Use turnover weights to aggregate product sub-indices by web site and further to aggregate the web scraped sub-indices with the in-person price collection subindices, by product.
- Investigate potential data sources for turnover with national accounts, tax authorities, business statistics, etc.
- Keep data collection in both online and physical shops if the prices or the varieties are different.

### Officials Met During the Mission
- Guillermo Gomez – Prices Executive Director, INEI
- Lilia Montoya Sánchez – Technical Director, INEI
- Jesús Córdova – Area Director, INEI
- Mónica Saavedra – Methodologist, INEI
- Esther Ponce – Analist, INEI
- William Chambi – Analist, INEI
- Richard Huiman – Analist, INEI
- Diego Guevara – Analist, INEI
- Milagros Sánchez – Analist, INEI

*Source: 1perea2022005 - Section 2*

---


_Source: https://www.imf.org/-/media/files/publications/cr/2022/english/1perea2022005.pdf_