## The Ottawa Group After 30 Years ## Source details **Canonical URL:** [The Ottawa Group After 30 Years](https://www.imf.org/-/media/files/data/cpi/companion-publication/the-ottawa-group-after-30-years.pdf) ## Other formats - [Markdown version](/-/media/files/data/cpi/companion-publication/the-ottawa-group-after-30-years.pdf.md) - [Structured JSON version](/-/media/files/data/cpi/companion-publication/the-ottawa-group-after-30-years.pdf.json) --- ### Purpose, origins, and scope - The Ottawa Group on Price Indices (United Nations International Working Group on Price Indices) first met in Ottawa on October 31–November 2, 1994. - The Group meets about every two years and over 18 meetings nearly all papers (540) and presentation slides were posted on the Ottawa Group website (maintained by Statistics Canada and Australian Bureau of Statistics and later transferred to UNECE). - Stated purpose (Ottawa Group): “The focus of the Group is on applied research, particularly in the area of consumer price indices. The Group examines advantages and disadvantages of various concepts, methods and procedures in the context of realistic operational environments, supported by concrete examples whenever possible. Participants are specialists and practitioners who work for, or are advisers to, statistical agencies in different countries or international organisations.” - Main topical focus: consumer price indices (CPI); producer price indexes discussed occasionally. ### Historical influence and outputs - Founders/conveners: Paul Armknecht (BLS), Bert Balk (Statistics Netherlands), Bohdan Schultz (Statistics Canada); Jacob Ryten approved Statistics Canada convening the first meeting. - Ottawa Group participants were influential in producing the Consumer Price Index Manual: Theory and Practice (2004) and in subsequent methodological discussions leading to the CPI Manual: Concepts and Methods (2020) and companion CPI Theory materials. - The Group’s agenda and follow-up topics (as listed by Jacob Ryten) included product quality adjustment, hedonic methods, treatment of new products, owner occupied housing, scanner data, seasonality, index formula choice, and measurement of “hard to measure” services. ### Major methodological themes reviewed - Four index-number theory approaches available in 1994: Basket approaches, Stochastic approaches, Test (axiomatic) approaches, Economic (Konüs) approaches. - Consensus/near-consensus findings from multiple approaches: - Fisher ideal index (P_F) recommended from basket, test/axiomatic and some economic perspectives. - Törnqvist-Theil index (P_T) recommended from stochastic and some economic perspectives. - Jevons (geometric mean, P_J) recommended for elementary (prices-only) indexes because it satisfies many tests including Time Reversal and Circularity. - Practical manuals influencing national practice: Turvey ILO Manual (1989), CPI Manual: Theory and Practice (2004), CPI Manual: Concepts and Methods (2020). --- ### Elementary (micro) index formula choice — empirical evidence and institutional responses - Empirical sensitivity: - Bohdan Schultz (Szulc) found that alternative micro-index formulae and linking frequency produced “differences in numerical results [that] are stunning, in some cases they are tenfold and more.” (Ottawa Group, 1994 summaries). - Szulc and others showed chained Carli formula at the elementary level can produce “tremendous upward bias” when prices are volatile. - Institutional moves: - U.S. BLS research (Reinsdorf; Moulton; Aizcorbe and Jackman) led to BLS adopting a geometric mean (Jevons) formula at the elementary level. - Statistics Canada moved from Dutot to Jevons at the micro-index level during the January 1995 basket update. - Comparative properties and recommendations (summarized from Dalén, Diewert, Woolford, de Haan): - Carli index P_C fails the Time Reversal Test and exhibits an upward bias; Carli satisfies certain basket-type intuitions but is discouraged for chained use. - Jevons index P_J satisfies many axiomatic tests (Time Reversal, Circularity) and was endorsed for elementary indexes where quantity data are unavailable. - Dutot index P_D is transitive and may be appropriate when products share identical units of measurement but can overweight higher priced items and fails commensurability invariance across units. - Practical recommendation: avoid Carli arithmetic mean of price relatives; use Jevons geometric index or Dutot average-price index if quantity data are unavailable; use unit values and Fisher-type aggregation when quantity data permit (Diewert). --- ### Scanner data, unit values, and chain-drift problems - Scanner data transformed possibilities: unit values as elementary prices; ability to compute exact expenditure-weighted indexes at outlet or market levels (Diewert, de Haan, Ivancic/Fox/Diewert). - De Haan (2008) supermarket scanner example (week 1 to week 191, 2005–2008): - Average matched products in consecutive two-week periods: 53 (range 43 to 63). - Chain outcomes at week 191 relative to week 1 for an example product: - chained Fisher index ended at 4.95% of the week 1 value, - chained Törnqvist index ended at 7.43% of the week 1 value, - chained Jevons index ended at 76.65% of the week 1 level. - Observation: chain drift can be severe and typically downward (stockpiling on sale), though upward drift can occur. - Ivancic, Fox and Diewert (2009) Australian scanner data (65 weeks, February 1997–April 1998, >100 stores, 19 item categories): - Toilet paper example — Fisher flexible-basket chained estimates: - Quarterly: (100.43 − 100 =) 0.43% - Monthly: (98.61 − 100 =) −1.39% - Weekly: (79.86 − 100 =) −20.14% - Quarterly vs weekly absolute differences for chained Fisher: - ~8% average absolute difference with item aggregation over stores. - ~14% average absolute difference when items disaggregated by store. - Recommendation: weekly chained index numbers, even superlative formulae, are not recommended; fixed-base/direct comparisons at monthly or quarterly frequency more reliable but face new/disappearing goods issues. - Multilateral remedies: - GEKS (geometric average of bilateral Fisher links across T periods) and TPD/WTPD multilateral time-dummy hedonic regressions proposed to satisfy circularity within a window and reduce chain-drift. - Limitation: standard multilateral methods (GEKS, CCDI, TPD, WTPD) have the property that products present in only one period of the T-period window do not affect resulting price levels for that period (i.e., new goods in period T may not affect multilateral indexes). --- ### Quantified empirical estimates of CPI upward bias (Diewert summary) - Estimated additive upward bias components (Diewert (1995a; summary)): - commodity substitution bias: .2% per year - outlet substitution bias: .25% per year - linking bias: perhaps .1% per year - new goods bias: at least .25% per year - implied upward bias total: at least .8% per year - Policy implication: these components were used to motivate index-method changes and revisions of historical inflation measures in some contexts. --- ### Quality adjustment, hedonic methods, and owner-occupied housing (OOH) - Hedonic regressions advocated for quality adjustment of new/differentiated products; Turvey argued hedonic techniques are “absolutely necessary” for house-price indexes due to uniqueness of dwellings. - Turvey’s OOH treatment menu (seven approaches) — choice depends on CPI purpose: - Acquisitions, several user-cost variants, and payment approaches (three payment variants). - 2004 Manual and 2020 Manual conclusions: - No consensus on best practice for seasonal commodities or single best approach to OOH; more than one OOH approach is required for different CPI purposes. - 2020 Manual highlighted unresolved measurement issues for telecommunication services, insurance and financial services, Owner Occupied Housing, and effects of digital goods and free services. --- ### Multilateral methods, time-dummy models, and alternative estimation strategies - Time Product Dummy (TPD) model (unweighted) — log-price regression ln p_tn = ρ_t + β_n + ε_tn; normalization ρ_1 = 0; P_TPD_t ≡ exp(ρ_t^*). Advantage: implementable with price data alone; disadvantage: does not incorporate expenditure-share weights. - Weighted Time Product Dummy (WTPD) — TPD with expenditure-share weights s_tn in least squares minimization. - Rolling Window GEKS (RWGEKS) and Rolling Window multilateral methods proposed to reduce chain-drift but do not fully eliminate it and fail circularity once windows advance. - LSLP and KBD econometric demand-side approaches: - Least Squares Linear Preferences (LSLP) multilateral indexes (estimate linear utility α·q) derive price P_t ≡ e_t / (α·q_t) and quantity Q_t ≡ α·q_t; share-equation estimation feasible even with missing products. - Konüs‑Byushgens (KBD) rank-1 quadratic functional form f(q) with parameter reduction A = ααT − ββT yields a flexible rank-1 substitution matrix; estimation via nonlinear least squares with normalizations and practical implementation strategies (Modified Expanding Window, expanding-window re-estimation). - Tests and desirable properties: - LSLP and KBD price/quantity levels satisfy a suite of tests including circularity/transitivity (no chain drift), invariance tests, responsiveness to new product prices (LSLP/KBD can respond to new-product price changes whereas many multilateral methods do not). - Limitation: linear utility models can understate benefits from enlarged choice sets (new product value), creating upward new-product bias at initial appearance; remedy: include additional periods to capture curvature or use nonlinear specifications. --- ### Research agenda and forward-looking priorities (from 2020 Manual Appendix 7 and Ottawa Group discussions) - Priority research topics (selected list as in Appendix 7 of the 2020 CPI Manual): - Scanner data and web-scraped prices. - Price updating of expenditure weights. - Use of administrative and credit-card data to form household-specific indexes. - Plutocratic versus democratic weighting of households. - Calculating elementary indexes using expenditure weights. - Quality adjustment and hedonic methods; valuation of new and free goods (reservation/shadow prices). - Treatment of seasonal products and long-term vs short-term linking strategies. - Owner Occupied Housing, insurance and financial services measurement. - Digitalization, measuring value of free digital services, wellbeing and sustainability metrics. - Methodological suggestions: - Retrospective calculations of superlative indexes as benchmarks to assess CPI bias. - Consider producing families of indexes for different purposes (e.g., nonrevisable vs revisable CPI; indexes tailored to monetary policy vs indexation), acknowledging many NSOs have not adopted multiple routine CPIs. - Interdisciplinary engagement recommended to address welfare measurement challenges posed by new/free products and digital goods. --- ### Key policy-relevant recommendations distilled from Ottawa Group outputs - Avoid Carli arithmetic mean of price relatives at the elementary (micro) level for chained series; favor Jevons (geometric) or Dutot (when unit commensurability holds) if no quantity data; use unit values and Fisher/Törnqvist aggregation when quantity data available. - Prefer monthly or quarterly aggregation for chained superlative indexes rather than weekly chaining; where chain-drift threatens validity, consider multilateral or rolling-window multilateral methods. - Where scanner or transaction-level data exist, sample values and quantities rather than prices alone to reduce new-introductions bias and to enable unit-value based elementary indexes. - Implement hedonic methods for quality adjustment where product heterogeneity or uniqueness is high (e.g., housing, durables); for routine CPI production, pragmatically balance feasibility and theoretical correctness. - Continue development and experimental use of multilateral, hedonic, and demand-estimation approaches (TPD/WTPD, GEKS, LSLP, KBD) with careful linking and revision policies (expanding or rolling windows) to manage chain-drift, new goods, and missing-product issues. - Maintain research and operational efforts on digital economy measurement, valuation of free goods, and owner-occupied housing given their substantial conceptual and practical implications for welfare measurement. *Source: The Ottawa Group on Price Indices — Introduction, meeting summaries (Ottawa, October 31–November 2, 1994), methodological reviews, and chapter summaries from The Ottawa Group After 30 Years and cited Manuals (Turvey (1989); ILO/IMF/OECD/UNECE/Eurostat/The World Bank (2004; 2020)) as provided in the content unit.* ### 1. Introduction ### the-ottawa-group-after-30-years — 1. Introduction ### Purpose, origins, and scope of the Ottawa Group - The Ottawa Group on Price Indices (also known as the United Nations International Working Group on Price Indices) had its first meeting in Ottawa on October 31- November 2, 1994. - The Group meets about every two years. - Purpose (as stated by the Ottawa Group): “The focus of the Group is on applied research, particularly in the area of consumer price indices. The Group examines advantages and disadvantages of various concepts, methods and procedures in the context of realistic operational environments, supported by concrete examples whenever possible. Participants are specialists and practitioners who work for, or are advisers to, statistical agencies in different countries or international organisations.” - The Ottawa Group has dealt with producer price indexes, but the main focus has been on consumer price indexes. - Over the 18 meetings referenced, almost all of the papers (540) and presentation slides presented at the meetings were posted on the Ottawa Group website, which has been maintained by Statistics Canada and the Australian Bureau of Statistics and later turned over to Carsten Boldsen at the UNECE in Geneva. ### Historical genesis and influence - Founding idea: Paul Armknecht (Bureau of Labor Statistics), Bert Balk (Statistics Netherlands) and Bohdan Schultz (Statistics Canada) conceived the Group after a Joint UNECE/ILO meeting in Geneva. - Bohdan Schultz proposed that Statistics Canada convene an Ottawa meeting; Jacob Ryten (Schultz’s superior) agreed, leading to the first meeting in 1994. - The Ottawa Group participants were influential in producing the 2004 Consumer Price Index Manual. - The document outlines that Section 4 discusses the first meeting in detail; Section 5 discusses the 2004 Manual and notes problems with its advice; Section 6 discusses current CPI Manuals; Section 7 concludes with outstanding problems and future developments; Appendix A explains a consumer demand approach addressing chain drift and quality adjustment. ### Context for the 1994 environment - To set the stage for the first Ottawa Group meeting, the document reviews alternative approaches to index number theory available to statistical offices in 1994 and reviews the 1989 ILO Manual on the CPI as existing “practical” advice to CPI compilers. --- ### Approaches to Index Number Theory in 1994 In 1994 there were four main approaches to bilateral index number theory available to price statisticians when price and quantity data on the same products were available for two periods: - Basket Approaches; - Stochastic Approaches; - Test or Axiomatic Approaches; - Economic Approaches. Each approach is described briefly below. ### Basket Approaches - Fixed basket approach (Lowe): form a representative annual basket q ≡ (q1,...,qN) and compute the Lowe price index P_Lo = p_t · q / p_0 · q where p_t · q = Σ_{n=1}^N p_{tn} q_n. - Two natural reference baskets: - Laspeyres (1871): P_L ≡ p_t · q_0 / p_0 · q_0. - Paasche (1874): P_P ≡ p_t · q_t / p_0 · q_t. - Symmetric averages of Laspeyres and Paasche proposed to reconcile differences: - Arithmetic mean (Sidgwick/Bowley): (1/2) P_L + (1/2) P_P. - Geometric mean (Fisher ideal index, 1922): P_F ≡ [P_L P_P]^{1/2}. - Time reversal test: desirable property P(p_t,p_0,q_t,q_0) = 1 / P(p_0,p_t,q_0,q_t). - Arithmetic mean of Paasche and Laspeyres fails the time reversal test. - Fisher index satisfies the time reversal test and is the only homogeneous symmetric average of P_L and P_P that satisfies time reversal. - Conclusion: From the basket viewpoint, the Fisher ideal index is a candidate “best” bilateral index formula. ### Stochastic Approaches - Unweighted (evenly weighted) stochastic approach: each price relative p_{tn}/p_{0n} is an estimate of a common inflation rate α, with model (1) p_{tn}/p_{0n} = α + ε_n, n = 1,2,...,N, where ε_n have mean 0 and variance σ^2. - Least squares estimator for α is the Carli (1764) price index: (2) P_C(p_0,p_t) ≡ Σ_{n=1}^N (1/N) p_{tn}/p_{0n}. - P_C does not satisfy the Time Reversal Test: P_C(p_t,p_0) ≠ 1 / P_C(p_0,p_t). - Logarithmic model: assume ln(p_{tn}/p_{0n}) = β + ε_n, n = 1,2,...,N, where β ≡ ln α and ε_n have mean 0 and variance σ^2. - Least squares / MLE estimator for β gives Jevons (1865) price index: - (4) P_J(p_0,p_t) ≡ Π_{n=1}^N (p_{tn}/p_{0n})^{1/N}. - P_J satisfies the Time Reversal Test. - Both Jevons and Carli treat each price relative equally and lack expenditure-based weighting; this was criticized. - Theil’s weighted stochastic solution: - Define period 0 expenditure share s_{0n} ≡ p_{0n} q_{0n} / p_0 · q_0 and period t share s_{tn}. - Theil proposed using arithmetic average weights (1/2)(s_{0n} + s_{tn}) on ln(p_{tn}/p_{0n}). - (5) ln P_T(p_0,p_t,q_0,q_t) ≡ Σ_{n=1}^N (1/2)(s_{0n} + s_{tn}) ln(p_{tn}/p_{0n}). - Exponentiating gives the Törnqvist-Theil price index P_T(p_0,p_t,q_0,q_t). - For the stochastic approach, P_T is regarded as a “best” index number formula. - Additional notes: - Jevons index satisfies Time Reversal; Carli index has a definite upward bias unless p_1 is proportional to p_0 (as shown by Fisher). - Estimation of hedonic regressions can be regarded as a stochastic approach (Court 1939; Dulberger 1989 for quality adjustment of computers). ### Test (Axiomatic) Approach to Bilateral Index Number Theory - The test approach evaluates index formulas by properties (tests) they should satisfy. - Strong identity test: P = 1 if p_0 = p_t = p for all p, q_0, q_t. - Weak identity test: P = 1 if p_0 = p_t = p and q_0 = q_t = q for all p and q. - Time reversal test discussed above is another example. - Key contributors: Walsh (1901) (1921) (1924), Fisher (1911) (1922), Eichhorn (1976), Eichhorn and Voeller (1976). - Diewert (1992) and Balk (2008) showed the Fisher price index P_F satisfies more “reasonable” tests than other commonly used indexes. - Conclusion: From the test/axiomatic viewpoint, the Fisher price index is the candidate “best” bilateral index. ### Economic Approach to Index Number Theory - Assumes all purchasers share the same utility function f(q), with q ≡ [q1,...,qN], and Q_t ≡ f(q_t) is the period t aggregate quantity corresponding to q_t. - Purchasers collectively choose q_t by solving the utility maximization problem: - (6) max_q { f(q) ; p_t · q ≤ e_t ; q ≥ 0^N } ≡ Q_t - where e_t is observed period t aggregate expenditures on the N products. - The unit cost function corresponding to f(q), c(p), is defined as the minimum cost of achieving the utility level (further development of the economic approach continues beyond the provided excerpt). *Source: the Ottawa Group on Price Indices — Introduction and Section 2 summary as provided.* ### 1. If purchasers face the period t vector of prices p ### the-ottawa-group-after-30-years - 1. If purchasers face the period t vector of prices p_t ### Economic approach: Konüs cost-of-living framework and aggregation identities - Define unit cost and aggregate price level: - c(p_t) ≡ min_q { p_t · q : f(q) = 1; q ≥ 0_N } ≡ P_t. (equation (7)) - Period t expenditure decomposition: e_t = p_t · q_t = c(p_t) f(q_t) = P_t Q_t. (equation (8)) - Bilateral Konüs cost-of-living and quantity indexes: - Price index between period 0 and t: c(p_t)/c(p_0) = P_t/P_0 (Konüs (1924) cost of living index under linearly homogeneous utility). - Quantity index: f(q_t)/f(q_0) = Q_t/Q_0. - Practical implication: specifying functional forms for f(q) or c(p) can yield known exact bilateral index formulas (theory of exact index numbers; founder Konüs (1924); labelled the “functional approach”). ### Exactness examples and flexible functional forms - Leontief/unit-cost linear case: - Suppose c(p) ≡ p · β = Σ_{n=1}^N p_n β_n with β_n > 0. - Then q_t = β Q_t, q_0 = β Q_0, P_0 = p_0 · β, P_t = p_t · β. - Laspeyres and Paasche equal the true cost-of-living index: c(p_t)/c(p_0) = p_t · q_0 / p_0 · q_0 (= P_L) = p_t · q_t / p_t · q_t (= P_P) = p_t · β / p_0 · β = P_t/P_0. - Interpretation: fixed-proportions consumption (Leontief preferences). - Quadratic unit-cost (matrix B) case and Fisher exactness: - Suppose c(p) ≡ [p · B p]^{1/2} with N×N symmetric matrix B satisfying regularity conditions (B must have one positive eigenvalue with a positive eigenvector; remaining eigenvalues equal 0 or negative). - Then cost-of-living c(p_t)/c(p_0) is exactly the Fisher index P_F = [P_L P_P]^{1/2}. - Diewert (1976) showed this functional form is flexible: it can approximate an arbitrary twice differentiable linearly homogeneous unit cost function around a positive price vector, so the Fisher index accommodates wide substitution possibilities. - Translog unit cost and Törnqvist-Theil exactness: - If ln c*(p) equals a constant plus linear and quadratic terms in the logarithms of prices (translog unit cost), then the Törnqvist-Theil price index P_T(p_0,p_t,q_0,q_t) equals c*(p_t)/c*(p_0) exactly. - Practical conclusion from economic approach: - Fisher index P_F emerges as a “best” choice under basket, test, and economic approaches. - Törnqvist-Theil index P_T emerges as a “best” choice under stochastic and economic approaches. - Diewert (1978) showed P_F and P_T approximate each other to second order around p_0 = p_t and q_0 = q_t, implying in many situations either index would be acceptable. ### Strengths, criticisms, and justifications for the economic approach - Strengths: - Economic approach accounts for substitution effects when products go on sale and consumption shifts. - Useful when adjusting prices for quality change because it compares utility/usefulness of new versus existing products. - Criticisms: - Assumptions required are very strong; many price statisticians are skeptical about its practical usefulness. ### Index number theory at the first stage of aggregation: unit values and intra-period inflation - Construction of elementary period prices and quantities: - When transactions in a period are represented by a single price and single quantity, the single price should be the unit value (transaction value divided by sum of quantities). - Problem with unit values under substantial within-period inflation: - Unit values overweight end-of-period transactions relative to beginning-of-period transactions, effectively implicit quality adjustment favoring later transactions. - Practical remedy suggested: shorten the accounting period until within-period price variation is negligible (Irving Fisher’s and Hicks’s suggestion—the Hicksian “week”), though often impractical. ### Index number theory when only price information is available: elementary indexes and tests - Elementary indexes (prices only) include: - Carli index P_C(p_0,p_t). - Jevons index P_J(p_0,p_t). - Dutot index P_D(p_0,p_t) ≡ (1/N) Σ_{n=1}^N p_{tn} / (1/N) Σ_{n=1}^N p_{0n}. - Interprets P_t as period t average price (1/N) Σ p_{tn}, P_0 analogously. - Test-approach evaluation of elementary indexes: - Carli index fails the Time Reversal Test and exhibits upward bias. - Dutot index fails Fisher’s Commensurability Test (not invariant to unit changes); use only when products share the same unit of measurement. - Dalén (1992) endorsed the Jevons index because it satisfies more tests than Carli and Dutot. - Jevons index properties: - Satisfies Circularity Test: P(p_1,p_2) P(p_2,p_3) = P(p_1,p_3). (equation (10)) - Satisfies Multiperiod Identity Test (weaker): P(p_1,p_2) P(p_2,p_3) P(p_3,p_1) = 1. (equation (11)) - These properties reduce chain drift associated with chaining bilateral elementary indexes across periods. - Empirical findings and institutional responses: - Szulc (1983) found chained Carli indexes can accumulate large upward bias in “price bounce” situations; motivated switches (e.g., Statistics Canada) from Carli to Dutot and Jevons for elementary indexes. ### Limitations of classical index number theory and implication of new goods - Classical theory assumes underlying prices and quantities p_{tn} and q_{tn} are positive so matched price ratios p_{tn}/p_{1n} exist. - At lower aggregation levels, the introduction of new goods and services leads to many unmatched prices; many classical approaches do not address lack of matching. - This mismatch motivates further practical guidance for constructing Consumer Price Indexes (CPIs). ### Practical guidance from the 1989 Turvey ILO Manual and ILO Resolution (Fourteenth International Conference of Labour Statisticians, Geneva, October 28–November 6, 1987) - Stated CPI purpose and construction principles (Turvey (1989; 124) quotation): - “The purpose of a consumer price index is to measure changes over time in the general level of prices of goods and services that a reference population acquire, use or pay for consumption. A consumer price index is estimated as a series of summary measures of the period-to-period proportional change in the prices of a fixed set of consumer goods and services of constant quantity and characteristics, acquired, used or paid for by the reference population. Each summary measure is constructed as a weighted average of a large number of elementary aggregate indices. Each of the elementary aggregate indices is estimated using a sample of prices for a defined set of goods and services obtained in, or by residents of, a specific region from a given set of outlets or other sources of consumption goods and services.” - Principles for defining elementary aggregates (Turvey (1989; 125)): - (a) Group related goods/services with similar price movements. - (b) Do not group goods/services with markedly different expected price movements. - (c) Distinguish elementary aggregates when weights (including regional or outlet weights) are available or can be estimated. - (d) Use regional or outlet weights in calculating the index even when separate regional or outlet sub-indices are not required. - (e) Describe elementary aggregates so any good/service can be unambiguously assigned. - “In the calculation of elementary aggregate indices, consideration should be given to the possible use of geometric means.” - Weights guidance (Turvey (1989; 126)): - Household expenditure survey is usually the main source of weights; surveys should be representative of household size, income level, regional location, socio-economic group. - Survey period should be a normal one and preferably cover a whole year if seasonal variations are important. - Weights should be examined periodically and revised at least once every ten years. - Practical problems noted in Turvey/Resolution: - Likely lack of weighting at the elementary level. - Monthly price collections from retail outlet surveys may be inconsistent with expenditure weights from household surveys. - Strongly seasonal products conflict with use of annual weights. - Treatment of disappearing products: replace with a similar product and attempt empirical valuation of characteristic differences; otherwise choose assumption of no change or that the full price difference reflects quality differences and link series accordingly (Turvey (1989; 128)). - Contextual limitation at time of Turvey Manual: - Scanner data was only beginning to emerge; practical CPI construction options were constrained by available data. *Source: the-ottawa-group-after-30-years (PDF chapter/section; material cites Konüs (1924), Frisch (1936), Diewert (1976, 1978, 1987), Turvey (1989) and the ILO Resolution of the Fourteenth International Conference of Labour Statisticians, Geneva, October 28–November 6, 1987).* ### Appendix 7 has the title “Price indices below the basic aggregation level”. ### Price indices below the basic aggregation level ### Szulc’s basic methodology for a Lowe index (annual weights with monthly elementary indexes) - Scope: N goods and services, all prices positive. Price vectors: - p_t ≡ [p_t1, ..., p_tN]; base month p_0 ≡ [p_01, ..., p_0N]; annual price p_a ≡ [p_a1, ..., p_aN]; annual quantities q_a ≡ [q_a1, ..., q_aN]. - Annual expenditure shares s_an defined as: - (12) s_an ≡ p_an q_an / p_a ⋅ q_a ; n = 1, ..., N. - Lowe index using month 0 as base and q_a as annual basket: - (13) P_Lo(p_0, p_t, q_a) ≡ p_t ⋅ q_a / p_0 ⋅ q_a = Σ_{n=1}^N (p_tn / p_on) s_oan - Hybrid expenditure shares s_oan defined as: - (14) s_oan ≡ p_on q_an / p_0 ⋅ q_a ; n = 1, ..., N. - Interpretation: the Lowe index is a share-weighted average of monthly relative prices p_tn / p_on using hybrid shares s_oan. ### Price updating of annual shares and equivalence to hybrid shares - Price updated shares (unnormalized) from annual shares s_an: - (15) s_oan* ≡ s_an (p_on / p_an) = p_0n q_an / p_a ⋅ q_a. - Sum of those updated shares: - (16) Σ_{n=1}^N s_oan* = p_0 ⋅ q_a / p_a ⋅ q_a. - True price updated annual expenditure shares (normalized): - (17) s_oan** ≡ s_oan* / (Σ_{m=1}^N s_oam*) = p_0n q_an / p_0 ⋅ q_a = s_oan (using (14)). - Equivalence result: - (18) P_Lo(p_0, p_t, q_a) ≡ p_t ⋅ q_a / p_0 ⋅ q_a = Σ_{n=1}^N (p_tn / p_on) s_oan = Σ_{n=1}^N (p_tn / p_on) s_oan** ; t = 0,1, ... . - Practical implementation requires: - (i) estimates of annual shares s_an (equation (12)); - (ii) monthly price relatives p_tn / p_on; - (iii) price relatives p_on / p_an that compare annual prices p_an to monthly base prices p_0n. - Caveat: in practice the N price ratios p_tn / p_on are elementary (bilateral) price indexes for categories of very similar products rather than “true” product price ratios; thus the representations in (18) are approximate equalities. ### Elementary (micro-) index formulas derived by Szulc - Definitions for t = 0,1 and N products p_t ≡ [p_t1, ..., p_tN]: - Price levels: - (19) P_A_t ≡ Σ_{n=1}^N (p_tn / N) ; P_G_t ≡ (Π_{n=1}^N p_tn)^{1/N} ; P_H_t ≡ [Σ_{n=1}^N (1/N)(p_tn)^{-1}]^{-1}. - Corresponding two-period price indexes: - (20) P_A(p_0, p_1) ≡ P_A_1 / P_A_0 ; P_G(p_0, p_1) ≡ P_G_1 / P_G_0 ; P_H(p_0, p_1) ≡ P_H_1 / P_H_0. - Direct means of price relatives: - (21) P_A*(p_0, p_1) ≡ Σ_{n=1}^N (1/N)(p_1n / p_0n) ; P_G*(p_0, p_1) ≡ (Π_{n=1}^N (p_1n / p_0n))^{1/N} ; P_H*(p_0, p_1) ≡ [Σ_{n=1}^N (1/N)(p_1n / p_0n)^{-1}]^{-1}. - Identifications and inequalities: - P_G(p_0, p_1) = P_G*(p_0, p_1) = Jevons index P_J(p_0, p_1) (see equation (4) in source). - P_A(p_0, p_1) equals Dutot index P_D(p_0, p_1) (see equation (9)). - P_A*(p_0, p_1) equals Carli index P_C(p_0, p_1) (see equation (2)). - Szulc showed: - (22) P_A*(p_0, p_1) ≥ P_G*(p_0, p_1) ≥ P_H*(p_0, p_1). - Observations on properties of elementary indexes: - Dutot index may overweight higher priced products and can be problematic when aggregated products are heterogeneous. - Carli index tends to give a higher measure of price change than other indexes; chained Carli can exhibit large upward bias if prices “bounced”. - Dutot and Jevons indexes are transitive (satisfy circularity test (10) in source) and thus Szulc favored them for that property. ### Szulc’s remarks on elementary index computation and practical issues - Szulc emphasized limited literature on derivation of indexes below basic aggregation level and provided explicit formulae for elementary indexes or micro-indices. - Practical complications: - Elementary price ratios used in Lowe index implementations are approximations; zero prices and substitution within elementary categories create methodological problems. - Approximation of p_on / p_an by an elementary index counterpart is likely problematic. - Szulc’s terminology: hybrid-value shares for s_oan (Turvey (1989; 167-169) referenced). ### Turvey Manual—purpose and broader CPI issues discussed - Purpose of the Manual (Turvey (1989; 1)): - Aimed at practising statisticians who must construct or revise a consumer price index (CPI), reflecting the international resolution reprinted in Appendix 1 and discussing practical detail. - Designed to help users of consumer price indices, including students and governments, to learn problems, limitations, and resources required to produce a reliable index. - Focuses on practice rather than academic literature, given practical constraints on obtaining current quantity data. - Data availability and relevance of index number theory: - Traditional theory assumed current price and quantity data; in practice statisticians often only have past annual weights and monthly price observations. - Emergence of electronic transactions and scanner data in the 1980s began to change data availability for more accurate indexes. ### CPI purposes, perspectives, and conceptual distinctions - Multiple purposes of CPI (Turvey (1989; 4-8))—Manual discusses these. - National (permanent resident) vs domestic perspective (Turvey (1989; 10)): - National perspective: household consumption of permanent residents (excludes tourist consumption and temporary residents). - Domestic perspective: sales of consumer goods and services by producers located in the country (production perspective used in System of National Accounts). - Choice affects sampling and why CPI may differ from national accounts deflator for household consumption. - Practical CPI micro price collection from retailers often includes purchases by tourists, businesses, and government agencies; cross-border household purchases are not captured by domestic retailer sales. ### Treatment of imputations, household production, and index variants - Inclusion of imputed values (household production, income in kind, government services) should depend on the CPI’s intended purpose (Turvey (1989; 12)). - For some purposes, add imputed values to measure total consumption; for others, exclude imputations to match the sales or incomes being deflated. - Implication: rationale for separate indexes—one largely free of imputations for central bank use and another with imputations to better measure actual household consumption. ### Concepts of consumption measurement and matching payments to use - Three concepts of consumption (Turvey (1989; 15-16)): - Acquisition: total value of goods and services delivered during a period. - Use: total value of goods and services actually consumed during a period. - Payment: total payments made during a period. - Differences are substantive, not just timing: payment following acquisition can involve interest; use over years reflects different price levels. ### Owner Occupied Housing (OOH) and consumer durables - Turvey’s three broad approaches and menu of seven treatments for OOH (Turvey (1989; 16-24)): - (A) Net acquisitions. - (B1) User cost (1): mortgage interest + conventional depreciation at replacement cost. - (B2) User cost (2): opportunity cost of invested capital + depreciation − accruing capital gains. - (B3) User cost (3): estimated rental value. - (C1) Payment (1): cash outlays on down payments, mortgage interest and repayments. - (C2) Payment (2): cash outlays on mortgage interest and repayments. - (C3) Payment (3): cash outlays on mortgage interest, excluding repayment elements. - Purpose determines appropriate OOH treatment: - Central bank inflation monitoring might prefer acquisitions approach (lower imputations). - National accounts may prefer a user cost approach. - Government indexing of pensions/transfers may prefer payments approach. - Data requirements for each approach are listed in Turvey (1989; 22). - Turvey suggested treating consumer durables similarly to OOH where practicable (Turvey (1989; 25)). - Practical complications for acquisitions approach in housing: need to decompose purchase price into structure and land components; depreciation applies to structure but not land. HICP uses acquisitions approach and has excluded OOH due to difficulties. ### Practical choices: point in time vs period index and sampling considerations - Choice between point-in-time and period-related index: - For deflating income/expenditure, index should relate to the period of the money flow. - For economic analysis, aligning the CPI with other period statistics is appropriate (Turvey (1989; 25)). - Practical considerations often force price collection at a point in time. - Chapters 3 and 4 of Turvey cover choice and weighting of elementary aggregates, sampling of products, sources of data, and selection of representative products—extending Appendix 7 material. *Appendix 7 is a re-publication of Szulc (1987); material and page references cited are from Turvey (1989).* ### Chapter 5 is on the details of how to collect prices in practice. The material on quality adjustment and ### the-ottawa-group-after-30-years - Chapter 5 (and related material on quality adjustment, missing prices, seasonality, and computation) ### Quality adjustment and replacement of disappearing products - Turvey defines two forms of quality judgements: - searching for a variety of the same quality as the one to be replaced so its price can be used instead of the old one; and - evaluating any difference in quality between the new and the old variety, putting a monetary value on differences that are not minimal. - Key practical requirement: distinguish differences relevant to consumer-perceived quality from irrelevant differences. - Turvey (1989; 77) stated: “Quality judgements take two forms. On the one hand, there is the search for a variety which is of the same quality as the one to be replaced, so that its price can be used instead of the old one. On the other hand, there is the evaluation of any difference in quality between the new and the old variety. In both cases, differences which are relevant to quality as seen by consumers have to be distinguished from irrelevant differences. In the first case, a variety has to be sought for which the relevant differences are minimal. In the second case, the new variety is chosen on other grounds and the problem is to put a monetary value on any differences which are not minimal.” Turvey (1989; 77). ### Hedonic regression and housing price indexes - Turvey describes hedonic regressions as regressing price on price-determining characteristics to provide quality-adjusted prices for new products. - He argues hedonic techniques are “absolutely necessary” for house price indexes because each property is essentially unique. - Example variable set used in a British house-price hedonic with about 12,000 observations per month: - House type: detached, semi-detached, terraced, bungalow, flat. - Number of: habitable rooms, bathrooms, separate toilets, garages, garage spaces. - Presence of a garden, of a plot of 1 acre or more. - Central heating: full, partial, none. - Freehold. - Location (12 regions). - Age of property in years. - Quote: “In the field of consumer price indices, the most promising use of the technique relates to the prices of existing dwellings. On the one hand, it is necessary because the dwellings sold in any period will hardly ever be the same dwellings sold in the preceding or reference period, so the need for it is particularly great. On the other hand, a reassuringly large number of price observations may be obtainable with descriptions and measurements of the relevant characteristics. For example, a British study uses about 12,000 house price observations per month, with the following variables: …” Turvey (1989; 81). - Author’s note: floor space and land plot area are important; splines on these characteristics may be preferable to linear specification; treatment of condominium apartments is more complicated. ### Missing prices: matching and imputation methods - Turvey’s principle: when an individual observation is unavoidably missing, calculations should omit the corresponding price so that “like is compared with like” (matching). Alternatively, impute a price using one of three methods: - (a) carry forward the previous observation, assuming no price change (simplest; acceptable only when there is not much inflation). - (b) assume the price would have moved in the same proportion as those prices within the elementary aggregate which were recorded. - (c) impute by using an observation from another, similar outlet not included in the regular price collection. - Turvey (1989; 92) wording: “When an individual observation is unavoidably missing, the calculations should omit the corresponding price from the data set with which current prices are compared, so that like is compared with like. In other words, the two sets of prices must be "matched". However, matching can also be achieved by using an imputed price, calculated in one of the following three ways: (a) ... (b) ... (c) ...” Turvey (1989; 92). - Caveat: replacing a missing price with a price of an alternative product can create quality-adjustment issues; Turvey discusses advantages and disadvantages of each method. ### Strong seasonality, fixed annual baskets, and rolling-year indices - Problem statement: strongly seasonal products are only available certain months; using a fixed annual basket for a monthly price index forces imputation of fictitious prices or unclear month-to-month interpretations when using month-varying weights. - Turvey observed: “This problem extends beyond seasonally varying prices, and arises when some goods or services are totally unavailable at certain times during the year, or are only available to a very limited extent when not in season, so that meaningful prices cannot be observed. ... The essence of the problem is that if one single set of weights is used, fictitious prices must be imputed for those months when there is no price to observe. Alternatively, if different weights are used for different months, reflecting the varying availabilities of items, the meaning of month-to-month changes in the index becomes unclear.” Turvey (1989; 103). - Turvey’s suggested “rolling year” annual index approach: - Calculate a 12-month centred moving average comparing the current 12-month cost of buying 12 reference-year monthly baskets with the total cost of the 12 reference-year baskets. - Practical downside: publication delay six months longer than usual. - Usefulness: can serve as a standard for judging other methods and as an objective, reproducible method of seasonal adjustment. - Quoted: “There is one procedure akin to seasonal adjustment which escapes both the need to impute fictitious prices and the alternative problem of interpreting month-to-month movements. This is the calculation of a 12-month centred moving average. Each month it compares the current 12-month cost of buying 12 reference-year monthly baskets with the total cost of the 12 reference-year baskets. From a practical point of view, such a moving average is not very useful, since it entails a publication delay six months longer than usual. However, it can serve as a standard for judging other methods.” Turvey (1989; 103). - Policy trade-off noted: - Rolling-year indices are less sensitive to month-to-month upticks in inflation because they depend on prices lagged up to 12 months; thus they may not alert central banks promptly. - Rolling-year indices are well suited for indexation purposes. - Turvey’s pragmatic conclusion: “The easiest solution would be to avoid the problem by totally omitting all goods and services not available in all 12 months.” Turvey (1989; 106). - Implication: statistical offices may need to produce more than one CPI to serve different uses (monetary policy vs indexation). ### Computation of the CPI, geometric means, and chaining - Chapter 6 largely elaborates Appendix 7 material due to Szulc, adding material on missing prices and seasonal products. - Turvey on geometric means: - He notes the geometric mean of price relatives is the ratio of geometric mean price levels. - He defends geometric means as superior and criticizes reluctance to use them for computational or explanatory difficulty reasons: - “In view of this superiority, it is not surprising that many statisticians regard the use of geometric means as the best solution. Why, then, are they so seldom used? One reason is that they make the calculations difficult, but this argument loses its validity once computers are used for calculating the index. A second reason is that it may be feared that their use is too difficult to explain to users of the index. But most indices have features which are difficult to explain, and in any case, the degree of complexity that is acceptable is growing, through time, in most countries. Statisticians should have the courage of their convictions.” Turvey (1989; 90-92). - Turvey on chaining: - He favors chaining annually but not within the year. - Concern about frequent chaining: a chained index usually fails the circularity test under oscillatory price and quantity behavior (most likely seasonal and annual); chaining more frequently than annually is “usually impracticable and in any case is undesirable.” - Benefit of annual chaining: follows evolution of consumption patterns and is more representative than using an unchanging set of weights. - Quote: “Chaining more frequently than annually is usually impracticable and in any case is undesirable. ... The advantage of an annually chained index is that it follows the evolution of the pattern of consumption. It is thus more representative than an index using an unchanging set of weights.” Turvey (1989; 110). ### Other chapters briefly noted - Chapter 7 addresses sources of error. - Chapter 8 addresses publication issues. *Source: Turvey (1989) as reviewed in The Ottawa Group After 30 Years.* ### 4. The First Ottawa Group Meeting ### 4. The First Ottawa Group Meeting ### Meeting purpose and opening remarks (Ottawa, October 31 to November 2, 1994) - The meeting was convened to "bring together an independent forum of specialists from different countries to exchange ideas on crucial problems of measuring price change and to propose concrete solutions." Ottawa Group (1994; 2). - The discussion stemmed from a long-standing debate about possible bias in the CPI: - In the 1970s it was commonly believed there was a downward bias in the CPI. - In recent years (circa the 1990s), it was believed, "at least in North America," that the bias is upward and that inflation is overestimated. Ottawa Group (1994; 2). - The Chair identified two main issues for the first meeting: - Micro level aggregation and its macro effects. - How to detect and estimate the bias of the consumer price index. Ottawa Group (1994; 2). - Related research areas the group identified for follow-up: - Relevance of the CPI as an indicator of the Cost of Living Index, - Harmonisation of European price indices, - Importance of agreeing on definition and concept of cost of living vs. fixed basket, - Defining a wider measurement of inflation (ex. whole economy price index), - Need for another economic approach to monitor inflation. Ottawa Group (1994; 2). ### Key empirical findings and methodological debates presented - Elementary-level formula bias and volatility: - Bohdan Schultz (Szulc) and earlier Szulc (1983) showed that use of the chained Carli formula at the elementary index level could lead to "tremendous upward bias" if prices were volatile; fixed-basket macro indexes are subject to possible upward substitution bias. - U.S. BLS research on elementary and upper-level bias (Armknecht, Moulton, Stewart; Reinsdorf; Moulton; Aizcorbe and Jackman): - Reinsdorf (1993) compared trends in US average prices for homogeneous product groups with official BLS price indexes and found substantial differences: - Example: official index for food showed average annual increases during the 1980’s of 4.2% per year while the weighted mean of average prices grew at only 2.1% per year. - Initial attribution to outlet substitution bias was later revised: the bias observed was due to elementary formula bias (use of the Carli formula at the elementary level). Moulton (1993), Reinsdorf and Moulton (1997), Reinsdorf (1998). - This research ultimately led the BLS to move to a geometric mean (Jevons) formula at the elementary level. Greenlees (2006; 24). - Aizcorbe and Jackman (1993) constructed superlative indexes (Fisher Ideal and Törnqvist) for 1982–1991 and provided estimates of upward substitution bias from using fixed-weight Laspeyres aggregation; their results became a basis for subsequent estimates, including those cited by the Boskin Commission. Greenlees (2006; 24). - Quality adjustment at the first aggregation stage: - Griliches and Cockburn (1994) suggested treating a brand name drug and its generic counterpart as the same product and using a unit value price over both as the period price (approach referenced in Paper 2). - Diverging methodological preferences: - North American participants tended to favor the economic approach to index number theory. - European and Australian participants tended to favor the Lowe index approach at higher aggregation levels and the basket, stochastic and test approaches in general. ### Papers presented at the meeting — summaries and principal messages - Paper 1 (Marta Haworth, 1994) - Problems encountered by the UK when moving from central-office product sampling to probability sampling. - Issues in validating collected price data. - Paper 2 (Paul Armknecht, Brent Moulton, Kenneth Stewart, 1994) - Summarized BLS efforts to reduce bias at the elementary and higher aggregation levels. - Highlighted Reinsdorf’s findings and subsequent attribution of bias to elementary formula choice rather than outlet substitution. - Discussed upper-level substitution bias literature (Manser and McDonald (1988); Aizcorbe and Jackman (1993)). - Discussed idea of performing quality adjustment at the first stage (Griliches and Cockburn (1994) example). - Overall: supported Jevons formula at the lowest level and provided context for biases later emphasized by the Boskin Commission Report. - Paper 3 (Sellwood, 1994) - Provided insight into formation and intended use of Eurostat’s HICP: mainly to measure inflation across EU member countries in a way useful to central banks. - Argued that imputed prices for Owner Occupied Housing and Household Production should not appear in a HICP; HICP as a supplement, not replacement, to national CPIs. - Addressed choice between Carli and Dutot at the elementary level: - Dutot gives too much weight to higher priced items (argument in favour of Carli). - Noted that Dutot satisfies circularity and time reversal tests while Carli fails both. - Expressed skepticism about the economic approach to CPI construction ("The classical model of a consumer maximising utility... seems remote from common experience." Sellwood (1994; 3)). - Paper 4 (Bohdan Schultz (Szulc), 1994) - Used Statistics Canada monthly micro data for over 50 commodities (December 1988 to January 1994, Ontario). - Compared six alternative elementary index formulae: - Ratios of mean prices: a ratio of equiweighted arithmetic mean prices (RAMP), a ratio of equiweighted geometric mean prices (RGMP), a ratio of equiweighted harmonic mean prices (RHMP). - Means of price relatives: an equiweighted arithmetic mean of price relatives (AMPR), an equiweighted geometric mean of price relatives (GMPR), an equiweighted harmonic mean of price relatives (HMPR). Schultz (1994; 4). - Calculated both fixed-base and chained indexes with monthly, annual and quinquennial linking from the December 1988 time base and found: - "Differences in numerical results are stunning, in some cases they are tenfold and more." - The choice of micro-index formula is "very important", often more important than macro-index formula choice. - Frequent linking greatly increases the differences induced by alternative micro-formulae; non-transitive formulae can "play havoc" with chain index numbers when relative prices tend to bounce. Schultz (1994; 2). - Explained Statistics Canada’s switch from Dutot to Jevons at micro-index level with the CPI basket update in January 1995 to avoid problems for categories with a broad price spectrum; noted both Dutot and Jevons satisfy Transitivity (Circularity) Test but Dutot overweights high-priced items and is not invariant to units of measurement. Schultz (1994; 2). - Paper 5 (Bert Balk, "On the First Step of the Calculation of a Consumer Price Index") - Treated the CPI as a group price index — an average of household-specific price indices — implying household-specific index form determines CPI form. - Argued that commonly used micro-indexes are difficult to formally justify given specified target indexes and that a universal micro-index formula is a misdirected search. - Concluded it is "hard to formally justify the use of a geometric mean." Balk (1994; 2). ### Methodological implications and consensus directions - Elementary (micro) formula choice is crucial: - Empirical evidence (Statistics Canada, BLS) pointed to substantial sensitivity of chained micro-indexes to the choice of formula and to linking frequency. - Both Statistics Canada and the BLS independently moved toward the Jevons (geometric mean) formula at the elementary level to mitigate observed biases associated with arithmetic-type measures (Carli/Dutot) and volatility. - Upper-level aggregation and substitution bias: - Superlative index comparisons (e.g., Fisher, Törnqvist) provided a simpler route to estimating upper-level substitution bias than estimating full consumer demand systems. - Aizcorbe and Jackman (1993) provided early superlative-based estimates that informed later assessments, including those referenced by the Boskin Commission. - Quality adjustment and product definition: - Treating brand and generic versions as a single product (unit value approach) was discussed as an approach to quality/variety adjustment at the first aggregation stage (Griliches and Cockburn (1994) referenced). *Ottawa Group meeting, October 31–November 2, 1994; summaries and papers as cited in chapter 4 of The Ottawa Group After 30 Years.* ### conclusion but it was not very helpful to national price statisticians who had to choose an elementary index ### conclusion but it was not very helpful to national price statisticians who had to choose an elementary index ### Dalén’s contributions and the test approach - Paper 6 was authored by Jörgen Dalén (1994). - Dalén (1992, 1994) delivered two key contributions: - He derived easy to understand numerical relationships between the various elementary index number formulae. - He systematically examined the properties or tests that the various elementary indexes satisfied, initiating a systematic test approach to elementary indexes. - Dalén (1994; 150) explained the upward bias inherent in the Carli formula: P_C(p0,p1)P_C(p1,p0) > 1 unless p1 is proportional to p0, in which case P_C(p0,p1)P_C(p1,p0) = 1. ### Definition and ordering of elementary indexes - Four elementary indexes discussed: - (i) the Carli index P_C(p0,p1) defined by (2); - (ii) the Jevons index P_J(p0,p1) defined by (4); - (iii) the Dutot index P_D(p0,p1) defined by (9); - (iv) the Harmonic index P_H*(p0,p1) defined by (21). - Szulc (1987; 12) established the inequalities (23): - P_H*(p0,p1) ≤ P_J(p0,p1) ≤ P_C(p0,p1). - The geometric average of the Carli and harmonic formulae, P_CSWD(p0,p1), is defined as (24): - P_CSWD(p0,p1) ≡ [P_C(p0,p1) P_H(p0,p1)]^{1/2}. - Fisher (1922; 472) first suggested P_CSWD as formula 101 and observed empirically that P_CSWD was very close to P_J. ### Justification of P_CSWD and Jevons as approximations to a generalized basket index - Dalén (1994; 151) noted: - Laspeyres collapses to Carli if base period expenditure shares are all equal. - Paasche collapses to Harmonic if period 1 expenditure shares are all equal. - If expenditure shares are equal in both periods 0 and 1, P_CSWD(p0,p1) approximates the Fisher index and hence a generalized symmetric basket index. - Carruthers, Sellwood and Ward (1980) and Dalén (1992) provided approximations showing: - P_J(p0,p1) ≈ P_CSWD(p0,p1) ≈ P_D(p0,p1) to the accuracy of various second order Taylor series approximations. - This provides a justification for the Jevons index as an approximation to a generalized basket type index. ### Tests and axiomatic results - Dalén (1992) applied eight different tests to bilateral elementary indexes; the Jevons index satisfied all eight tests. - Diewert (1995a; 6-16) extended the test approach: - The Jevons index P_J(p0,p1) satisfied some 16 tests. - The Dutot index P_D(p0,p1) satisfied 15 of the 16 tests. - Note: The Dutot index does not satisfy the commensurability test (it is not invariant to changes in units of measurement), so it should only be used when products in scope are measured in the same units. ### Woolford’s empirical comparison and interpretation - Paper 7 was authored by Keith Woolford (1994). - Woolford examined the “big three” elementary indexes used by statistical agencies: Jevons, Carli, Dutot. - He noted Carli and Dutot were consistent with a basket type index. - He favored Dutot for handling price bouncing behavior (it satisfied the circularity test) and for macro basket consistency, but worried that higher priced items received too much weight and that the index did not depend on price relatives. - Woolford used Australian Bureau of Statistics data to construct indexes for the Fresh fruit and vegetables sub-group for one city across twenty three elementary aggregates (aggregation using weighted arithmetic means). - Empirical result: Jevons and Dutot were approximately equal and Carli showed much higher rates of inflation; ABS had been using Carli. ### Scanner data, unit values, and implications for elementary indexes - Paper 8 was authored by Alain Saglio (1994) using two years of Nielsen scanner data on chocolate bar sales in France: - The average price index decreased 1.6% per year while the Laspeyres index decreased only 0.2% per year; Saglio attributed the difference to substitution effects as households switched to lower cost outlets and brands. - Scanner data introduced the ability to compute elementary indexes using price and quantity information and to use unit values as elementary prices. - Diewert (1995a) advocated unit values (value of units sold divided by total quantity sold) as elementary prices when transaction-level data are available and discussed shop-specific unit values versus market-area aggregation (outlet effect, brand effect, packaging effect). - Practical considerations: finer classification reduces matches across time periods; classifying unit values entails trade-offs and some empirical aggregation may be acceptable. ### Access to scanner data and institutional choices - Diewert raised the dilemma for Statistical Agencies in accessing scanner data: - (i) buy data from private firms, risking loss of control over data collection; - (ii) set up an information processing subsidiary, risking charges of unfair competition. - Over time, many private firms have made scanner data available to National Statistical Offices, and marketing firms have sold household price and quantity data used to construct CPIs. ### Empirical bias estimates and aggregation recommendations (Diewert) - Diewert summarized empirical bias evidence (Diewert (1995a; 35)): - commodity substitution bias: .2% per year - outlet substitution bias: .25% per year - linking bias: perhaps .1% per year - new goods bias: at least .25% per year - implied upward bias total: at least .8% per year - These five sources of bias were regarded as additive. - Diewert’s end-of-paper recommendations: - Statistical Agencies should avoid the Carli arithmetic mean of price relatives formula (2). - If quantity information is not available at the elementary level, use either the Jevons geometric price index (4) or the Dutot average price index (9). - At the individual outlet level, the best elementary average price for a homogeneous commodity is its unit value; if outlet unit values are available, aggregate over outlets using Fisher ideal price index in the second stage. - Values and quantities should be sampled rather than just prices to greatly reduce new introductions bias. - Statistical Agencies should consider purchasing electronic point of sale data or establishing Divisions to process such data. - Recent economic history will need revision given substantial outlet substitution and elementary price index biases uncovered by Reinsdorf and Moulton. ### Meeting follow-ups and agenda for future Ottawa Group work - Jacob Ryten’s closing remarks listed ten future topics for the Ottawa Group: - price indices for difficult areas such as insurance and gambling fees payable to a state agent; - product quality adjustment and the use of hedonic methods; - steps for harmonization of CPI; - clarifying concepts (e.g., What is a cost of living index?); - treatment of new products and outlets in price indices; - treatment of durable goods in the CPI; - index formulae at macro level, including the linking problem; - organization and techniques related to price surveys; - linkage between temporal and spatial price comparisons; - problem of seasonality and price indices; - measurement of inflation. - These topics were discussed in the next 16 meetings of the Ottawa Group and remain active. ### Overall conclusion - The first Ottawa Group meeting demonstrated that the Turvey paradigm for a CPI needed updating. - Key methodological shifts included: - adoption of the test/axiomatic approach to elementary indexes (with Jevons performing strongly), - recognition of scanner data and unit values as transformative data sources, - quantified estimates of multiple upward biases summing to at least .8% per year, - clear recommendations to avoid Carli and to favor Jevons or Dutot when quantities are unavailable, and to use unit values and Fisher-type aggregation when quantity data permit. *Ottawa Group (1994); Dalén (1992, 1994); Carruthers, Sellwood and Ward (1980, 1994); Fisher (1922); Szulc (1987); Woolford (1994); Saglio (1994); Diewert (1995a).* ### 5. The First Ottawa Group CPI Manual ### 5. The First Ottawa Group CPI Manual ### Recognition of need and timeline - Members of the Ottawa Group recognized the need for a revised CPI manual in 1998; the OECD coordinated and maintained the chapter outline as part of the Inter-secretariat Working Group on Price Statistics (IWGPS). - The Consumer Price Index Manual: Theory and Practice was completed in 2004 (editor: Peter Hill). - It took six years from the 1998 decision until the 2004 Manual was finished. ### Structure and contributors of the 2004 Manual - Editor: Peter Hill. - Chapters and authors: - Preface: Peter Hill, Paul Armknecht and W. Erwin Diewert - Reader’s guide: Peter Hill - 1 An introduction to consumer price index methodology: Peter Hill - 2 Uses of consumer price indices: Peter Hill - 3 Concepts and scope: Peter Hill and Fenella Maitland-Smith - 4 Expenditure weights and their sources: Valentina Stoevska and Carsten Boldsen - 5 Sampling: Jorgen Dalen, A. Sylvester Young and Bert Balk - 6 Price collection: David Fenwick - 7 Adjusting for quality change: Mick Silver - 8 Item substitution, sample space and new products: Mick Silver - 9 Calculating consumer price indices in practice: Carsten Boldsen and Peter Hill - 10 Some special cases: Keith Woolford, David Fenwick - 11 Errors and bias: John Greenlees and Bert Balk - 12 Organization and management: David Fenwick - 13 Publication, dissemination and user relations: Tom Griffin - 14 The system of price statistics: Kimberly Zieschang - 15 Basic index number theory: W. Erwin Diewert - 16 The axiomatic and stochastic approaches to index number theory: W. Erwin Diewert - 17 The economic approach to index number theory: The single-household case: W. Erwin Diewert - 18 The economic approach to index number theory: The many-household case: W. Erwin Diewert - 19 Price indices using an artificial data set: W. Erwin Diewert - 20 Elementary indices: W. Erwin Diewert - 21 Quality change and hedonics: Mick Silver - 22 The treatment of seasonal products: W. Erwin Diewert - 23 Durables and user costs: W. Erwin Diewert ### Theoretical coverage and gaps - Theoretical chapters in the 2004 Manual largely reflected the state of index number theory in 1994, with additional detail beyond earlier treatments (e.g., Turvey Manual). - Chapter 21 provided more systematic methods for dealing with quality change (including hedonic regressions discussed in prior manuals). - Chapter 22 recommended Rolling Year Mudgett-Stone indexes and year-over-year monthly indexes (with month specific weights) as checks on seasonal components of month-to-month CPI, but concluded: - “It is evident that more research needs to be carried out on the problems associated with the index number treatment of seasonal commodities. There is, as yet, no consensus on what is best practice in this area.” (ILO/IMF/OECD/UNECE/Eurostat/The World Bank (2004; 417)). - Chapter 23 decomposed user cost for a dwelling into separate land and structure components, noted that the acquisitions approach tends to give a much smaller weight to OOH than the user cost and rental equivalence approaches, and concluded that more than one approach to the treatment of OOH is required to address different CPI purposes. National Statistical Offices have not routinely provided alternative OOH indexes. ### Problems that emerged after 2004: chaining, drift, and alternatives - The 2004 Manual discussed when to use fixed-base or direct indexes versus chained indexes, noting that chaining is inadvisable when prices oscillate or bounce (e.g., regular seasonal fluctuations or price wars) but recommended chained symmetrically weighted indices for roughly monotonic changes. (ILO/IMF/OECD/UNECE/Eurostat/The World Bank (2004; 281)). - The Manual advised that chaining is advisable if prices and quantities of adjacent periods are more similar than of more distant periods and suggested constructing similarity measures and a linking “tree” of most-similar observations. (ILO/IMF/OECD/UNECE/Eurostat/The World Bank (2004; 281)). - The 2004 Manual referred to a 2002 Diewert discussion paper for explicit measures of absolute and relative price dissimilarity (later published as Diewert (2009)), but noted limitations when prices are not positive or products are missing in compared periods. ### Empirical challenges to chaining guidance - Ivancic and Fox (2011b) empirically tested dissimilarity indexes for chaining decisions and concluded: - “Dissimilarity indexes do not appear to be sufficient to resolve the issue of when to chain.” Ivancic and Fox (2011b; 1). - For their data set, chaining was: - appropriate in 47 out of 76 cases, - satisfactory in 20 cases, - not satisfactory in 9 out of 76 cases. - Ivancic and Fox (2011b; 5) reported only 9 out of 76 circumstances where both direct price and quantity dissimilarity indexes were less than their chained counterparts. ### De Haan (2008) scanner-data evidence on chain drift - Jan de Haan (2008) demonstrated severe chain drift even when using superlative bilateral links with scanner data from more than 100 supermarkets (Dutch supermarket chain): - Data covered week 1 in 2005 to week 35 in 2008: 191 weeks. - Average number of matched products in each consecutive two-week period: 53 (range 43 to 63). - Example product (“Product XXX Tablets”): normal price about 6.5 Euros; on sale for 12 of the 191 weeks at ~ one half the regular price; sales rose from virtually 0 to over 3000 for 7 of the 12 sale weeks (stockpiling). - Chain outcomes at week 191 relative to week 1: - chained Fisher index ended at 4.95% of the week 1 value, - chained Törnqvist index ended at 7.43% of the week 1 value, - chained Jevons index (using bilateral matched products at each link) ended at 76.65% of the week 1 index level. - De Haan noted chain drift is typically downward due to stockpiling but upward drift can also occur (Feenstra and Shapiro (2003) found upward drift in Törnqvist; Persons (1928) discussed Fisher chain drift examples). ### Remedies and trade-offs examined by de Haan - Increasing the observation period length reduces short-term noise and lowers or may remove chain drift: - Moving from weekly to monthly chaining produced much more reasonable series; Jevons suggested ~10% decline for detergents while Fisher and Törnqvist suggested little change, though volatility remained. Jan de Haan (2008; 18). - Extending to quarterly chaining produced larger measured price increases over the period (quarterly chained Fisher and Törnqvist rose by some 20% over the entire period), and differences remained between chained Fisher and chained Törnqvist (in quarter 14 the chained Törnqvist was 5.41 points below the chained Fisher). Jan de Haan (2008; 21-22). - De Haan warned that quarterly aggregation may introduce unit value bias (high-priced products get more weight in unit values) and that a quarter may be too long for a representative unit value. - Using a fixed base eliminates chain drift but struggles with rapid product turnover: as time passes, fewer matches exist between current and base period, making fixed-base indexes unreliable for elementary strata with high churn. ### Multilateral index solutions proposed by the Ottawa Group - Ivancic, Fox and Diewert (2009) proposed multilateral indexes over a window of consecutive periods to address the de Haan problem: - Multilateral indexes satisfy the circularity test within the window, preventing chain drift within that window. - Suggested multilateral formulas include the GEKS index and the Weighted Time Product Dummy (WTPD) index. - GEKS index explanation and definition: - Let P_F(r,t) denote the matched-product bilateral Fisher index comparing period t to period r. For T periods, form T alternative Fisher-based series using each period as base. Take the geometric average of these alternative price levels for symmetry (Gini (1931) approach). - Unnormalized GEKS price levels for t = 1,2,...,T are defined as: - p_GEKS_t ≡ [∏_{r=1}^T P_F(r,t)]^{1/T}. (Equation (27)) - GEKS treats all time periods symmetrically; normalized GEKS price levels P_GEKS_t are obtained by normalizing p_GEKS_t so that the period 1 index equals a chosen normalization. *5. The First Ottawa Group CPI Manual.* ### 1. Thus we have the following definition for the period t normalized price levels P ### The Ottawa Group After 30 Years — Time Product Dummy and Multilateral Index Methods ### GEKS price-level normalization and GEKS-Törnqvist - Definition: P_GEKS_t ≡ p_GEKS_t / p_GEKS_1 ; t = 1,...,T. (equation (28)) - If the matched product Törnqvist-Theil bilateral index number P_T(r,t) is used instead of the Fisher formula P_F(r,t) in definitions (27) and (28), the resulting (normalized) GEKS-Törnqvist price levels are P_GEKS-T_t for t = 1,...,T. - Note: GEKS multilateral index is also referred to as the CCDI index in the literature. ### Time Product Dummy (TPD) hedonic regression model (unweighted) - Model for period t and product n when n ∈ S(t): - p_tn ≈ π_t α_n ; t = 1,...,T; n ∈ S(t). (equation (29)) - Taking logs and adding errors ε_tn yields: ln p_tn = ρ_t + β_n + ε_tn ; t =1,...,T; n ∈ S(t). (equation (30)) - ρ_t ≡ ln π_t for t = 1,...,T and β_n ≡ ln α_n for n = 1,...,N. - Estimation via least squares: - min_{ρ,β} { Σ_{t=1}^T Σ_{n∈S(t)} [ln p_tn − ρ_t − β_n]^2 }. (equation (31)) - Normalization for uniqueness: - ρ_1 = 0. (equation (32)) → π_1 = 1. - Exponentiation to obtain estimates: - π_t^* ≡ exp[ρ_t^*] ; t = 1,...,T. - α_n^* ≡ exp[β_n^*] ; n = 1,...,N. (equation (33)) - Aggregate definitions using expenditures e_t ≡ p_t · q_t = Σ_{n∈S(t)} p_tn q_tn: - P_TPD_t ≡ π_t^* ; Q_TPD_t ≡ e_t / P_TPD_t ; t = 1,...,T. (equation (34)) - If α_n^* and quantity data are used directly: - Q_TPD_t^* ≡ Σ_{n=1}^N α_n^* q_tn = Σ_{n∈S(t)} α_n^* q_tn ; P_TPD_t^* ≡ e_t / Q_TPD_t^* ; t = 1,...,T. (equation (35)) - Inequalities in general (when errors ε_tn ≠ 0): - P_TPD_t^* ≤ P_TPD_t ; Q_TPD_t^* ≥ Q_TPD_t ; t = 1,...,T. (equation (36)) - Advantage: can be implemented using price information alone. - Disadvantage: does not incorporate economic importance (expenditure shares) of each product. ### Weighted Time Product Dummy (WTPD) - Weighted hedonic regression assumes same proportional price variation (assumptions (29)) but uses expenditure-share weights s_tn: - min_{ρ,β} { Σ_{t=1}^T Σ_{n∈S(t)} s_tn [ln p_tn − ρ_t − β_n]^2 }. (equation (37)) - Imposition of normalization ρ_1 = 0 yields unique solution given sufficient product overlap. - Resulting directly estimated and indirectly estimated weighted TPD price and quantity levels: P_WTPD_t, Q_WTPD_t and P_WTPD_t^*, Q_WTPD_t^*. - Empirical implementation: Ivancic, Fox and Diewert (2009) estimated direct WTPD indexes on Australian scanner data. ### Treatment of missing products and properties of multilateral methods - If product n is missing in period t, define p_tn ≡ q_tn ≡ s_tn ≡ 0. - Let S(t) be products purchased in period t; S*(n) be periods in which product n was purchased. Assume each product is sold in at least one period and there is some product overlap. - A troublesome property of multilateral methods: if a product is present in only one period of the T-period window, it has no effect on resulting price levels. Consequence: new products entering in period T do not affect multilateral indexes. This property holds for GEKS, CCDI, TPD and WTPD. - The Rolling Window GEKS (RWGEKS) method: choose a window length (e.g., 13 months), compute GEKS for the window, make those 13 price levels “permanent”, then when month 14 arrives compute GEKS for months 2–14 and use the ratio of month 14 to month 13 to update permanent index, and continue rolling forward. This reduces chain-drift though does not fully eliminate it and does not pass the circularity test once new windows are added. ### Scanner data evidence (Australian data set) — empirical findings and stylized magnitudes - Data description (Ivancic, Fox and Diewert (2009; 12)): - Scanner data collected by A.C. Nielsen for four supermarket chains in a major Australian capital city. - Over 100 stores included; these stores account for approximately 80% of grocery sales in this city. - Data span 65 weeks, collected between February 1997 and April 1998. - Information on 19 supermarket item categories. - Observations refer to average weekly price (weekly unit value) and total weekly quantity sold of each item transacted in each store in each week. - Minimum number of observations for a category: 225,789 for “butter”. - Maximum number of observations for a category: 2,639,642 for “juices”. - Demonstrated chain-drift of chained Laspeyres with extreme sensitivity to chaining frequency: - Example for item category toilet paper over a 15 month period: - Quarterly, fixed-basket Laspeyres: (106.71 − 100 =) 6.71% - Weekly, fixed-basket Laspeyres: (11,955 − 100 =) 11,855% - Demonstrated likely downward bias of chained Fisher indexes with time aggregation effects: - Fisher flexible-basket chained estimates for toilet paper (no item aggregation over stores): - Quarterly: (100.43 − 100 =) 0.43% - Monthly: (98.61 − 100 =) −1.39% - Weekly: (79.86 − 100 =) −20.14% - Average absolute difference between weekly and quarterly flexible-basket chained Fisher estimates: - Approximately 8% with item aggregation over stores. - Approximately 14% when items are disaggregated over stores. - Empirical conclusions (Ivancic, Fox and Diewert (2009; 17)): - The use of weekly chained index numbers, even superlative formulae, is not recommended due to erratic results. - Fixed-base or direct comparisons of a current period with a base period give reasonably reliable results at monthly or quarterly frequency, but suffer from problems with new and disappearing goods over time. ### Comparison of GEKS, WTPD and practical linking strategies - Ivancic, Fox and Diewert found WTPD series approximated GEKS series fairly well; overall little difference between methods for their data. - Rolling Window GEKS compared with GEKS over entire sample: - Very little difference; GEKS and RWGEKS series plotted virtually on top of each other for their study. - Linking strategies for rolling-window multilateral indexes: - Ivancic, Diewert and Fox (2011): link movement of rolling window indexes for the last two periods in the new window to the last index value generated by the previous window. - Krsinich (2016): link to the previous window index value for the second period in the previous window (window splice). - De Haan (2015): suggested linking period in the middle of the old window (half splice). - Ivancic, Diewert and Fox (2011): suggested geometric average of all links for the last period in the new window to observations in the old window as linking factor. - Diewert and Fox (2021): examined alternative linking strategies; average or mean linking appears statistically safest. - Alternate approach: ever-expanding window (applied in Diewert and Shimizu (2024) and discussed in Diewert (2023) Appendix). - Limitations: - Rolling Window methods reduce but do not eliminate chain drift; circularity remains once new windows are added. Longer time series can still reveal chain-drift. ### CPI Manual 2020 and implications for scanner data and multilateral indexes - Motivation: scanner data availability and chain-drift issues prompted an update of the 2004 CPI Manual; work began in 2015 and resulted in Consumer Price Index Manual: Concepts and Methods 2020. - The Manual was edited by Brian Graf with lead institution the IMF. - Purpose: explain recommended methods to calculate a CPI; companion publication Consumer Price Index Theory explains underlying economic and statistical theory. - The 2020 Manual: - Update of the 2004 Manual with new chapters including Chapter 10 on Scanner Data and updating CPI weights. - Contributors and reviewers drawn from IMF, ILO, Eurostat, OECD, national statistical offices and external experts. - Structure: 14 chapters and 7 Appendices (appendices include topics such as HICP, COICOP 1999 and 2018, spatial comparisons, basic index number formulas, and the Consumer Price Index Research Agenda). - Companion CPI Theory Manual: draft chapters available due to ongoing theoretical research; intended to accompany the 2020 Manual with extensive references. *Source: The Ottawa Group After 30 Years (chapter from the companion publication PDF).* ### 7. Looking Ahead ### 7. Looking Ahead ### Research agenda from Appendix 7 of the 2020 CPI Manual - Scanner data - Web scraped prices - Price updating of expenditure weights - The use of administrative data to form indexes - Plutocratic versus democratic weighting of households - The use of credit card data to form household specific price indexes - Calculating elementary indexes using expenditure weights - Quality adjustment - The treatment of seasonal products - The use of target price indices for the CPI - On the choice of formula for calculating higher level price Indices - The use of long-term and short-term Links - Retrospective Calculations of Superlative Price Indexes - Consumer price indexes for different groups and geographic areas - Measuring “hard to measure” services - Insurance and financial services - Owner Occupied Housing - Digitalization - Well being and sustainability (These topics are listed in Appendix 7 of the 2020 CPI Manual prepared by IMF/ILO/UNECE/Eurostat/OECD/The World Bank.) ### Quality adjustment, new products, and free goods — implications for welfare measurement - The Reinsdorf and Schreyer working paper identifies three reasons why the CPI will overestimate the cost of living and hence underestimate progress in real welfare: - (1) insufficient adjustment for quality changes; - (2) delayed inclusion of truly new products; - (3) disregarding the appearance and use of free products. - The Manual notes: "Solving these issues involves addressing both conceptual and practical measurement issues." - In a cost of living index context, the theoretically correct way to include truly new products and products offered for free is the use of estimated "reservation" or "shadow" prices; feasible in research but "for the regular production of the monthly CPI, this is usually not feasible and other approaches must be implemented." (IMF/ILO/UNECE/Eurostat/OECD/The World Bank (2020; 458)) - The Manual recommends further interdisciplinary engagement: "It may be useful to invite experts from other areas of official statistics to discuss measuring welfare and economic well-being." (IMF/ILO/UNECE/Eurostat/OECD/The World Bank (2020; 458)) ### Digitalization and the CPI - The Manual highlights significant measurement challenges posed by the digital economy, including: - effects of services provided for free (or without direct payment) on the internet; - defining and identifying digital goods and services, including different types of internet purchases, services for free, and shared economy services. - The Manual references the OECD Statistics Working Paper "Measuring Consumer Inflation in a Digital Economy" (Reinsdorf and Schreyer 2020) as documenting the need to clarify conceptual issues and develop methods to better measure the digital economy in the CPI. ### Services, insurance, and housing measurement priorities - The Manual emphasizes unresolved measurement issues across services: - Telecommunication services continue to create issues for compilers; coordination with the Voorburg Group on Service Statistics is recommended where relevant. - Insurance and financial services raise measurement problems (net versus gross approaches, choosing appropriate deflators for premiums); more discussion and research are needed to guide compilers. - Owner Occupied Housing is listed as a priority research area. - The Manual cites theoretical work relevant to these issues (Diewert (1993b; 1995b; 2014); Diewert, Fixler and Zieschang (2016)). ### Data sources and index construction methods - Topics flagged for further research include: - Use of scanner data and web-scraped prices; Cavallo (2017) and Cavallo (2020) are noted on these topics. - Price updating of expenditure weights (see discussion around equations (12)-(17) in section 3 of the chapter). - Use of long-term and short-term links to combine superlative formulas for long-term links with short-term updating; the approach has been used in Sweden and adopted by the United States and is recommended for further exploration. - Retrospective calculations of superlative price indices as benchmarks to assess CPI quality and potential bias; sharing experiences to develop best practices is encouraged. - Choice of aggregation formula for higher-level indices: arithmetic aggregation is widely used, but alternatives (geometric aggregation, averages, CES/Lloyd–Moulton indices) warrant research. ### Diewert’s expectations on the future of statistical agencies (as speculated in Diewert (2001)) - Firms will submit price and quantity data on consumer products to NSOs via the internet. (Observed to have occurred.) - Less certainty that NSOs will obtain price and quantity data directly from households; however, market research firms have collected household purchase data and made them available to researchers and NSOs (at a price). Limitations: product codes are highly aggregated. - Web scraping of prices would lead to more accurate CPIs; in particular, information on used durables (e.g., cars) would improve depreciation estimates and facilitate a user cost approach to durables. - Statistical agencies are likely to produce families of indexes; specifically, Diewert expected at least two CPIs: (i) a Nonrevisable CPI and (ii) a Revisable CPI. He also anticipated different CPIs for different purposes. (The Manual notes that "for the most part, this prediction has not come about.") - For strongly seasonal products, Diewert expected NSOs to produce at least two indexes for seasonal product strata: one focusing on year-over-year price change in the same month (using seasonal baskets) and another measuring month-to-month price change. (This prediction has also not materialized.) - Diewert expected many unresolved problems with hedonic regression models to be solved; subsequent work (Triplett (2004) and others) has advanced the field. ### Practical methodological suggestion in the chapter - The Appendix to the chapter suggests a simplified version of the Diewert-Feenstra estimation of reservation prices methodology that could be used by National Statistical Offices in real time. *Source: IMF/ILO/UNECE/Eurostat/OECD/The World Bank (2020) and Diewert (2001), as cited in the chapter.* ### section 2 of Diewert and Shimizu (2024). ### Section 2 — Diewert and Shimizu (2024) ### The role of household time allocation and three theoretical innovations - Diewert speculated that availability of information on households’ allocation of time would enable incorporation of many theoretical innovations into the design of consumer price indexes. Three possible innovations were singled out: - (i) Implementation of Becker’s (1965) theory of the allocation of time. - Becker’s model: households combine their time with market goods and services to produce finally demanded “commodities” that yield direct utility (example: combining services of a bed with time to produce “sleep utility”). - The theory includes disutility of time spent working on the external market and commuting. - Advantage: time costs spent on each consumption activity can be valued at some opportunity cost of time and added to the cost of purchasing goods and services from the marketplace. (Diewert (2001; 109-110).) - Progress: implementations and extensions cited include Schreyer and Diewert (2014) and Diewert, Fox and Schreyer (2017). - Note: incorporating the time constraint into standard consumer theory enables better valuation of the contribution of free goods and services to household standard of living. - (ii) Implementation of an extended version of Becker’s model to cover household nonmarket production. - Issue: Becker’s model did not cover home production for market sale. Growth of self-employment and contracting out means many households produce goods/services at home that are sold to others. - Implications for the cost of living (COL) index: - The extended COL would need to include production-type inputs (materials, office equipment) when production occurs at home. - Traditional consumer purchases (heating fuel, telephone services, transportation, home computers, etc.) would need allocation between business and personal use. - Outputs produced by household market production would need to be accounted for, in addition to intermediate inputs. (Diewert (2001; 110).) - Additional measurement literature references include Abraham and Mackie (2005), Hill (2009) and Schreyer and Diewert (2014). - (iii) Implementation of an extended version of Becker’s model to medical economics. - Illness, disease and accidents are constraints that reduce capabilities and thus utility (examples: broken leg prevents tennis/jogging; vision deterioration prevents reading/watching TV). - Within Becker’s allocation of time framework, disease or accident adds constraints to the utility maximization problem reducing welfare; medical treatments can remove or lessen those constraints and thus add to consumer welfare. - This extension opens the possibility of welfare-based evaluations of effects of medical treatments. (Diewert (2001; 110).) ### Household time data: key to improving COL and welfare measurement; practical difficulties - Central claim: obtaining information on household allocation of time is key to improving measurement of the cost of living and household welfare. - Practical progress: progress in implementing household time surveys has been slow and is likely to remain slow. - Core measurement challenge: conceptual difficulty in measuring household allocation of time across overlapping activities (e.g., child care while listening to music or watching television) and accurate household recording of time spent on alternative activities. - Comparative ease: measuring household (digital) monetary transactions is conceptually much simpler than measuring time allocation. ### Additional problematic measurement areas addressed by Ottawa Group members - Measuring household welfare and the construction of consumer price indexes under pandemic conditions (see Diewert and Fox (2022a) (2022b)). - Measuring the effects of environmental change (see Brandt, Schreyer and Zipperer (2014)). - General conclusion: researchers presenting papers at Ottawa Group meetings have been instrumental in improving consumer price indexes over the years. ### Appendix — The Consumer Demand Approach to the Construction of Multiperiod Indexes (overview and key results) A1. The estimation of consumer demand functions using the unit cost function - Set-up: - N related products, T periods; period t aggregate quantity vector q_t = [q_t1,...,q_tN], period t price vector p_t = [p_t1,...,p_tN]. - If product n not available in period t, q_tn = 0; for now missing prices set to 0 (with footnote that the unobserved Hicksian reservation price p_tn* would be the appropriate value). - Dual unit cost function c(p) for positive price vectors: - (A1) c(p) ≡ min_q { p·q : f(q) = 1; q ≥ 0_N }. - If purchasers maximize utility f(q) subject to budget and face same p_t: - Observed period t aggregate quantity vector satisfies: - (A2) q_t = ∇c(p_t) u_t ; t = 1,...,T - where observed period t expenditure e_t ≡ p_t·q_t ≡ Σ_{n=1}^N p_tn q_tn, unobserved utility u_t, and ∇c(p_t) ≡ [∂c(p_t)/∂p_1, ..., ∂c(p_t)/∂p_N]. - Also: - (A3) e_t = c(p_t) u_t ; t = 1,...,T. - Rearrangement gives system: - (A4) q_t = e_t ∇c(p_t)/c(p_t) ; t = 1,...,T. - Estimation approach: - Postulate functional form for c(p); compute partial derivatives; add errors to (A4); estimate parameters. - Multiperiod price and quantity levels: - (A5) P_t ≡ c(p_t); Q_t ≡ e_t / c(p_t) ; t = 1,...,T. - Unit-of-measure invariance issue: P_t/P_1 not invariant to units; remedy by converting to share equations. - Expenditure share s_tn defined: - (A6) s_tn ≡ p_tn q_tn / p_t·q_t = p_tn q_tn / e_t ; n = 1,...,N ; t = 1,...,T. - Share equation form: - (A7) s_tn = p_tn [∂c(p_t)/∂p_n] / c(p_t) ; n = 1,...,N ; t = 1,...,T. - Estimation by least squares minimization (no explicit error structure specified): - (A8) min_parameters of c(p) { Σ_{t=1}^T Σ_{n=1}^N [ s_tn − p_tn [∂c(p_t)/∂p_n] / c(p_t) ]^2 }. - Missing products complication: - If product n missing in t with unknown reservation price p_tn* > 0, s_tn = 0. - Cost-function approach with missing prices becomes extremely difficult because p_tn* appears in partial derivatives and must be estimated. - Conclusion: cost function approach to estimate reservation prices fails when missing products are present; motivates alternative approach. A2. The estimation of inverse demand functions - Start from utility maximization: - (A9) max_q { f(q) : p_t·q = e_t ; q ≥ 0_N } ; t = 1,...,T. - First-order conditions: - (A10) ∇f(q_t) = λ_t p_t ; t = 1,...,T. - (A11) p_t·q_t = e_t ; t = 1,...,T. - Solve for λ_t using Euler’s Theorem (linear homogeneity of f): - (A12) λ_t = q_t·∇f(q_t) / e_t = f(q_t) / e_t ; t = 1,...,T. - Substitute into (A10) to obtain inverse demand system: - (A13) p_t = e_t ∇f(q_t) / f(q_t) ; t = 1,...,T. - Define aggregate quantities and price levels: - (A14) Q_t ≡ f(q_t) ; P_t ≡ e_t / f(q_t) ; t = 1,...,T. - To achieve unit invariance, convert (A13) to share equations: - (A15) s_tn = q_tn [∂f(q_t)/∂q_n] / f(q_t) ; n = 1,...,N ; t = 1,...,T. - Important: (A15) valid even with missing products (if product n missing in t, q_tn = s_tn = 0 and equation becomes 0 = 0). This is why inverse demand estimation is feasible with missing products while direct demand estimation is not. - Estimation by least squares minimization (no explicit error structure specified): - (A16) min_parameters of f(q) { Σ_{t=1}^T Σ_{n=1}^N [ s_tn − q_tn [∂f(q_t)/∂q_n] / f(q_t) ]^2 }. - Reservation price estimation when product missing: - (A18) p_tn* ≡ e_t [∂f(q_t)/∂q_n] / f(q_t). A3. Econometric estimation of linear preferences (homogeneous linear utility) - Utility functional form: - (A19) f(q, α) ≡ Σ_{n=1}^N α_n q_n = α·q. - Least squares problem: - (A20) min_{α’s} { Σ_{t=1}^T Σ_{n=1}^N [ s_tn − α_n q_tn / (α·q_t) ]^2 }. - Non-uniqueness: if α* solves (A20), then λ α* also solves for any λ > 0. Scale normalization required. - Non-existence example: two periods and two products with no overlap leads to infeasibility (illustrated by (A21) and (A22)). - Conditions to obtain unique solution: - Assume at least one product present in all T periods; reorder so product 1 always present: - (A23) q_t1 > 0 ; t = 1,...,T. - Normalization: - (A24) α_1 ≡ 1. - With these, solution α* unique with α_1* = 1. - Aggregate quantities and prices: - (A25) Q_t ≡ α*·q_t ; P_t ≡ e_t / (α*·q_t) ; t = 1,...,T. - These Least Squares Linear Preferences (LSLP) multilateral price indexes arise from estimating linear utility functions. National Statistical Offices already estimate linear preferences in some multilateral index practices and hedonic time-dummy regression models implicitly assume linear preferences (see Diewert (2022) and Diewert and Shimizu (2024)). A4. Test properties of the LSLP multilateral price and quantity levels - Definitions: price and quantity levels P_t ≡ e_t / (α*·q_t) and Q_t ≡ α*·q_t depend on underlying p_1,...,p_T and q_1,...,q_T. - Tests (numbered and stated exactly as in the source): - Test 1: Quantity Levels Identity Test: If q_r = q_t, then Q_r = Q_t. - Test 2: Basket Test for Price Levels: If q_r = q_t = q, then P_t/P_r = p_t·q / p_r·q. - Test 3: The Product Reversal Test (Invariance of the price and quantity levels to changes in the ordering of the products). - Test 4: Linear Homogeneity Test for Quantity Levels: If q_t = λ q_r for some λ > 0, then Q_t = λ Q_r. - Test 5: Within Period Linear Homogeneity Test for Quantities: Q_1(p_1,...,p_T; λ q_1,q_2,...,q_T) = λ Q_1(p_1,...,p_T; q_1,q_2,...,q_T), Q_2(...)=Q_2(...), ..., Q_T(...)=Q_T(...) for all λ > 0. (Corollary: all price levels homogeneous of degree 0 in the quantity vector q_1.) - Test 6: Within Period Linear Homogeneity Test for Prices: P_1(λ p_1,...,p_T; q_1,q_2,...,q_T) = λ P_1(...), P_2(...)=P_2(...), ..., P_T(...)=P_T(...) for all λ > 0. (Corollary: all quantity levels homogeneous of degree 0 in the price vector p_1.) - Test 7: Invariance to the Ordering of Periods Test: If we permute the ordering of the periods, then the resulting price and quantity levels are equal to the same permutation of the initial price and quantity levels. - Test 8: Invariance of the Price and Quantity Levels to Changes in the Units of Measurement (proof shows appropriate scaling of α* yields identical P_t and Q_t). - Test 9: Circularity or Transitivity Test : The Price and Quantity Levels defined by (A25) satisfy: - (A30) Q_t/Q_r = (Q_t/Q_s)(Q_s/Q_r) ; P_t/P_r = (P_t/P_s)(P_s/P_r) ; 1 ≤ r < s < t ≤ T. - Interpretation: econometric price and quantity indexes are free from chain drift. - Test 10: Price Responsiveness Test: Suppose that product N is a new product that is purchased in period T. Then the price level function P_T(p_1,...,p_T; q_1,...,q_T) responds to a change in the price of this new product, p_TN; i.e., P_T(...) is not constant with respect to variations in p_TN. - Empirical note: experiments show LSLP price levels satisfy this test. - Contrast: Zhang, Johansen and Nygaard (2019) pointed out that standard multilateral indexes like GEKS, CCDI, GK or the TPD multilateral indexes do not satisfy this test: appearance of a new product does not affect the price level for the current period for those methods. Thus LSLP indexes have a possible advantage. - Limitation of LSLP: underestimation of benefits of increased choice set (new product problem) when preferences are non-linear. Example: - Two-period, two-product example: period 1 only product 1 available with budget e_1 > 0 and price p_11 > 0 so q_11 = e_1/p_11. Period 2 product 2 appears; even with e_2 = e_1 and p_21 = p_11, the true utility ratio u_2/u_1 = q_1* / q_11 ≥ 1, but solving (A20) can give Q_2 = Q_1, hence measured utility ratio equals 1 while true ratio ≥ 1. - Conclusion: linear utility models lead to quantity index too low and upward new product bias in the period when new product first appears. This is a fundamental problem for linear utility models including GK indexes, bilateral superlative indexes based on matched products and their multilateral extensions, and hedonic time-dummy models. - Remedy: use additional periods (e.g., three-period extension) to obtain information on curvature of indifference curves and approximate true nonlinear preferences (approach pursued by Diewert and Feenstra (2017) (2022)). A5. Econometric estimation of a rank 1 substitution matrix (Konüs‑Byushgens quadratic form) - Konüs and Byushgens functional form (homogeneous quadratic / rank-1 substitution matrix): - (A30) f(q) ≡ ( q·A q )^{1/2} = ( Σ_{i=1}^N Σ_{j=1}^N a_{ij} q_i q_j )^{1/2} ; a_{ij} = a_{ji} ; 1 ≤ i ≤ j ≤ N. - A is symmetric N×N with (N+1)N/2 unknown a_{ij} parameters. - Inverse demand equations become: - (A31) p_t = e_t A q_t / ( q_t·A q_t ) ; t = 1,...,T. - Share-equation form (unit invariance and validity with missing products): - (A32) s_tn = q_tn ( Σ_{j=1}^N a_{nj} q_tj ) / ( Σ_{i=1}^N Σ_{j=1}^N a_{ij} q_ti q_tj ) ; t = 1,...,T ; n = 1,...,N. - Note: if product n not available in t, q_tn and s_tn equal 0 and the equation becomes 0 = 0. *Source: Diewert and Shimizu (2024), Section 2 and Appendix material as provided.* ### 0. Thus the system of estimating equations can accommodate missing (reservation) prices and zero ### the-ottawa-group-after-30-years - 0. Thus the system of estimating equations can accommodate missing (reservation) prices and zero ### Parameterization of the KBD utility and reduction of parameters - The KBD utility function defined by (A30) involves an N by N symmetric matrix A with (N+1)N/2 unknown parameters aij. - To reduce parameters, A is defined as (A33) A ≡ ααT − ββT, where αT ≡ [α1,...,αN] and βT ≡ [β1,...,βN]. - This parameterization reduces the number of unknown parameters in A from (N+1)N/2 to 2N. - Implication noted: The matrix A will have rank at most equal to 2 with one positive eigenvalue and 1 negative or 0 eigenvalue (if β = 0N). ### Share equations and validity with missing products - With A defined by (A33), the system of share equations becomes (A34): - stn = qtn[αn α·qt − βn β·qt]/[(α·qt)2 − (β·qt)2]; t = 1,...,T; n = 1,...,N. - Equations (A34) are valid when products are missing because when product n is missing in period t, s tn = q tn = 0. - Under the assumption that f(q,α,β) defined by (A35) is positive, marginal utilities have the form (A36): - fn(q,α,β) = [αn α·q − βn β·q]/[(α·q)2 − (β·q)2]1/2. - Generalized price relation for available products (A37): - ptn = et fn(qt,α,β)/f(qt,α,β); t = 1,...,T; n ∈ S(t). - Multiplying (A37) by qtn/et yields share equations (A38): - stn = qtn fn(qt,α,β)/f(qt,α,β) = qtn[αn α·qt − βn β·qt]/[(α·qt)2 − (β·qt)2]; t = 1,...,T; n = 1,...,N. ### Least squares estimation problem, normalization, and identification - Estimates for αn and βn are obtained by solving the nonlinear least squares minimization (A38): - minα,β Σt=1T Σn=1N {stn − qtn(αn α·qt − βn β·qt)/[(α·qt)2 − (β·qt)2]}2. - Non-uniqueness: if (α*,β*) solves (A38), then (λ α*, λ β*) also solves it for any nonzero λ. - Identification requires at least one normalization (e.g., α1 = 1) and assumptions on product availability: - product 1 present in all periods; - each product present in at least one period; - if product n is only available in one period in the window, impose βn = 0. - Practical note: when a product first appears in the last period T, αn can be estimated but βn cannot; therefore set βn = 0 for such products. ### Aggregate levels, reservation prices, and substitution matrix - Period t aggregate quantity and price levels with solution (α*,β*) are defined by (A39): - Qt ≡ f(qt,α*,β*) = [(α*·qt)2 − (β*·qt)2]1/2; Pt ≡ et/Qt; t = 1,...,T. - These KBD price and quantity levels satisfy the same 10 tests as the Least Squares Linear Preferences definitions (A25) in section A2. - Hicksian reservation prices for products n not present in period t are computed after estimation via (A40): - ptn* ≡ et fn(qt,α*,β*)/f(qt,α*,β*); t = 1,...,T; n ∉ S(t). - These imputed reservation prices were not used in the nonlinear least squares minimization (A38) and are calculated after obtaining (α*,β*). - The period t inverse substitution matrix (N by N matrix of second order partial derivatives) is denoted ∇2f(qt,α*,β*). For the KBD functional form (A35), (A41) gives: - ∇2f(qt,α*,β*) ≡ − [f(qt,α*,β*)]−3 [α*(β*·qt)2 − β*(α*·qt)2][α*(β*·qt)2 − β*(α*·qt)2]T. - Properties: - For a general linearly homogeneous and concave f(q), ∇2f(qt) must be negative semidefinite and satisfy ∇2f(qt) qt = 0N; rank at most N−1. - If β* = 0N, then ∇2f(qt,α*,β*) = 0N 0N T (N by N matrix of zeros). - If α* and β* are both nonzero and α* ≠ β*, the substitution matrix defined by (A41) will have rank equal to one. - The functional form f(q,α,β) defined by (A35) is a semiflexible functional form of rank 1 (Diewert and Wales (1988) terminology). ### Practical considerations, implementation strategy, and applicability - Regularity conditions required: ensure (α·qt)2 − (β·qt)2 > 0 to compute the positive square root f(q,α,β). - In practice, solve the nonlinear least squares by first setting β = 0N and solving the nonlinear least squares defined by (A20) to get starting values for α; choose starting values for β close to 0N. - Recommendation for real-time index construction: use the Modified Expanding Window method (as in Diewert and Shimizu (2024) section 7): - Start with a window of 12 months of data and estimate the KBD model for that window. - For subsequent months, expand the window by adding a new month of share equations and re-estimate to obtain new Qt and Pt. - If the index can be revised, use the new estimates to form revised price indexes back to period 1. - If the index cannot be revised, use the current window price and quantity levels for the last month in the window as the new last period levels. - Empirical guidance: - Estimation accuracy for αn and βn improves as the window of observations expands. - Diewert and Feenstra (2017) (2022) show that estimating a rank 1 inverse substitution matrix can work well for datasets with a relatively small number of products. - For large numbers of products, the Least Squares Linear Preferences model can be estimated for many products more easily; estimating the KBD model with large datasets becomes problematic. - Practical approach for large datasets: decompose into smaller datasets, eventually change the base period, or use linear models (LSLP, GK, or Weighted Time Product Dummy models) that often generate indexes very close to KBD indexes. - The KBD model is particularly useful when a number of genuinely new products with added value are appearing. *the-ottawa-group-after-30-years* ### References ### The Ottawa Group After 30 Years — References ### Quality adjustment, time-dummy methods, and linking choices - Krsinich (2016; 401) noted that estimates for the quality adjustment parameters αn in the Time Product Dummy method are not reliably determined until the products have been present in the marketplace for several periods. - It seems “best” to link the price and quantity level estimates for the last period in the current period regression back to the price level in period 1 because the period 1 price level set equal to 1 is the “true” period 1 price level and never changes whereas our new price levels for periods 2 to t1 are changing as we add another period of data to the regression. Thus we are following Krsinich (2016) in our choice of “best” linking period. - We are also following Chessa (2016) in utilizing the idea of an expanding window of observations. - The idea of using an infinitely expanding window of observations arose in Diewert (2022) in his discussion of the predicted share method that used dissimilarity measures to link the current period to past periods. ### Scanner data, micro-indices, and chain drift - Abdirahman, M., D. Coyle, R. Heys and W. Stewart (2020), “A Comparison of Deflators for Telecommunications Services Output’, Economie et Statistique / Economics and Statistics 517-518-519, 103–122. - Abdirahman, M., D. Coyle, R. Heys and W. Stewart (2022), “Telecoms Deflators: A Story of Volume and Revenue Weights”, Economie et Statistique / Economics and Statistics 530-531, 43-59. - Chessa, A.G. (2016), “A New Methodology for Processing Scanner Data in the Dutch CPI”, EURONA 1, 49–69. - de Haan, J. (2004), “Estimating Quality-Adjusted Unit Value Indices: Evidence from Scanner Data,” Paper presented at the Seventh EMG Workshop, Sydney, Australia, December 12–14. - de Haan, J. (2008), “Reducing Drift in Chained Superlative Price Indexes for Highly Disaggregated Data”, paper presented at the Economic Measurement Workshop 08, University of New South Wales, December 10. - de Haan, J. (2015), “Rolling Year Time Dummy Indexes and the Choice of Splicing Method”, website: https://stats.unece.org/ottawagroup/meeting/14 - de Haan and E. Opperdoes (1997), “Estimation of the Coffee Price Index Using Scanner Data: the Choice of the Micro Index”, website: https://stats.unece.org/ottawagroup/meeting/3 - de Haan, J. and H. van der Grient (2011), “Estimating Chain Drift in Price Indexes Based on Scanner Data”, website: https://stats.unece.org/ottawagroup/meeting/11 - de Haan, J. and H. van der Grient (2011), “Eliminating Chain Drift in Price Indexes Based on Scanner Data”, Journal of Econometrics 161, 36-46. - Diewert, W.E. (2023), “Scanner Data, Elementary Price Indexes and the Chain Drift Problem”, pp. 445-606 in Advances in Economic Measurement, D. Chotikapanich, A.N. Rambaldi and N. Rhode (eds.), Singapore: Palgrave Macmillan. - Diewert, W.E. and K.J. Fox (2021) “Substitution Bias in Multilateral Methods for CPI Construction Using Scanner Data,” Journal of Business and Economic Statistics 40:1, 355-369. - Ivancic, L., K.J. Fox and W.E. Diewert (2009), “Scanner Data, Time Aggregation and the Construction of Price Indexes”, website: https://stats.unece.org/ottawagroup/meeting/11 - Ivancic, L., W.E. Diewert and K.J. Fox (2011), “Scanner Data, Time Aggregation and the Construction of Price Indexes”, Journal of Econometrics 161, 24-35. - Ivancic, L. and K.J. Fox (2013a), “Understanding Price Variation across Stores and Supermarket Chains: Some Implications for CPI Aggregation Methods”, Review of Income and Wealth 59, 629-647. - Ivancic, L. and K.J. Fox (2013b), “Can Dissimilarity Indexes Resolve the Issue of When to Chain Price Indexes?”, Economic Letters 118:1, 6-9. - Reinsdorf, M. (1999), “Using Scanner Data to Construct CPI Basic Component Indexes”, Journal of Business & Economic Statistics 17:2, 152-160. - Silver, M. (1995), “Elementary Aggregates, Micro-Indices and Scanner Data: Some Issues in the Compilation of Consumer Price Indices”, Review of Income and Wealth 41, 427-435. - Zhang, L.-C., I. Johansen and R. Nygaard (2019), “Tests for Price Indices in a Dynamic Item Universe” - von Auer, L. (2019), “The Nature of Chain Drift: Implications for Scanner Data Price Indices”, website: https://stats.unece.org/ottawagroup/meeting/16 ### Hedonic methods, quality adjustment, and new goods valuation - Court, A.T. (1939), “Hedonic Price Indexes with Automotive Examples”, pp. 99-117 in The Dynamics of Automobile Demand, New York: The General Motors Corporation. - Dulberger, E. (1989), “The Application of a Hedonic Model to a Quality Adjusted Price Index for Computer Processors, pp. 37-75 in Technology and Capital Formation, D.W. Jorgenson and W. Landau (eds), Cambridge MA: MIT Press. - Diewert, W.E. (2022), “Quality Adjustment Methods”, Chapter 8 in Consumer Price Index Theory, Washington D.C.: International Monetary Fund, Washington D.C. Website: https://www.imf.org/en/Data/Statistics/cpi-manual - Diewert, W.E. and R.C. Feenstra (2017), “Estimating the Benefits and Costs of New and Disappearing Products”, Discussion Paper 17-10, Vancouver School of Economics, University of British Columbia, Vancouver, B.C., Canada, V6T 1L4. - Diewert, W.E. and R.C. Feenstra (2022), “Estimating the Benefits of New Products”, pp. 437-473 in Big Data For the Twenty-First Century Economic Statistics, K.G. Abraham, R.S. Jarmin, B.C. Moyer and M.D. Shapiro (eds.), Chicago: University of Chicago Press. - Brynjolfsson, E., A. Collis, W. E. Diewert, F. Eggers and K. J. Fox (2019), “GDP-B: Accounting for the Value of New and Free Goods in the Digital Economy.” NBER Working Paper 25695, Cambridge, MA. - Hausman, J.A. (1996), “Valuation of New Goods under Perfect and Imperfect Competition”, pp. 20 -236 in The Economics of New Goods, T.F. Bresnahan and R.J. Gordon (eds.), Chicago: University of Chicago Press. - Hausman, J.A. (1999), “Cellular Telephone, New Products and the CPI”, Journal of Business and Economic Statistics 17:2, 188-194. - Triplett, J. (2004), Handbook on Hedonic Indexes and Quality Adjustments in Price Indexes, Directorate for Science, Technology and Industry, DSTI/DOC(2004)9, Paris: OECD. - Diewert, W.E. and R.C. Feenstra (2022), “Estimating the Benefits of New Products”, pp. 437-473 in Big Data For the Twenty-First Century Economic Statistics. ### Price index theory, history, axiomatic approaches, and classical literature - Afriat, S.N. (1972), “The Theory of International Comparisons of Real Income and Prices”, pp. 13-69 in International Comparisons of Prices and Outputs, D.J. Daly (ed.), NBER, New York: Columbia University Press. - Balk, B.M. (2008), Price and Quantity Index Numbers, New York: Cambridge University Press. - Diewert, W.E. (1974), “Applications of Duality Theory,” pp. 106-171 in M.D. Intriligator and D.A. Kendrick (ed.), Frontiers of Quantitative Economics, Vol. II, Amsterdam: North-Holland. - Diewert, W.E. (1976), “Exact and Superlative Index Numbers”, Journal of Econometrics 4, 114-145. - Diewert, W.E. (1978), “Superlative Index Numbers and Consistency in Aggregation”, Econometrica 46, 883-900. - Diewert, W.E. (1992), “Fisher Ideal Output, Input and Productivity Indexes Revisited”, Journal of Productivity Analysis 3, 211-248. - Diewert, W.E. (1995a), “Axiomatic and Economic Approaches to Elementary Price Indexes”, website: https://stats.unece.org/ottawagroup/meeting/1 - Fisher, I. (1911), The Purchasing Power of Money, London: Macmillan. - Fisher, I. (1922), The Making of Index Numbers, Houghton-Mifflin, Boston. - Jevons, W.S., (1865), “The Variation of Prices and the Value of the Currency since 1782”, Journal of the Statistical Society of London 28, 294-320; reprinted in Investigations in Currency and Finance (1884), London: Macmillan and Co., 119-150. - Konüs, A.A. (1924), “The Problem of the True Index of the Cost of Living”, translated in Econometrica 7, (1939), 10-29. - Laspeyres, E. (1871), “Die Berechnung einer mittleren Waarenpreissteigerung”, Jahrbücher für Nationalökonomie und Statistik 16, 296-314. - Paasche, H. (1874), “Über die Preisentwicklung der letzten Jahre nach den Hamburger Borsennotirungen”, Jahrbücher für Nationalökonomie und Statistik 12, 168-178. - Törnqvist, L. (1936), “The Bank of Finland's Consumption Price Index”, Bank of Finland Monthly Bulletin 10, 1-8. - Törnqvist, L. and E. Törnqvist (1937), Vilket är förhällandet mellan finska markens och svenska kronans köpkraft?”, Ekonomiska Samfundets Tidskrift 39, 1-39 reprinted as pp. 121-160 in Collected Scientific Papers of Leo Törnqvist, Helsinki: The Research Institute of the Finnish Economy, 1981. - Stone, R. (1956), Quantity and Price Indexes in National Accounts, Paris: OECD. - Theil, H. (1967), Economics and Information Theory, Amsterdam: North-Holland Publishing. ### CPI construction, aggregation, seasonality, and measurement practice - Armknecht, P.A., B.R. Moulton, K,J. Stewart (1994), “Improvements to the Food at Home, Shelter and Prescription Drug Indexes in the U.S. Consumer Price Index”, website: https://stats.unece.org/ottawagroup/meeting/1 - Australian Bureau of Statistics (2016), “Making Greater Use of Transactions Data to Compile the Consumer Price Index”, Information Paper 6401.0.60.003, November 29, Canberra: ABS. - Balk, B.M. (1980), “A Method for Constructing Price Indices for Seasonal Commodities”, Journal of the Royal Statistical Society, Series A 143, 68-75. - Balk, B.M. (1981), “A Simple Method for Constructing Price Indices for Seasonal Commodities”, Statistische Hefte 22 (1), 1–8. - Dalén, J. (1992), “Computing Elementary Aggregates in the Swedish Consumer Price Index,” Journal of Official Statistics 8, 129-147. - Dalén, J. (1994), “Sensitivity Analyses for Harmonizing European Consumer Price Indices”, pp. 147-171 in International Conference on Price Indices: Papers and Final Report, First Meeting of the International Working Group on Price Indices, November, Ottawa: Statistics Canada. Website: https://stats.unece.org/ottawagroup/meeting/1 - Mudgett, B.D. (1955), “The Measurement of Seasonal Movements in Price and Quantity Indexes”, Journal of the American Statistical Association 50, 93-98. - Reinsdorf, M. (1993) “The Effect of Outlet Price Differentials on the U.S. Consumer Price Index,” pp. 227-254 in M.C. Foss, M.E. Manser and A.H. Young (eds.), Price Measurements and Their Uses, NBER Studies in Income and Wealth 57, Chicago: University of Chicago Press. - Reinsdorf, M. (1998) “Formula Bias and Within-Stratum Substitution Bias in the US CPI”, Review of Economics and Statistics 58:2, 175-187. - Moulton, B.R. (1993) “Basic Components of the CPI: Estimation of Price Changes,” Monthly Labor Review 116 December, 13-24. - Hill, P. (1993), ‘‘Price and Volume Measures’’, pp. 379-406 in System of National Accounts 1993, Brussels/Luxembourg, New York, Paris, New York, and Washington, D.C.: Commission of the European Communities, IMF, OECD, World Bank and United Nations. - ILO/IMF/OECD/UNECE/Eurostat/The World Bank (2004), Consumer Price Index Manual: Theory and Practice, Peter Hill (ed.), Geneva: International Labour Office. - IMF/ILO/UNECE/Eurostat/OECD/The World Bank (2020), Consumer Price Index Manual: Concepts and Methods 2020, B. Graff(ed.), Washington D.C.: The International Monetary Fund. ### Selected methodological, historical, and policy-relevant contributions - Boskin, M.J., E. Dulberger, R, Gordon, Z. Griliches, and D.W. Jorgenson (1996), Toward a More Accurate Measure of the Cost of Living, Final Report to the U.S. Senate Finance Committee, Washington, D.C. Website: https://www.ssa.gov/history/reports/boskinrpt.html - Boskin, M.J., E. Dulberger, R, Gordon, Z. Griliches, and D.W. Jorgenson (1998), “Consumer Prices, the Consumer Price Index and the Cost of Living”, Journal of Economic Perspectives 12:1, 3-26. - Stiglitz, J.E., A. Sen and J.P. Fitoussi (2009), Report by the Commission on the Measurement of Economic Performance and Social Progress, Eurostat. https://ec.europa.eu/eurostat/documents/118025/118123/Fitoussi+Commission+report. - Greenlees, J.S. (2006), “The BLS Response to the Boskin Commission Report”, International Productivity Monitor Number 12 (Spring), 23-41. - Schreyer, P. (2012). “Output, Outcome and Quality Adjustment in Measuring Health and Education Services”, Review of Income and Wealth 58(2), 257–278. - Schreyer, P. and W.E. Diewert (2014), “Household Production, Leisure and Living Standards”, pp. 89-114 in Measuring Economic Sustainability and Progress, D.W. Jorgenson, J.S. Landefeld and P. Schreyer (eds.), Chicago IL: University of Chicago Press. - Burnett-Issacs, K., N. Huang and W.E. Diewert (2020), “Developing Land and Structure Price Indexes for Ottawa Condominium Apartments”, Journal of Official Statistics 36, 763-802. --- _Source: https://www.imf.org/-/media/files/data/cpi/companion-publication/the-ottawa-group-after-30-years.pdf_