Over the last two decades, CPIs have become multi-source reliant statistics where prices are obtained not only from traditional price collection in the field but also from new data sources such as: scanner data; administrative data; and data from the Internet using web scraping techniques. Depending on the data source, different strategies can be adopted for constructing elementary aggregates and for compiling consumer price indices (CPIs). In particular, the widespread use of scanner data, which contain sales transaction information on prices paid, quantities purchased and product characteristics for all items of some outlets in a certain period of time, poses certain methodological demands on index methods.

Multilateral index methods, such as the Gini, Eltetö, Köves and Szulc (GEKS), Geary Khamis (GK), the Time product Dummy (TPD) methods and spatial chaining methods, traditionally applied in the context of spatial price comparisons, have provided a solution to a number of problems encountered in the temporal context with bilateral methods. More specifically, multilateral methods allow to take into account all the products that are available in the different periods by also explicitly weighting each product according to its importance in each period. These methods have the potential to reduce the chain-drift associated with chained bilateral indices (De Haan and Van der Grient, 2011; Ivancic, Diewert and Fox, 2011).

Given these advantages, multilateral index methods, have been recommended as suitable price index compilation methods for transaction data, despite their additional complexity compared with bilateral methods. Nevertheless, consensus on key issues has yet to be achieved.

The use of multilateral index methods based on scanner data gives rise to an aggregation problem. Product specification has been recognized as a critical step that could jeopardize any gains in bias reduction that we would typically expect from using scanner data. While scanner data helps reducing lower-level substitution bias, other biases can appear because products are specified too tightly or too broadly. In addition, in a dynamic product universe, products may enter or exit with unusually high or low prices. It is known that this creates biases in traditional matched model methods and these biases propagate to multilateral index methods. Aggregation of prices or quantities when commodities disappear or appear has been regarded as one of the most difficult tasks (Lamboray, and Krsinich, 2015; Abe, Enda, Inakura and Tonogi 2015).

There are a variety of multilateral indices which all share similar basic features, but can also give different estimates of price changes. At present there is no real consensus as to which combination of approaches is the best way to calculate a multilateral index (Diewert and Fox, 2022). In order to understand which multilateral method should be used to aggregate detailed price and quantity data an approach based on the definition of a “true” cost of living index may be adopted.

In addition, indices have to be updated as new price data is collected and the only way to do this without affecting measures of past price changes is to splice multilateral index numbers calculated over different windows (time periods) together (Chessa, 2019). This reintroduces a limited amount of chain drift, albeit much smaller than that seen for traditional methods.

The choice of window length also involves the tension, on the one hand, between using as much of the data as possible and, on the other, using only bilateral comparisons which are reliable. In the context of temporal comparison using multilateral indices, as time passes, recent price movements will be increasingly affected by prices and price changes in the distant past. This will result in a loss of “characteristicity”. Characteristicity, defined in the context of spatial comparisons (involving many countries and multilateral methods), requires that any set of multilateral comparisons satisfying the transitivity property should retain the essential features of the binary comparisons constructed without the transitivity requirement. An approach for solving this tension between the different levels of reliability of the bilateral indices is based on the weighted GEKS with alternative specifications of the matrix of weights based on how one may wish to measure reliability (Rao 1997, Rao and Timmer, 2003; Brunetti, Fatello, Laureti and Polidoro, 2022).

The aim of the Session is to present, discuss and evaluate various methodological approaches with particular focus on:

1. The best multilateral method to use by comparing alternative methods with some theoretical benchmark index;

2. Methodology for extending the resulting series when new observations become available;

3. Treatment of new and disappearing goods;

4. Seasonality and chain drift;

5. Use of different systems of weights for introducing information on the reliability of the underlying bilateral comparisons for the weighting matrix of GEKS method

The session will be organized with the presentation of five papers. Then the floor will be opened for discussion by participants.

1. Title: “Multilateral methods and product specification”

Authors: Claude Lamboray

Affiliation: Eurostat and Statistics Luxemburg

Speaker: Claude Lamboray

Email: claude.lamboray@statec.etat.lu 

Abstract/Short summary: Product specification has been recognized as a critical step that can have a significant impact on the results obtained with multilateral methods. Tightly specified products may cause a bias because there may not be sufficient product matches over time. Broadly specified products may cause a bias because there may be quality differences between the underlying transactions that make up the product. In this paper, we examine these biases in the context of multilateral methods and provide tentative guidance on which product specification to select in practice.

2)Title (tentative): Comparing index extension methods for multilateral methods

Authors: Antonio Chessa

Speaker: Antonio Chessa

Affiliation: Statistics Netherlands CBS

Abstract/Short summary: Multilateral indices are transitive on a fixed time interval, but an essential question is how the drift-free properties can be preserved when data from the next period become available. The time window has to be adjusted in order to include new data. The indices calculated on the adjusted window may differ from the previously calculated indices, which, however, cannot be revised. Various methods have been suggested to overcome this issue but the behaviour of different methods still needs to be better understood. This paper will present the results of a comparative study of extension methods for index series. 

3) Title: "On the Aggregation of Commodities with Product Turnover"

Authors: Naohito Abe (Hitotsubashi University), Noriko Inakura (Osaka Sangyo University), DS Prasada Rao (University of Queensland), Akiyuki Tonogi (Hitotsubashi University)

Speaker: Naohito Abe

Affiliation: Hitotsubashi University

Email: nabe@ier.hit-u.ac.jp  naohito.abe@gmail.com 

Abstract/Short summary: Aggregation of prices or quantities when commodities disappear or appear has been regarded as one of the most difficult tasks. Product turnover occurs at various aggregation levels from commodity level scanner data, seasonal products such as fruits, to categorical aggregates such as foreign trips during the Covid19-pandemic. The current method to handle product turnovers is to use (1) chain indexes or (2) Feenstra's variety effects. Both methods are subject to chain drifts. In addition, the second method cannot be applied to aggregate level unless the elasticity of substitution is greater than unity. In this paper, we propose a method to aggregate prices and quantities with product turn overs that is free from chain drifts. While the standard Könus' cost of living assumes that the only supply shocks cause the changes in prices and quantities, our method can handle both demand and supply shocks. A procedure to decompose the actual changes into demand and supply shocks is also provided, which can be applied to various levels of aggregation, from scanner data to international comparisons.

4) Title: “Scanner Data and the Use of Multilateral Index Number Methods”

Author: W. Erwin Diewert

Speaker: W. Erwin Diewert

Affiliation: University of British Columbia and University of New South Wales

Email:  erwin.diewert@ubc.ca

Abstract/Short summary: The paper looks at the use of multilateral index number methods to address the chain drift problem that arises when scanner data is aggregated. The chain drift problem arises in the context of price and quantity data that is subject to frequent fluctuations due to either dynamic pricing (i.e., sale prices occur frequently) or to seasonal fluctuations. A multilateral index number method which looks very promising is a method that links periods based on how similar the structure of prices and quantities is when constructing bilateral indexes linking the data for two periods. 

5) Title (tentative) On the use of weighted GEKS and TPD for the calculation of consumer price indices

Authors: Tiziana Laureti (University of Tuscia), Federico Polidoro (Istat) and Prasada DS Rao (The University of Queensland)

Speaker: Tiziana Laureti

Affiliation: University of Tuscia

 Email: laureti@unitus.it

Abstract/Short summary: The paper focuses on the analysis of the properties of the weighted GEKS, based on superlative bilateral indexes, and Weighted Time Product Dummy methods for computing CPIs using scanner data. Different alternative systems of weights are considered for introducing information on the reliability of the underlying binaries for the weighting matrix.  We also compare the different methods with a theoretical benchmark index based on the calculation of a target “true” cost of living index under the hypothesis of Constant Elasticity of Substitution (CES) purchaser preferences.

Proposal Submitted for sponsorship by the ISI or Associations(s): International Statistical Institute and International Association for Official Statistics

Organizer: Tiziana Laureti, Full Professor of Economic Statistics, Tuscia University, Italy Director of the Department of Economics, Engineering

Chair: Luigi Biggeri, Emeritus Professor, University of Florence, Italy and past president of Italian National Statistical Office (Istat)

References

Abe, N., Enda, T., Inakura, N., and Tonogi, A. (2015). Effects of new goods and product turnover on price indexes. Institute of Economic Research, Hitotsubashi University.

Brunetti A, Fatello S, Laureti T., Polidoro F. (2022) The use of weighted GEKS for the calculation of consumer price indices: an experimental application to Italian scanner data, In 17th Meeting of the Ottawa Group on Price, Rome, Italy 7-10 June 2022.

Chessa, A. G. (2019). A comparison of index extension methods for multilateral methods. In 16th Meeting of the Ottawa Group on Price, Rio de Janeiro, Brazil 08-10 May 2019

De Haan, J., & Van der Grient, H. A. (2011). Eliminating chain drift in price indices based on scanner data. Journal of Econometrics, 161(1), 36-46.

Diewert, W. E., & Fox, K. J. (2022). Substitution Bias in Multilateral Methods for CPI Construction. Journal of Business & Economic Statistics, 40(1), 355-369.

Eurostat (2022) Guide on Multilateral Methods in the Harmonised Index of Consumer Prices, Luxembourg: Publications Office of the European Union, 2022

Ivancic, L., Diewert, W. E., and Fox, K. J. (2011). Scanner data, time aggregation and the construction of price indexes. Journal of Econometrics, 161(1), 24-35.

Lamboray, C., and Krsinich, F. (2015, May). A modification of the GEKS index when product turnover is high. In 14th meeting of the Ottawa Group.

Rao, D. S. Prasada, (1997) Aggregation Methods for International Comparison of Purchasing Power Parities and Real Income: Analytical Issues and Some Recent Developments, Bulletin of International Statistical Institute, LVII, 197–200.

Rao, D. S. Prasada, and Timmer, M. P. (2003). Purchasing Power Parities for Industry Comparisons Using Weighted Elteto–Koves–Szulc (EKS) Methods. Review of Income and Wealth, 49(4), 491-511.

The Proposed session will be of considerable interest to many WSC attendees and especially to official statisticians from national statistical agencies across the world who are currently using or considering the use of electronic point-of-sales scanner data from retail outlets for computing their Consumer Price Index (CPIs). Price paid by consumers can now be observed at the point-of-sale across many products at high frequency, potentially allowing NSIs to, for example, to compile and publish reliable measures of monthly or even weekly measures of price change. However, there is has been no consensus on the most suitable method for calculating price indices with high frequency transactions data, such as scanner data. Scanner data have the potential for improving the timeliness of Consumer Price Index (CPI) releases and the ability to better capture changes in consumer expenditure patterns during pandemics and other crises.

Conventional fixed basket price indexes are ill-suited for this task as they are not able to capture rapid product turnover thus becoming quickly unrepresentative of consumer spending patterns, while chained indices often report excessive and unrealistic price changes, and suffer from ‘chain drift’.

Multilateral price index numbers, designed primarily for the purpose of spatial (regional as well as cross-country) price comparisons, have been advocated as a possible solution to the problem of chain drift and especially for treatment of scanner data. However, there are many multilateral index methods for NSIs to choose from and many choices to make when applying them, including sampling designs, product specifications, splicing methods to extend these indices when data from new time periods are added. Recently, Eurostat published a guide, “Multilateral Methods in the Harmonised Index of Consumer Prices” to support countries in understanding and implementing multilateral methods in the context of the HICP (Eurostat, 2020).

Given the current state of developments with respect to data availability and choice of index number methods, there is an urgent need to examine the issues involved and provide advice and recommendations to national statistical offices.  The proposed session brings together experts working in this field with the aim of evaluating alternative methods to handle high-frequency price data and provide recommendations to practitioners. Outstanding questions like those illustrated above need to be resolved if national statistical agencies are to take maximum advantage of the data revolution.

Organiser: Prof. Tiziana Laureti 

Chair: PROF. EM. Luigi Biggeri  

Speaker: Antonio Chessa 

Speaker: Erwin Diewert 

Speaker: Mr Claude Lamboray 

Speaker: Naohito Abe 

Speaker:  Prof. Tiziana Laureti