1. A Matched-Model Geometric Mean Price Index for Supermarket Scanner Data Nicolette de Bruijn Peter Hein van Mulligen Jan de Haan Sixth EMG Workshop, 13 – 15 December 2006, UNSW, Sydney

2. Contents Historical overview Advantages of using scanner data Daily practice Matched-model approach Simulations on scannerdata and results Conclusions Future plans

3. Historical overview 2002 Introduction of use of scanner data in the CPI - 2 main supermarket organisations - field observations in these chains were cancelled 2005 Base year shift to 2004 and some improvements 2007 Base year shift to 2006 1 supermarket

4. Advantages of using scanner data (1) Improving quality: Large numbers of items Average real transaction prices No more observation mistakes Detailed and better weights Efficiency: Less survey work in the field Less administrative burden for retailers

5. Advantages of using scanner data (2) COICOPDescriptionArticles in scanner data Articles in field survey 011140Bakery products59621 011430Cheese31514 012120Tea1051 More articles in scanner data than field survey articles: abundance of scanner data Descriptions of articles in scanner data are narrower than descriptions for field survey articles

6. Use of the data EANs: unique products EANs must be classified (COICOP/CBL) All data are used for construction of weights Weights per EAN on basis of turnover shares New basket and new weights each year Prices: unit values per EAN per supermarket Laspeyres index per COICOP/CBL-group per supermarket Scanner data indexes are aggregated with indexes based on field surveys for other supermarkets and shop types

7. Why this study? The current method is time-consuming - classification of EANs into CBL/COICOP- groups - choosing successors for disappearing EANs Therefore it is impossible to use more scanner data from other supermarkets CBS would like to scale up on scanner data of supermarkets A more efficient method is needed  matched model without explicit QA

8. Differences between current method and proposed method Geometric mean vs. Arithmetic mean Explicit QA vs. class mean imputation Fixed weight per EAN vs. no weights Fixed basket vs. monthly basket

9. Benchmark versus scenario 1 Benchmark: CBL index on basis of geometric mean Scenario 1: CBL index on basis of arithmetic mean (current situation) Advantages of geometric mean: - substitution, less sensitive in heterogenic groups

10. Scenario 2: disappearing EANs About 100 disappearing EANs per month Imputation for unimportant EANs Important EANs are ‘replaced’ by hand possibility to correct explicitly for quality changes Finding successors and applying explicit quality corrections is time-consuming Scenario 2: no explicit quality adjustments but average index change of CBL-group

11. Scenario 3: weights Scenario 3: no explicit weights are used, each EAN gets the same weight Problems scenario 3: weights are required because of heterogeneity within CBL-groups Equal weighting yields unacceptable differences for CBL- and COICOP-groups But fixed weights are necessary, as monthly weights result in an upward drift

12. Results

13. Matched-model simulation Monthly basket of matched EANs Geometric mean and no explicit adjustments for quality changes Using monthly weights or no weights at all is not appropriate, therefore: rough weighing Sample on basis of turnover shares per CBL-group Variation in coverage: 60%, 70%, …, 95% NB: simulation on (official) ‘basket items’

14. Results matched model

15. Conclusions When the process is fully automated we can scale up the amount of supermarket scanner data Classifying EANs into COICOP/CBL-groups will remain time consuming; a solution has to be found before scaling up

16. The future 2007 Pilot study: implementation of matched model approach for 2 supermarkets to detect practical and logistic difficulties 2008 –Matched-model geometric mean index in production –Reduction of field surveys in supermarkets