Workshop on Price Index Compilation Issues February 23-27, 2015 Data Collection Issues Gefinor Rotana Hotel, Beirut, Lebanon

2 Data Collection Methods What methods can be used to collect data? What are the advantages of each method? What are the disadvantages? What is the most effective method? How do we decide which method to use for the CPI? What factors? What is ideal method – CPI vs PPI? One size fits all?? February 23-27, 2015METAC and IMF Statistics Dept.

Period to Period Price Collection Who do we collect data from? What types of data should be collected and why? How often? How do we prepare for price collection? Which data collection method is best? What are the challenges? February 23-27, 2015METAC and IMF Statistics Dept. 3

Data Collection Forms design Complete specifications Prices (including historic?) Reasons for price change New specifications All data collected returned to HQ February 23-27, 2015METAC and IMF Statistics Dept. 4

Data Validation Includes Data editing Non-statistical & statistical checking Outliers Missing prices Credibility checking February 23-27, 2015METAC and IMF Statistics Dept. 5

The Challenge “CPI estimates are subject to errors that may arise from a variety of sources” –ILO Because prices in shops can change and the CPI production timetable can be very tight, editing needs to be done in a short period of time. Precision matters! February 23-27, 2015METAC and IMF Statistics Dept. 6

The Challenge “CPI estimates are subject to errors that may arise from a variety of sources” –ILO Because prices in shops can change and the CPI production timetable can be very tight, editing needs to be done in a short period of time. Precision matters! Validation feeds into long-term methodological development February 23-27, 2015METAC and IMF Statistics Dept. 7

The Challenge “CPI estimates are subject to errors that may arise from a variety of sources” –ILO Because prices in shops can change and the CPI production timetable can be very tight, editing needs to be done in a short period of time. Precision matters! Validation feeds into long-term methodological development February 23-27, 2015METAC and IMF Statistics Dept. 8

Data Validation in the field Facilitated by electronic data capture What are the appropriate checks? February 23-27, 2015METAC and IMF Statistics Dept. 9

Data Validation in the field Facilitated by electronic data capture What are the appropriate checks? February 23-27, 2015METAC and IMF Statistics Dept. 10

Data Validation at the Office Two steps: Detection of possible errors Verification and correction All collected prices should be examined. But not all with the same degree of scrutiny. Most time and price editing effort should focus in areas where. Elementary aggregates have relatively small price samples. Elementary aggregates with high expenditure weights. Elementary aggregates with large weights but few quotes. February 23-27, 2015METAC and IMF Statistics Dept. 11

Data Editing Two tools available for checking for errors and outliers: Non-statistical checking Statistical checking The following techniques work best with large volumes of data. But still applicable to small samples. February 23-27, 2015METAC and IMF Statistics Dept. 12

Data Editing – non-statistical checking Non-statistical checking Manual inspection of data Includes price change and min-max tests Comparisons can be expanded to include comparison with prices of similar products collected from similar outlets by different collectors in the same region (i.e. movements/levels compared with price data collected in parallel from other outlets). Refined over time. Should not automatically result in deletion February 23-27, 2015METAC and IMF Statistics Dept. 13

Data editing - Statistical checking The main conceptual difference with non-statistical checking is that statistical checking techniques automatically calculate the limits for acceptable movement. Automatic adjustment of acceptable limits as new price data is received. Data intensive for reliable results. Best suited for use at the head and regional offices. Compares each price change with changes in the other items from a given price sample. The price sample is the one to which the item being checked belongs. The sample for testing may be a combination of price samples for similar products. February 23-27, 2015METAC and IMF Statistics Dept. 14

Statistical checking - methods The various methods Median and quartile values The acceptable limits of price change are set as a predefined multiple of the range between the median and the quartiles. Assumes normal distribution Modified median and quartile values Transforms distances from the median to increase effectiveness of check Tukey algorithm Applies adjustment to the inter quartile range February 23-27, 2015METAC and IMF Statistics Dept. 15

A Key Benefit of Statistical Checking The limits are set by the data and they can change over time As further data is received As price levels change But these methods are data intensive. Remember that the objective is not to flag observations for deletion but to identify those that need closer scrutiny. February 23-27, 2015METAC and IMF Statistics Dept. 16

Dealing with Outliers Outliers are defined as price movements that are exceptionally large compared with the majority of movements. The strategy adopted by most statistical institutes is to reduce the impact of exceptional observations rather than exclude them from the index. The tests for outliers are the same as those for identifying potential errors, as described (described above). If outliers are to be modified, they are usually modified to lie on the pre-defined boundaries of acceptable movement or to be imputed by the movement of a suitable sample of prices. Price collectors & supervisors are responsible for providing information about the reasons for extreme price movements or levels and why price quotes have been accepted as valid. February 23-27, 2015METAC and IMF Statistics Dept. 17

Missing prices – minimising the number when in the field There are a number of ways of minimising the occurrence of missing observations. For example. Maintaining the relevance of the sample of items priced. Early contact with respondents that are usually late returning forms. Substitution and quality change/adjustment. Separate issue. February 23-27, 2015METAC and IMF Statistics Dept. 18

Credibility checking Other types of errors are more demanding Exceeding pre-defined movement limits Other errors may require checking with the shopkeeper. Problems caused by unusual prices and price movements can be avoided by training price collectors to recognise these situations and to collect relevant explanatory information during the initial price collecting visit. February 23-27, 2015METAC and IMF Statistics Dept. 19