Editing and Validation Prices and PPPs* ICP-Western Asia Regional Meeting Beirut, Lebanon May 26, 2005 Yonas Biru *Presentation adopted from Chapter 7 of the ICP Handbook

Editing & Validation The primary objective is to eliminate or reduce the incidence of non-sampling errors & to correct them when detected The ICP uses a hierarchy of checking and validation procedures First, outliers of individual prices are identified and screened at item level within a country Second, outliers of average prices can be identified and screened for different products within the same basic heading within the same country Third, outliers of average prices are identified and screened for the same products in different countries.

Tool Pack For Data Validation The Tool Pack Supports both intra-country (Tables 1 and 2) and inter-country data editing and validation (Quaranta Tables) Intra-country Validation Process (National Coordinators) PCM: Screening of individual price observations PAM: Screening of average prices both at product and at the BH level Inter-country Validation Process (Regional Coordinators) PAM: Screening of average prices for the same products in different countries are checked against each other

Prevention is Better than Correction The amount of data editing and validation work at the regional level depends on the quality of the data collected Good survey frame and survey management practices are critical to minimize non-sampling errors Well defined product specifications accompanied with product images and clear instruction and questionnaires help minimize product related errors Data collectors and data entry clerks should be well trained and regularly supervised Closely monitored and supervised field work helps eliminate or reduce errors

Validation Is An Iterative Process National Coordinators (NC) send edited price data to the Regional Coordinator (RC) RC computes Basic Heading PPPs, examines Quaranta Tables (QT) & sends feedback to NCs for correction or validation NCs recheck data, correct errors or validate the first submission or withdraw the data in question. RC computes Basic Heading-PPPs using revised data RC may get back to NCs for further checking & validation RC generates Basic heading PPPs using fully edited prices Editing is completed when NCs endorse their data as final Acceptance of the price data implies endorsement of BH-PPP results

Time Is Of The Essence Running the full sequence of editing and validation for a given quarter can take as long as five to six months The process must be carefully planned and tightly scheduled NCs need to realize that a delay on their end will drag the regional program RCs need to provide feedback to NCs without delay RC, however, cannot start the process before data from at least 50% of the countries are submitted Timely validation of data is crucial to put in place the necessary corrective measures for the subsequent quarter

County Level Editing & Validation Editing should start immediately after data collection and should be undertaken on a regular basis The objective Is to identify extreme observations or outliers (Outlier are observations diverge so much from the average) The policy is not to reject outliers outright but to investigate further if they are indeed outliers It is recommended that a separate editing and validation process should be carried out in each region within a country Country level editing are then carried out under the general guidance and supervision of the national coordinator Steps should be taken to avoid errors detected in the early rounds of price collection from being repeated in later rounds After the preliminary editing at the country level the data is transferred to regional coordinator for further checking

Table 1 and 2 of the Tool Pack Data editing is carried out using standard format and tables Table 1 provides Average Prices, Max and Min prices as well as Standard Deviation and Var. Co. If the Var. Co. for a given product is > 30% we go to the corresponding Table 2 for further investigation Table 2 provides individual price observations, ratio to average price, Min and Max prices, as well as Min Max ratio and “t- value” ‘t-values’ measure divergence between an individual price and the average price divided by the standard deviation At 95% degree of confidence the ‘t-ratio’ falls between 2 & -2 Ratio between individual price and average price should fall between 1.25 and.75

1 2

3

4

After Country Level Editing Data Is Transferred To RC

Regional Editing & Validation The primary objective is to eliminate or reduce the incidence of non-sampling errors by comparing prices across countries Editing is done in cooperation with National Coordinators Types of potential error sources Price Error – Wrong price is recorded for the right product, the right UOM and/or the right outlet (this may happen when data is collected or entered into the Tool Pack) Product Error – Price represents the wrong specification, including error in UOM, product specification and outlet type (this is harder to detect at the country level)

The Quaranta Tables QTs represent a set of diagnostic tables for a basic heading and one for each product level details within the BH QTs consist of data submitted by NCs and other indicators computed by RCs QTs are useful to screen the estimated national average prices for possible errors by comparing the dispersion of the average prices for the same product in different countries Data editing is done after national average prices have been converted into a common unit of currency by using Exchange Rates (XR) or PPPs.

Data Required for Quaranta Tables Average Prices Expressed in National currencies Exchange Rate (XR) for the survey period Basic Heading PPPs (the BH-PPPs are unweighted PPPs calculated by the CPRD method.) Expenditure weights at the BH level The number of products and the number of representative products priced by the country under investigation The BH weights and the number of products are useful indicators of the importance and coverage of the BH The BH weights and the number of products prices inform decision making in the validation process

Other Indicators BH-PLI: (Price Level Indexes at the BH level) This is also known as CUP price BH-PLI is defined as (BH-PPP) / (XR) The BH-PLI facilitate the comparison of price levels both across basic headings and within basic headings. XR-PRICE: These are prices expressed in a common currency PPP-PRICE Are prices expressed in PPP terms CV (Coefficient of Variation)

Coefficient Of Variations There are 3 different CVs in the Quaranta Tables Var. Co for the individual price variations on which each national average price is based The second Var. Co is the average Var. Co that measures the relative dispersion of the PPP prices or the PPP price ratios for the same product in different countries The average of CVs for the individual product prices can be used as an indicator of the overall reliability of the basic heading parities The third Var. Co is a country Var. Co. It measures the dispersion in the PPP Price Ratio for the different products in single country.

Quaranta Tables

XR-Price: National Average Prices expressed in a common currency (D) PPP-Price: National Average Prices expressed in PPP (D) PLI: (PPP-Price/EX-Price) shows price relatives – County “A” is most expensive and “E” least expensive Items: No. of items and no. of representative items in the BH Var. Co: Variation Coefficient

Quaranta Tables 1. PLI% for country “A” warrants a closer look, but the Var. Co seems OK 2. Var. Co for country “C” warrants a closer look, but the PLI% seems OK. 3. Both PLI% & Var. Co of Country “E” warrant closer investigation

Item Level Details XR- Ratio: Ratio of XR-Price to GeoMean of XR-Price PPP-Ratio: Ratio of PPP-Price to GeoMean of PPP-Price Var. Co for country “A” shows Var. Co for individual price observations The second type of variation (75.63) measures the relative dispersion of the PPP prices (or PPP Price Ratios) for the same product in different countries

Flagging and Editing Outliers XR-PRICE and the BHPPP-PRICE are used to obtain product price level indices (XR-PLI and BHPPP-PLI) Product price level indices that fall outside specified ranges are flagged. Usually 50 and 150 are used as lower and upper bound thresholds. In the first instance, both the indices are considered together What do we do if XR-PLI flags the product as an outlier, but the BHPPP-PLI did not? Usually the BHPPP-PLI is accepted. In the Quarants Tables the Var. Co is computed for PPP-PLI.

Data Validation When the BHPPP-PLI of a country is flagged, it is necessary to look closely at the average price underlying the index. All prices considered suspect are sent back to countries for verification. Usually countries either change the price or withdraw it. Countries check if the price is the wrong price, in which case they will either correct it or simply withdraw it. If the price is correct then they would confirm its validity and send it back to the regional coordinator.

Potential Sources of Errors The price may be correct but it may be on clearance The quality of the product priced is of higher or lower quality The country has priced the right product in the wrong outlet The product specification is misrepresented Product may be imported when the specification is for local brand The product is unrepresentative As the result of this process sometimes both price and representativeness could be changed Countries may also supply more info about the product priced This can result in revising the product specification