IMPLEMENTING UNIT VALUE INDICES IN THE ANNUAL OECD INTERNATIONAL TRADE IN COMMODITY STATISTICS (ITCS) DATABASE Handling missing values, outliers in unit value variations, representativity of the indices, data conversion across classifications Working Party on Trade in Goods and Services 7-9 November 2011 OECD Statistics Directorate

Outline of presentation Background Data universe and formulae Supplementary units or net weights? Conversion issues Estimation of missing values Dealing with outliers Specific products

Background 1/2 Advantages of UVIs and Quantities Indices: proxies for X & M price indices and for volume measures available at product detail level and in a timely manner international comparability analysis of : –terms of trade – price and non-price competitiveness in X &M –quantity effect from the transmission of inflation via foreign trade Context follow-up of the WPTGS 2010 recommandation of the IMTS 2010 results of the short Survey WPTGS 2010

Background 2/2 Stocktaking of the implementation: – on tracks with PWB – process with SAS –chained Laspeyres Paasche Fisher indices – on 34 countries –HS88 at a total level / partner World Issues that are being looking at : –missing values (of quantities and trade values) –outliers –conversion across HS classifications

Data Universe 1/3 ITCS database : –UNSD / OECD joint data processing system –Contains Information on –Trade values in US dollars –Quantities (liters, m²…) –Net Weight (kg) –Data Availability –HS (1988,1996,2002 & 2007) – SITC (rev.2, rev.3) – ISIC rev.3

Data Universe 2/3 CIF / FOB Valuation Calculation based on 6 digits of HS1988 Estimation of missing quantity/weight by the UNSD method : 1) use of the available info (i.e. net weight proxied from quantity) 2) use of a Standard Unit Value Exclusion of 6 digits commodities without information on weights (whole chapter 99 )

Data Universe 3/3 UV = Value in US dollars Quantity Unit Values Indices are sensitive to exchange rates fluctuations; not Quantity Indices … (Pt)

Computation of Quantity indices LASPEYRES PAASCHE FISHER Quantity ratio form Weighing system

Computation of Unit Value indices LASPEYRES PAASCHE FISHER Quantity ratio form Weighing system

Defining the denominator of the UV Supplementary Units or Net Weights ?

Should we use supplementary units? Supplementary units more accurate than Net weights for some commodities BUT How to handle changes of quantity units if for instance the quantity unit is one year, number of items (5) and the next year thousands of items (9) On long series, net weights are more reported UNSD estimated that 75% of supplementary units is KG => OECD choice : net weight in kg

Representativity of the sample: Issue with historical series 2009 : –<50% Israel X (80% for Israel M) –>75% for All others OECD countries (M and X) 1999 : more problematic – <25% Canada(M and X), Australia M and USA M –<60% + New Zealand (M), USA (X), Norway (X), Australia (X) –<75% + Japan (X), Norway (M), Mexico (M), New Zealand (X) and Netherlands (M & X) 1989 / 21 countries –<75% Canada (M & X), USA (M&X), New Zealand (M) and Australia (M & X), Japan (X), Norway (M & X) => Thresholds on UVI for coverage ?

Conversion issues use of HS 1988 conversion of HS 1996, HS 2002 HS 2007

Conversion of HS

Conversion issues: looking at HS1988

Handling gaps in series Estimating Missing Values and Quantities: Unconvincing tests of 2 methods

Estimation of Trade Values cmd

Estimation of Quantities based on Standard Unit Values (SUV) SUV – Compiled by UNSD – median unit value of – each 6-digit commodity/year/flow –after elimination of outliers – of a sample of Unit Value of –available data of the latest reporting year – that respect a certain number of criteria (box 1)

Missing values estimation conclusions Thresholds for estimation for trade values: 1% of values estimated for the whole chapter Calculated indices present dubious movements : Large fluctuations for chained type Indexes at 6 digits level => Issues for more disaggregated indices (2&4 digits)

Outliers detection

Outlier detection an unrealistic price growth in the product specific distribution of unit value ratios CEPII indices : Unit values ratios are compared with the product specific median change in unit values computed over the whole period.

Outlier detection Value that lie far from the middle of the distribution in either direction Mexico and Italy : >100 obs : Asymmetric Fence Method <100 obs : Mean Absolute Deviation UNSD : Tukey Method

AFM and MAD Formulae Assymetric Fence Method Mean Absolute Deviation UV =logarithm of the unit value Q1, Q2; Q3 1 st 2 nd 3 rd quartiles of log unit values of trade distribution c, k, A parameters

Specific products chapters % of outliers within a chapter using symetric yearly variation detection method % of outliers within a chapter using AFM MAD detection method 84: Nuclear reactors, boilers, machinery, etc 15% 85: Electrical, electronic equipment 3%5% 29: Organic chemicals 1%2%

Comparing OECD total level indexes with those available from other international organisations and other frameworks (SNA)

Percentage Change of Quantity Indices for Iceland Imports

Percentage Change of Quantity Indices for Turkey Imports

Percentage Change of Unit Values Indices for Japan Import

Percentage Change of Unit Values Indices for Italy Export

Next Steps Following the Program of Work Begining of 2012 : 34 countries at a 2 digits level applying methodologies for outliers and missing values Finding some specific treatments for specific chapters (including those that lose 30 % of their trade just by changing classification (chapters ) Summer 2012 : Matrix of exports and imports unit value and quantity indices available online for comments at the 2012 WPTGS

Questions to delegates Thresholds : on customs transactions ? Measure of Quantity: choice of supplementary quantity or net weight values? HS Conversion issues : How to deal with cmd of HS change ? Estimation of Missing Values : What kind of methodology do you recommend? More disaggregated indices (2 digits indices or more detailed) do you have special warnings or experiences to share ?