Assessing validation effectiveness – results from a recent MEETS project and future plans Anette Hertz, Head of section, Statistics Denmark

 Introduction  Method to calculate efficiency  Problems encountered in the project  Results  Example from the project  Future plans  Comments Agenda 2

 The motivation:  We experience diminishing resources, however we try to keep the same quality level. Hence we must be more efficient in our validation of the Intrastat declerations.  How effective are the individual validation procedures?  Is it possible to optimize the validation of the Intrastat declarations by reorganizing the order of the validation procedures? Introdoction 3

 The plan is to compare initially reported trade to the final trade figures.  How big is the change in: -Value -Quantity -Supplementary unit  How many changes of the type: -Change in date -Flow -Country -Transaction code -Commodity code -PSI Method to calculate efficiency 4

 Problems retriving before and after data on some of the separate validation procedures  For some validation procedures it was possible to do this thanks to our status code system  Not possible to get an accurate resource estimate  We asked the clerical workers for their best estimate  Still looking for a more accurate way to estimate this Problems encountered in the project 5

Time usedMan years Effect on value in million DKK VAT against Intrastat80% of 21,6+884, -127 Great differences in weight and supp. Unit3% of 10,03-38 PSI check15% of 10,15 Country check1,5% of 10,015 Doubles25% of 1, 5% of 20,35 Absolute error detection Probable error detection60% of 42,4+223, Check for large lines10% of 20, Earmarkings10% of 30,3 Transaction code check20% of 10,2 Calculated weights5% of 10,05 Total 5,3+1107, Results 6

 The standard unit values are at the base of recontacting PSIs for validation of probable errors  Since we only send out letters to PSIs for validation in around 1% of the 36 mio records we receive each year it is important that it is the right 1% we send out  We found out that the standard unit value check is not prioritized by the clerical workers  The result:  The accuracy of the probable error detection model has decreased because of this Example from the project 7

 Plan to develope a more sophisticated status code system  When possible to seperate the different validation processes, analyze if resources can be diminished by reorganizing the procedures  Plan to measure the efficiency of VAT against Intrastat validation by running the statistical production with and without this validation process  Find ways to help improve our standard unit values and make them more robust Future plans 8

 Do you have any experience on:  Measuring efficiency of validation procedures  Estimate the use of resources  Validate and preserve standard unit values Comments 9

Thank you for your attention Questions?