1. 1 Local and Global Scores in Selective Editing Dan Hedlin Statistics Sweden

2. 2 Local score Common local (item) score for item j in record k: w k design weight predicted value z kj reported value  j standardisation measure

3. 3 Global score What function of the local scores to form a global (unit) score? The same number of items in all records p items, j = 1, 2, … p Let a local score be denoted by  kj … and a global score by

4. 4 Common global score functions In the editing literature: Sum function: Euclidean score: Max function:

5. 5 Farwell (2004): ”Not only does the Euclidean score perform well with a large number of key items, it appears to perform at least as well as the maximum score for small numbers of items.”

6. 6 Unified by… Minkowski’s distance Sum function if  = 1 Euclidean  = 2 Maximum function if   infinity

7. 7 NB extreme choices are sum and max Infinite number of choices in between = 20 will suffice for maximum unless local scores in the same record are of similar size

8. 8 Global score as a distance The axioms of a distance are sensible properties such as being non-negative Also, the triangle inequality Can show that a global score function that does not satisfy the triangle inequality yields inconsistencies

9. 9 Hence a global score function should be a distance Minkowski’s distance appears to be adequate for practical purposes Minkowski’s distance does not satisfy the triangle inequality if < 1 Hence it is not a distance for < 1

10. 10 Parametrised by Advantages: unified global score simplifies presentation and software implementation Also gives structure: orders the feasible choices …from smallest: = 1 …to largest: infinity

11. 11 Turning to geometry…

12. 12 Sum function = City block distance p = 3, ie three items

13. 13 Euclidean distance

14. 14 Supremum (maximum, Chebyshev’s) distance

15. 15 Imagine questionnaires with three items Record k Euclidean distance

16. 16

17. 17 The Euclidean function, two items A sphere in 3D Threshold 

18. 18 The max function A cube in 3D Same threshold 

19. 19 The sum function An octahedron in 3D

20. 20

21. 21 The sum function will always give more to edit than any other choice, with the same threshold

22. 22 Three editing situations 1.Large errors remain in data, such as unit errors 2.No large errors, but may be bias due to many small errors in the same direction 3.Little bias, but may be many errors

23. 23 Can show that if… 1.Situation 3 2.Variance of error is 3.Local score is Then the Euclidean global score will minimise the sum of the variances of the remaining error in estimates of the total

24. 24 Summary Minkowski’s distance unifies many reasonable global score functions Scaled by one parameter The sum and the maximum functions are the two extreme choices The Euclidean unit score function is a good choice under certain conditions