1. S T A T I S T I K A U S T R I A 14.4.2010 1 www.statistik.at Quality Assessment of register-based Statistics A Quality Framework Manuela LENK Directorate Population Statistics Register based census © STATISTIK AUSTRIA I n f o r m a t i o n e n Wir bewegen

2. S T A T I S T I K A U S T R I A 14.4.2010 2  Switch from traditional to register-based census in Austria 2011  Compared to other countries transition time is relatively short  Census Test was carried out for 2006  Seven base registers and several comparision registers  Registers are combined using unique identifiers (bPK)  Gathering experience for the register-based census Introduction 2

3. S T A T I S T I K A U S T R I A 14.4.2010 3  Quality is influenced by (according to the quality report as requested by Eurostat)  Quality of the administrative data source  Existence of an unique key or rather the availability to link a person  Comparision between survey and administrative data  Item imputation  Consequence  Establishing a quality framework for the register based census 2011 – cooperation with WU Vienna Quality Issues in Census Test 3

4. S T A T I S T I K A U S T R I A 14.4.2010 4  Quality assessment of statistics based on administrative data  Quality Indicator for  Raw Data – data obtained from administrative sources  Each attribute in each register  Whole register (data source)  Census 2011  Attributes of the Final Data Pool Objectives 4

5. S T A T I S T I K A U S T R I A 14.4.2010 5  Assessment of quality is derived from hyperdimensions (HD) for each attribute in each register.  Documentation  Pre-Processing  External Sources  Imputation Hyperdimensions 5

6. S T A T I S T I K A U S T R I A 14.4.2010 6  Documentation  Focus is on processes taking place before the data is transfered to Statistics Austria  Data treatment at the source maintaining the register  Reliability of the data source  Quality criteria  Relevance of attributes for the source (e.g. legal foundation)  Availability for certain reference dates  Compatible definitons  Registers are benchmarked using a perfect pseudo register  Informations are extracted from the meta database Hyperdimensions (1) 6

7. S T A T I S T I K A U S T R I A 14.4.2010 7  Pre-Processing  Process of Data editing from Raw Data to the Edited Data  Analysis of Raw Data  Error corrections  Recoding  Plausibility checks  Quality criteria  Proportion of missing unique keys  Proportion of values out of domain  Item non-response  Automated generation out of database Hyperdimensions (2) 7

8. S T A T I S T I K A U S T R I A 14.4.2010 8  External Source  Checks accuracy of the data  record linkage with survey data (e.g. Labour Force Survey)  Quality criteria  Consistency of attribute values  For some attributes no external source for comparision exists, thus it is being replaced by  Expert interviews  Rating of accuracy – expert knowledge obtained from working experience  Subjective – should be considered when allocating the weight for this hyperdimension Hyperdimensions (3) 8

9. S T A T I S T I K A U S T R I A  Each hyperdimension receives a weight in accordance with its relevance for the project.  Within these hyperdimensions quality criteria are determined and quality indicators are derived for each hyperdimension (HD D, HD P, HD E ).  After weighting these dimensions (v D, v P, v E ) we get an aggregated quality indicator q. 14.4.2010 9 Quality Indicators (1) 9

10. S T A T I S T I K A U S T R I A 14.4.2010 10  Finally we are able to derive a quality indicator q ij for each attribute in each register  We could derive an aggregated quality indicator for each register too (weighted row sum). Quality Indicators (2) 10

11. S T A T I S T I K A U S T R I A 14.4.2010 11  Register-based census  Quality assessment within the Census Database  Combination of existing registers to a new data base.  Quality assessment of the Final Data Pool  Census Database including item imputations. Application on linked data sets 11

12. S T A T I S T I K A U S T R I A 14.4.2010 12  Unique Attributes  attribute exists exactly once in one register and is transfered directly to the CD  q ij = q  j = q Ψj linklink  Multiple Attributes  attribute exists in more than one register. Information from different sources are combined in the CD using certain decision rules – e.g. majority principle (Sex)  Quality indicators for this attribute from different registers are combined to an overall quality indicator q  j (e.g Dempster-Shafer Theory)  Additional check using external data source HD E leads to a corrected quality indicator q Ψ j linklink  Derived Attributes  several attributes are linked to a new attribute  Quality indicator q  j is generated by using the quality indicators of the input attributes  Additional check using external data source HD E leads to a corrected quality indicator q Ψ j linklink Assessment of the Census Database (CD) 12

13. S T A T I S T I K A U S T R I A 14.4.2010 13  Census Database including item imputations  Hyperdimension Imputation (HD I )  Quality indicators depend on methods of imputation  Shall be generated with appropriate valuation methods  The weight for this hyperdimension is approximated by the proportion of imputation Quality assessment of the Final Data Pool 13

14. S T A T I S T I K A U S T R I A Process-oriented Quality Framework back back 14

15. S T A T I S T I K A U S T R I A Conclusion  Framework should act as a road map  Next milestones are  Establishing check lists (questionnaires)  Specify how to derive the quality indicators for each hyperdimension and how to aggregate them  General approach  Application to other projects is possible 15

16. S T A T I S T I K A U S T R I A 14.4.2010 16  Additional Information  Quality Framework  Fiedler/Schwerer/Berka/Humer/Moser: Quality Assessment for Register based Statistics in Austria  Register-based Census  www.statistik.at www.statistik.at Outlook 16