© Federal Statistical Office Germany, Division IB, Institute for Research and Development in Federal Statistics Sheet 1 Surveys, administrative data or integrated models: A decision by quality indicators? Jörg Enderer, Dieter Schäfer European Conference on Quality in Official Statistics Q2010, Helsinki, 4-6 May 2010

© Federal Statistical Office Germany, Division IB, Institute for Research and Development in Federal Statistics Sheet 2 The project: Use of administrative data in short-term statistics Turnover tax data from fiscal authorities Employment data from Federal Employment Agency monthly data: short term statistics other administrative data annually data: business register data from statistical surveys

© Federal Statistical Office Germany, Division IB, Institute for Research and Development in Federal Statistics Sheet 3 Reshaping STS - Results 2004 SURVEYSSURVEYS Full replacement of surveys: Craft statistics (German particularity) Integrated model of smaller survey and administrative data: Service sector, wholesale trade, maintenance of motorvehicles Surveys unchanged, additional use for non-covered enterprises: Building installation Surveys unchanged: Manufacturing, site preparation and civil engineering, retail trade, hotels and restaurants 2011

© Federal Statistical Office Germany, Division IB, Institute for Research and Development in Federal Statistics Sheet 4 Favorable testing conditions Availability of aggregated results and micro data for surveys and administrative data in terms of periods economic activities regions  Extensive possibilities to compare the different methods for the same field survey administrative data integrated models of survey and administrative data

© Federal Statistical Office Germany, Division IB, Institute for Research and Development in Federal Statistics Sheet 5 Questions (ex post) Which quality criteria were relevant for the choice of the method? Could the criteria be described by quality indicators? By indicators included in ESQR? Did the indicators lead to a clear decision for a certain method?

© Federal Statistical Office Germany, Division IB, Institute for Research and Development in Federal Statistics Sheet 6 Relevant quality criteria  Relevance  Accuracy  Timeliness  Punctuality  Comparability  Accessibility  Clarity  (Coherence) Important criteria for the tests Criteria independent from choice of method

© Federal Statistical Office Germany, Division IB, Institute for Research and Development in Federal Statistics Sheet 7 Quality criterion: Relevance Target: User needs have to be fulfilled Surveys: Do we get what we want? Check of microdata => Accuracy issue Administrative data: Differences in definitions, statistical units, allocation to economic activities Quantitative indicators: Differences in concepts in % No uniform results for different activities and regions Elimination of important differences by (complex) estimation models => Problems of accuracy?  Role of indicators: Information on big deviations of administrative data; no quantitative indicators for smaller deviations in definitions

© Federal Statistical Office Germany, Division IB, Institute for Research and Development in Federal Statistics Sheet 8 Quality criteria: Timeliness and punctuality Target: Comply with existing standards of timeliness Indicators: Time lag in days; additional indicators for subprocesses How does the fulfillment of the target affect the accuracy? Administrative data: Suitable and automatic IT procedures had to be developed Higher time restrictions for editing and estimation methods  Administrative Data: In some cases target not achievable with reasonable accuracy, e.g. t+30 for manufacturing, retail trade  Role of Indicators: Minimum Standards

© Federal Statistical Office Germany, Division IB, Institute for Research and Development in Federal Statistics Sheet 9 Quality criterion: Accuracy Targets: Assure accuracy of the status quo Different dimensions of accuracy: Internal comparision: Revisions for each method Comparison of results of different methods Accuracy of estimations for administrative data ESQR - Indicators: Average size of revisions, response rates, edit failure rates, coefficient of variation, coverage rates Additional indicators for administrative data like misclassification rates, linkage rates, risk indicators for differences in statistical units (tax groups)

© Federal Statistical Office Germany, Division IB, Institute for Research and Development in Federal Statistics Sheet 10 Accuracy: Limits of indicators Size of revisions Surveys: Correction of missing data and outliers Administrative data: Correction of missing data, outliers and update of data Higher size of revisions does not necessarily mean poorer quality Does higher edit failure rate mean less accuracy? Different indicators are difficult to weight against each other: Coefficient of variation against misclassification?  Role of indicators: Vital but not without sound interpretation

© Federal Statistical Office Germany, Division IB, Institute for Research and Development in Federal Statistics Sheet 11 Decision-making by indicators? Minimum standards for single quality criteria are important Minimum standards for relevance and timeliness can be reached at the expense of accuracy Accuracy is very complex and difficult to judge for one method (esp. non sampling errors) Comparison of accuracy for different methods even more difficult ESQR – Indicators are very important, but administrative data need partly other (additional) indicators Quantitative indicators need background information for sound interpretation Composite/overall quality indicators are no way out

© Federal Statistical Office Germany, Division IB, Institute for Research and Development in Federal Statistics Sheet 12 Decision-making process in the project 1.The decision was a social process managed by written summaries, discussions between producers and communication with main users 2.Standards on relevance, timeliness and punctuality must be met 3.Minimum standards for accuracy must be ensured, e.g. acceptable size of revisions estimation rates coefficients of variation 4.In addition pros and cons have to be balanced.

© Federal Statistical Office Germany, Division IB, Institute for Research and Development in Federal Statistics Sheet 13 Dieter Schäfer and Jörg Enderer Many thanks for your attention Any questions?