Sample surveys versus business register evaluations: Session 2: Survey area and business register interaction Sample surveys versus business register evaluations: A challenge for consistency Joachim Weisbrod, Statistisches Bundesamt (Destatis) Workshop „Survey Frames and Feedback“, Dublin, 24th – 25th April 2012

The consistency issue (1) Why does consistency matter? Consistency is a quality feature in itself (CoP) Minimizing conflicting information enhances the confidence in official statistics alleviates the explanation of statistical results saves resources and lowers burden The aim is to draw a coherent and consistent picture of the economy

The consistency issue (2) Trends in business statistics : Integrated data pool with multiple possibilities of evaluations (data warehouse approach) Combination of data from different sources (mix-models) Certain results in business statistics are achievable at reasonable costs by register evaluations only (e.g. enterprise demography) Evaluations of existing data (e.g. the Business Register (BR)) reduce the burden on enterprises considerably Certain degree of consistency is conditional for this aim

Register based system of business statistics Key element : System of business statistics in which the business register (BR) plays a central role in all production phases of statistics Tasks of the BR Determination of the target population Monitoring tool for the different data collection methods Coordination of samples Frame for raising the sample(s) Direct source of statistical information (e.g. business demography) Information from the BR and information from surveys should be consistent

Consistency between surveys and BR Register evaluations and survey results for the same target population should be more or less the same for the same characteristics In practice there are differences between both because of different reasons

Consistency between BR and structural surveys (1) Comparison between BR and structural surveys (2007) Business register Structural surveys enterprises (n) employment (n) turnover (in 1000 €)

Consistency between BR and structural surveys (2) Ratios of enterprises, employment, turnover (2007) Business register Structural surveys enterprises (n) employment (n) turnover (in 1000 €)

Ideal requirements for the BR BR is “up to date” BR contains the “truth” in terms of Completeness units NACE code turnover employment regional information for each reference period Unfortunately, this is not always the case

BR is based on administrative sources Situation in Germany BR is based on administrative sources Use in the BR is not a primary target for admin data NACE-Classification not always reliable Up-to-dateness Regarding classification method itself Statistical units not always clear Problems of timing Only limited influence on the time schedule and the workflow of administrative institutions Plausibility checks of administrative data is very time consuming

Reasons for „inconsistencies“ Incorrect information in the BR Sources of the BR have quality deficits BR was not „up to date“ when drawing the sample Changes in reality in the laps of time Information from the survey itself is not consistent with the information in the BR We assume survey information always to be more reliable than information from the BR

Problems of inconsistencies The the feedback process between survey and BR Exchange of relevant information between different statistical domains and the BR in a large decentralised system is a complex process Instantaneous use of the information has to be ensured This is more difficult in a decentralised system of official statistics with limited resources (like in Germany) Domain statistics (with separate legal foundations) re-enforce the problem Sample surveys have additional problems A unit of the sample represents others in the same strata Unit non-responses effect the expansion of results From surveys we receive primarily information about units lost

Challenges: sample survey data vs. BR data Example Feedback to the BR Register ID NACE Turnover : 101 C… 40 102 103 55 104 45 105 60 106 50 107 148 149 150 151 D… 100 Register updated ID NACE Turnover : 101 C… 40 102 103 55 104 45 105 out of business 106 107 50 148 J… 60 149 150 151 D… 100 10% Sample ID NACE Turnover 102 C… 40 105 60 106 50 148 150 Survey result ID NACE Turnover 102 C… 40 105 out of business 106 55 148 J… 150 60 Expansion results for section C units: 30 turnover: 1550 ? Section C totals units: 50 turnover: 2500 Section C totals units: 48 turnover: 2385

Challenges: sample survey data vs. BR data We cannot avoid inconsistencies completely but we can minimize the inconsistencies under given conditions by improving the quality of the BR itself (see example below) classification or re-classification of the statistical units in the BR reducing differences between the state of the BR and the reference period by improving the communication between the register and the different surveys (organisation of the feedback process) better coordination of the surveys themselves , efficient and instantaneous use of information coming from the surveys

Workflow on the time axis (1) Illustration: Survey 2007 and register data Send questionnaire Draw population expansion Draw sample Draw population 2006 2007 2008 2009 Permanent updates of adresses, survey information Register data 2005 (turnover, employment) 2006 provisional 2006

How to improve the situation speed up processing in the BR (limited possibilities) to check the correct classification of units in certain time intervalls thorough analyses of unit non-responses in all surveys wrong adresses Wrong NACE-codes Wrong size classes improved feedback from surveys to BR Instantaneous use of the survey information in the BR Use the latest information for the expansion (problems with quality measures) Not treated as exits

Workflow on the time axis (2) Illustration: Survey 2009 and register data Send questionnaire expansion Draw population Draw sample t 2008 2009 2010 2011 Permanent updates of adresses, survey information register data 2008 (turnover, empl) 2009 provisional 2009

Conclusions Measures envisaged to reduce the formation of inconsistencies. Since the quality of the BR is the basis for high quality business statistics we attempt to get the necessary support for (resources) rolling reviews in the BR, careful analyses of non-responses in the surveys, more effective organisation of the communication and feedback between surveys and BR and improvement of exchange of information between the different domains of business statistics.

Measures to improve consistency by adjustment procedures Conclusions (cont.) Measures to improve consistency by adjustment procedures Because perfect consistency between surveys and BR is not achievable in practice we were looking for transparent adjustment procedures which can at least improve consistency. In that context we analysed Calibration Methods (Adjustment of important results of surveys to the most recent values of the BR) and Imputation Methods (Addition of information for start-ups and units changing the domain of economic activity. Missing variables are imputed.) Both methods also have disadvantages

Conclusions (cont.) Calibration Imputation Disadvantage: difficult to adjust all variables; inconsistencies for non-adjusted variables remain; the procedure produces systematically biased results; systematic errors cannot be estimated from the survey; for sample surveys the calculation of sampling errors is complicated. Imputation Disadvantages: regarding sample surveys results for original units of the sample deviate at random from the results of BR evaluations; inconsistencies because of still existing misclassifications in the BR remain; units performing another economic activity can cause larger sample errors of results in the new activity; imputation methods have to be developed and systematic errors could hardly be assessed.

We decided in favour of the imputation method: Conclusions (cont.) We decided in favour of the imputation method: With regard to the uncertainties of the correct classifications of units in the BR we prefer the adjustment method which gives priority to the information from the surveys. New entries are taken into account additively. This is easier to handle. Inconsistencies between NACE structures of the surveys and the BR could only be avoided by regular checks of the economic activities of all units in the BR. Since this cannot be reaalised in the short run we are looking for support for rolling reviews of classification checks.

Thank you for your attention Joachim Weisbrod Statistisches Bundesamt Gustav-Stresemann-Ring 11, 65189 Wiesbaden, Germany Phone: +49 (0)611 /752234; Fax: +49 (0)611/753953 E-Mail: joachim.weisbrod@destatis.de www.destatis.de