1. Sample surveys versus business register evaluations:
Session 2: Survey area and business register interaction
Sample surveys versus business register evaluations:
A challenge for consistency
Joachim Weisbrod, Statistisches Bundesamt (Destatis)
Workshop „Survey Frames and Feedback“, Dublin, 24th – 25th April 2012

2. The consistency issue (1)
Why does consistency matter?
Consistency is a quality feature in itself (CoP)
Minimizing conflicting information enhances the confidence in official statistics
alleviates the explanation of statistical results
saves resources and lowers burden
The aim is to draw a coherent and consistent picture of the economy

3. The consistency issue (2)
Trends in business statistics : Integrated data pool with multiple possibilities of evaluations (data warehouse approach)
Combination of data from different sources (mix-models)
Certain results in business statistics are achievable at reasonable costs by register evaluations only (e.g. enterprise demography)
Evaluations of existing data (e.g. the Business Register (BR)) reduce the burden on enterprises considerably
Certain degree of consistency is conditional for this aim

4. Register based system of business statistics
Key element : System of business statistics in which the business register (BR) plays a central role in all production phases of statistics
Tasks of the BR
Determination of the target population
Monitoring tool for the different data collection methods
Coordination of samples
Frame for raising the sample(s)
Direct source of statistical information (e.g. business demography)
Information from the BR and information from surveys should be consistent

5. Consistency between surveys and BR
Register evaluations and survey results for the same target population should be more or less the same for the same characteristics
In practice there are differences between both because of different reasons

6. Consistency between BR and structural surveys (1)
Comparison between BR and structural surveys (2007)
Business register
Structural surveys
enterprises (n)
employment (n)
turnover (in 1000 €)

7. Consistency between BR and structural surveys (2)
Ratios of enterprises, employment, turnover (2007)
Business register
Structural surveys
enterprises (n)
employment (n)
turnover (in 1000 €)

8. Ideal requirements for the BR
BR is “up to date”
BR contains the “truth” in terms of
Completeness
units
NACE code
turnover
employment
regional information
for each reference period
Unfortunately, this is not always the case

9. BR is based on administrative sources
Situation in Germany
BR is based on administrative sources
Use in the BR is not a primary target for admin data
NACE-Classification not always reliable
Up-to-dateness
Regarding classification method itself
Statistical units not always clear
Problems of timing
Only limited influence on the time schedule and the workflow of administrative institutions
Plausibility checks of administrative data is very time consuming

10. Reasons for „inconsistencies“
Incorrect information in the BR
Sources of the BR have quality deficits
BR was not „up to date“ when drawing the sample
Changes in reality in the laps of time
Information from the survey itself is not consistent with the information in the BR
We assume survey information always to be more reliable than information from the BR

11. Problems of inconsistencies
The the feedback process between survey and BR
Exchange of relevant information between different statistical domains and the BR in a large decentralised system is a complex process
Instantaneous use of the information has to be ensured
This is more difficult in a decentralised system of official statistics with limited resources (like in Germany)
Domain statistics (with separate legal foundations) re-enforce the problem
Sample surveys have additional problems
A unit of the sample represents others in the same strata
Unit non-responses effect the expansion of results
From surveys we receive primarily information about units lost

12. Challenges: sample survey data vs. BR data
Example
Feedback to the BR
Register ID
NACE
Turnover :
101 C… 40
102
103 55
104 45
105 60
106 50
107
148
149
150
151 D…
100
Register updated ID
NACE
Turnover :
101 C… 40
102
103 55
104 45
105
out of business
106
107 50
148 J… 60
149
150
151 D…
100
10% Sample ID
NACE
Turnover
102 C… 40
105 60
106 50
148
150
Survey result ID
NACE
Turnover
102 C… 40
105
out of business
106 55
148 J…
150 60
Expansion results for section C
units:
turnover: 1550 ?
Section C totals
units:
turnover: 2500
Section C totals
units:
turnover: 2385

13. Challenges: sample survey data vs. BR data
We cannot avoid inconsistencies completely but we can minimize the inconsistencies under given conditions
by improving the quality of the BR itself (see example below)
classification or re-classification of the statistical units in the BR
reducing differences between the state of the BR and the reference period
by improving the communication between the register and the different surveys (organisation of the feedback process)
better coordination of the surveys themselves ,
efficient and instantaneous use of information coming from the surveys

14. Workflow on the time axis (1)
Illustration: Survey 2007 and register data
Send questionnaire
Draw population
expansion
Draw sample
Draw population
2006
2007
2008
2009
Permanent updates of adresses, survey information
Register data (turnover, employment)
2006 provisional
2006

15. How to improve the situation
speed up processing in the BR (limited possibilities)
to check the correct classification of units in certain time intervalls
thorough analyses of unit non-responses in all surveys
wrong adresses
Wrong NACE-codes
Wrong size classes
improved feedback from surveys to BR
Instantaneous use of the survey information in the BR
Use the latest information for the expansion (problems with quality measures)
Not treated as exits

16. Workflow on the time axis (2)
Illustration: Survey 2009 and register data
Send questionnaire
expansion
Draw population
Draw sample t
2008
2009
2010
2011
Permanent updates of adresses, survey information
register data 2008 (turnover, empl)
2009 provisional
2009

17. Conclusions
Measures envisaged to reduce the formation of inconsistencies.
Since the quality of the BR is the basis for high quality business statistics we attempt to get the necessary support for (resources)
rolling reviews in the BR,
careful analyses of non-responses in the surveys,
more effective organisation of the communication and feedback between surveys and BR and
improvement of exchange of information between the different domains of business statistics.

18. Measures to improve consistency by adjustment procedures
Conclusions (cont.)
Measures to improve consistency by adjustment procedures
Because perfect consistency between surveys and BR is not achievable in practice we were looking for transparent adjustment procedures which can at least improve consistency. In that context we analysed
Calibration Methods
(Adjustment of important results of surveys to the most recent values of the BR)
and
Imputation Methods
(Addition of information for start-ups and units changing the domain of economic activity. Missing variables are imputed.)
Both methods also have disadvantages

19. Conclusions (cont.) Calibration Imputation Disadvantage:
difficult to adjust all variables; inconsistencies for non-adjusted variables remain;
the procedure produces systematically biased results;
systematic errors cannot be estimated from the survey;
for sample surveys the calculation of sampling errors is complicated.
Imputation
Disadvantages:
regarding sample surveys results for original units of the sample deviate at random from the results of BR evaluations;
inconsistencies because of still existing misclassifications in the BR remain;
units performing another economic activity can cause larger sample errors of results in the new activity;
imputation methods have to be developed and systematic errors could hardly be assessed.

20. We decided in favour of the imputation method:
Conclusions (cont.)
We decided in favour of the imputation method:
With regard to the uncertainties of the correct classifications of units in the BR we prefer the adjustment method which gives priority to the information from the surveys.
New entries are taken into account additively. This is easier to handle.
Inconsistencies between NACE structures of the surveys and the BR could only be avoided by regular checks of the economic activities of all units in the BR.
Since this cannot be reaalised in the short run we are looking for support for rolling reviews of classification checks.

21. Thank you for your attention
Joachim Weisbrod
Statistisches Bundesamt
Gustav-Stresemann-Ring 11, Wiesbaden, Germany
Phone: +49 (0)611 /752234; Fax: +49 (0)611/753953