1. Sources of Measurement Errors in Business Surveys
Mojca Bavdaz

2. Outline Measurement errors & their sources. Empirical study.
Sources of measurement errors redefined.
Implications.

3. Measurement errors ‘Theoretical’ definition of a measurement error:
‘deviation’ of the observed survey value from the true value (Groves, 1991);
‘difference’ or ‘discrepancy’ between the observed survey value and the true value (Hansen et al., 1951; Sukhatme and Sukhatme, 1970).
‘Practical’ definition of a measurement error:
observational gap between the ideal measurement and the response obtained (Groves et al., 2004);
error which occurs at the time of data collection (Biemer and Lyberg, 2003).

4. Sources of measurement errors (1)
In household surveys (e.g. Groves, 1989):
respondent;
interviewer;
instrument (the survey questionnaire);
mode of data collection.

5. Sources of measurement errors (2)
In business surveys, the addition of the fifth source:
records (Ponikowski, Meily, 1989);
information system (Biemer, Fecso, 1995; Biemer, Lyberg, 2003);
organization (O'Brien, 2000).

6. Empirical study Business survey under study: Aim: Research methods:
Quarterly Business Survey on Trade (SORS).
Aim:
examination of actual response process.
Research methods:
in-depth interview;
observation.
Sample size (n=28).
Challenges of implementation.

7. Three-dimensional model of business survey response

8. Sources of measurement error redefined
Existing sources
Identified sources
Respondent
→ +
Business participants
Interviewers
Survey staff
Survey questionnaire (Survey instrument) →
Survey instrument
Mode of data collection
Survey characteristics
Records/Information system/Organization
Organizational context
INTERACTIONS!

9. Implications Improved tools:
Three-dimensional model for comprehensive & systematic analysis of response process in business surveys.
Redefined sources of measurement errors for development of approaches to measurement error prevention & reduction.

10. Implications: Example 1
Self-administered business survey.
No interviewer.
But really no survey staff involved?
One-time & rare contacts make huge impact.
Training of (all) survey staff.

11. Implications: Example 2
Recurring business survey.
No source of measurement error.
But really no impact of recurrence?
Learning curve effect & routine.
Support & verification of first-time questionnaire completion (at business & respondent level).

12. Future research More research on specific aspects:
comprehension of economic concepts;
role of business participants & survey staff;
circulation of survey instrument;
impact of ‘soft’ issues;
...
Quantification through experiments.
Generalization to non-business organizational surveys.

13. Sources of Measurement Errors in Business Surveys
Mojca Bavdaz