1. Q2008 - Rome Italy, July 20081 Towards Producing and Using Response Burden Data for Establishment Surveys at Statistics Sweden Johan Erikson, Dan Hedlin, Boris Lorenc Statistics Sweden

2. Q2008 - Rome Italy, July 20082 Background A Statistics Sweden project in 2007 to establish measures and procedures for monitoring and reporting response burden in establishment surveys (May-October 2007)‏ Coincided with publication of Eurostat’s Handbook for Monitoring and Evaluating Business Survey Response Burdens (August 2007)‏

3. Q2008 - Rome Italy, July 20083 Background A Statistics Sweden project in 2007 to establish measures and procedures for monitoring and reporting response burden in establishment surveys (May-October 2007)‏ Coincided with publication of Eurostat’s Handbook for Monitoring and Evaluating Business Survey Response Burdens (August 2007)‏

4. Q2008 - Rome Italy, July 20084 Background A Statistics Sweden project in 2007 to establish measures and procedures for monitoring and reporting response burden in establishment surveys (May-October 2007)‏ Coincided with publication of Eurostat’s Handbook for Monitoring and Evaluating Business Survey Response Burdens (August 2007)‏

5. Q2008 - Rome Italy, July 20085 Background A Statistics Sweden project in 2007 to establish measures and procedures for monitoring and reporting response burden in establishment surveys (May-October 2007)‏ Coincided with publication of Eurostat’s Handbook for Monitoring and Evaluating Business Survey Response Burdens (August 2007)‏ Regulations, SCM, earlier Stats Sweden work

6. Q2008 - Rome Italy, July 20086 Agenda Background Relation between Handbook and the results of the Statistics Sweden response burden (RB) project –The process of measuring response burden according to Handbook –Suggestions according to the Statistics Sweden RB project Towards the implementation –the mini-measurement and its implementation –glance into the future Conclusions

7. Q2008 - Rome Italy, July 20087

8. 8 The Handbook: Intro Basically discusses the following main topics: I.Monitoring the burden (Ch. 2)‏ 9 core questions (q’s)‏ II.Evaluating analytically the burden, relating it to causal factors (Ch. 3)‏ About 17 additional q’s III.Sampling and data collection procedures (Ch. 4 & Ch. 5)‏ IV.Reporting (Ch. 6)‏

9. Q2008 - Rome Italy, July 20089 The Handbook: Intro ‏ Basically discusses the following main topics: I.Monitoring the burden (Ch. 2)‏ 9 core questions (q’s)‏ II.Evaluating analytically the burden, relating it to causal factors (Ch. 3)‏ About 17 additional q’s III.Sampling and data collection procedures (Ch. 4 & Ch. 5)‏ IV.Reporting (Ch. 6)‏

10. Q2008 - Rome Italy, July 200810 The Handbook: Intro (cont’d) ‏ In discussing the topics I-II, identifies some key distinctions, amongst them: –actual vs. perceived response burden –survey burden vs. time spent on survey –response process steps: data collection vs. filling in the questionnaire (Q)‏

11. Q2008 - Rome Italy, July 200811 The Handbook: Actual/Perceived Distinction Actual burden = Perceived burden? No, that's why both actual & perceived needed! –“The aim of this handbook is to provide a tool for measuring perceived and actual response burden in business surveys that will help statistical organisations and other parties carry out their own response burden (PRB) surveys.” [p.1] –“This publication is a handbook written for statistical organisations and other institutions that carry out business surveys. Its main topic is how to monitor and evaluate perceived response burden (PRB) among business survey respondents.” [p.5]

12. Q2008 - Rome Italy, July 200812 The Handbook: Burden/Time Distinction Burden = Time? No, that's why both time & burden needed! –“...it is not the time use itself but the perception of time and effort that is likely to affect response quality. Time passes quickly if the topic is interesting and the respondent feels that he or she is competent to answer the questions posed.” [p.6] –“Actual burden is traditionally measured by time use. As we have mentioned earlier, we have chosen more detailed time measures than what has been traditionally used. We have designed two questions to measure time to collate information and one to measure time to fill in the questionnaire.” [p.11]

13. Q2008 - Rome Italy, July 200813 The Handbook: Steps of the Response Process Distinction The response process has these two steps (amongst others): –data collection (corresponds to step 5 of the Hybrid model of Willimack, Nichols & Sudman)‏ –questionnaire fill in (corresponds to step 7 of the Hybrid model)‏

14. Q2008 - Rome Italy, July 200814 The Handbook: I. Monitoring Burden (Ch.2) ‏ 4 dimensions, 8 indicators, 9 variables:

15. The Handbook: II. Analysing Causes of Burden (Ch.3) ‏ 4 dimensions, 6 indicators, 17 variables:

16. Q2008 - Rome Italy, July 200816

17. Q2008 - Rome Italy, July 200817

18. Q2008 - Rome Italy, July 200818

19. Q2008 - Rome Italy, July 200819 Issues Discussed in the RB-Project ‏ Given burden ≠ time, is Perceived length of time (q.1 of Monitoring) properly an indicator of Perceived burden (dim.1)? Is Self-reported time (q.3 of Monitoring) properly an indicator of Actual burden (dim.2)? –from the premise of actual burden ≠ perceived burden –from the measurement perspective: a retrospective question‏ Perceived time asked only regarding data collection (q.1), Perceived burden asked only regarding Q completion (q.2). Why? Why not the converse?

20. Q2008 - Rome Italy, July 200820 Issues Discussed in the RB-Project ‏ (cont’d) ‏ Is motivation to participate in surveys exhausted by tackling the utility of participation for the business and the society? –Cooperation principles (mentioned in the Handbook)‏ –Social exchange –Helpfulness, etc. Practicalities about the questionnaire: –Length –Content (e.g. Why causes in Monitoring [its dim.3]?)‏ –Question formulation (e.g. Time to collect information - business (q.4), does that include or exclude Time to collect information - self (q.3)?)‏ –Question order (e.g. the complex branching instructions pertaining to the first question)

21. Q2008 - Rome Italy, July 200821 Positions Taken in the RB-Project ‏ Measure respondent burden or measure respondent work? –Measure burden to begin with, but bring later on even positive aspects of survey participation into focus Monitor or evaluate? –Monitor on a broad front, evaluate in conjunction with changes or actions for improvement. (Same as in the Handbook.)‏ Actual or perceived burden? –Perceived, as well as an (assumed) correlate of actual time: the self-reported time (for oneself and for others)‏ it took to participate in the current survey

22. Q2008 - Rome Italy, July 200822 Positions Taken in the RB-Project ‏ (cont’d) ‏ Breakdown of burden in the evaluation Q: along questions or along steps of the response process? –Response process steps Motivation –More than utility: left out for the moment Explanatory variables –Number of people involved (q.3 of Evaluation)‏ –Assessment of data availability (-)‏ –Previous experience with the same survey (q.9 of Evaluation)‏ –Respondent’s role in the current data collection (q.4 of Eval.) –[Data regarding simultaneous surveys can be taken out of our own data base]

23. Q2008 - Rome Italy, July 200823 Basic Proposal of the RB-Project ‏ Perceived burden measured on three levels: –Monitoring on a broad level (level A; contains the mentioned explanatory variables)‏ –Breakdown of burden along the steps of the response process (level B); measured in conjunction with carrying out an improvement project‏ –Identification of probable causes (level C); measured in conjunction with carrying out an improvement project‏

24. Q2008 - Rome Italy, July 200824 RB-Project ‏ : Level A Variables –Perception regarding the current participation General level of burden –A correlate of actual burden self-reported time it took for own participation (retrospective) reported time it took for other’s participation (retrospective) –Explanatory variables ‏Number of people involved Assessment of data availability‏ Previous experience with the same survey Possibly, suggestions for improvement‏

25. Q2008 - Rome Italy, July 200825 RB-Project ‏ : Level B Variables –Perceptions regarding the current participation Burden to understand the request for data Burden to distribute and coordinate the data collection Burden to collect data Burden to process data Burden to provide data in the required form –Explanatory variables Respondent’s role in the current data collection

26. Q2008 - Rome Italy, July 200826 RB-Project ‏ : Level C Variables –Perceptions regarding the current participation Causes of the burden to understand the request for data Causes of the burden to distribute and coordinate the data collection Causes of the burden to collect data Causes of the burden to process data Causes of the burden to provide data in the required form

27. Q2008 - Rome Italy, July 200827 RB-Project ‏ : Conclusion Level A measured systematically and continuously in all enterprise surveys –But, not each occurrence of an recurring survey: a sampling procedure needed (not treated here) Levels B/C used in conjunction with evaluation or improvement of the data collection procedures –Provide input for simplification, questionnaire evaluation work, etc. –Possibly followed by qualitative methods on an even smaller scale

28. Q2008 - Rome Italy, July 200828 The Mini RB-measurement Implemented gradually since a year ago Measures self-reported time Consists of a single question, posed at the end of the survey –Time within the business it took to participate in the current survey Routines to process, present and use these data are currently implemented –sampling –estimation –useful summary tables –input to other processes

29. Q2008 - Rome Italy, July 200829 Conclusion Do measure response burden But, only if the collected data are used for the benefit of the data provider Otherwise, no real justification to do this kind of survey

30. Q2008 - Rome Italy, July 200830 Thank You Johan Erikson Dan Hedlin Boris Lorenc firstname.lastname@scb.se