1 1 Reflections about the Impact Business Questionnaires have on Perceived Response Burden and Survey Quality Gustav Haraldsen Statistics Norway

2 “What we have found most striking when reviewing these articles, is the apparent lack of NSI efforts in actually measuring response burden before and after the redesign processes” Dag F. Gravem, Tora Löfgren and Yngve Bergström Statistic Norway An initiative QDET methods (Response burden indicators) Response rate or response qualitity change?

3 NSI’s reasons for reducing questionnaire communication burdens (Re)Design Lower Response burden Higher quality Lower cost Nonresponse bias Response rate Time Validity Reliablility Question elements Design elements Communication mode Cognitive burden Time burden

4 Comprehend Retrieve Judge Report Encode Select Prioritize Release Motivate Figure 2: The business survey respondent’s environment Wording Task Response format Validity Reliability Questionnaire design Communication mode Time burden No of questions LowHigh No of minutesLowReasonable low Effective HighIneffectiveReasonable high

5 Some review results Changing from paper to web may rather increase than reduce the response burden and the nonresponse. Web functionality may make some response tasks easier, but still have little effect on the perceived response burden. It’s an open question if changing from paper to web reduces the time burden of filling in the questionnaire. Design improvements may lead to higher time and response burden, but also to higher quality. Time burden affects perceived response burden and the response rate, probably also response quality

6 Thank you for your attention this far, ….. there is more to come How to proceed Response burden + Quality QDET Response burden + Quality Be more specific about design elements, burdens and quality improvements Run a three steps design

7 Perceived Response Burden 9,6 29,3 39,9 15,5 5,8 Very easy Quite easy Neither easy nor difficult Quite difficult Very difficult Sources of Response Burden E-technical E-Usability Other Layout No of questions Wait for information Terms Calculations Need of help Mismatch Response categories Perceived Response Burden and Sources of Response Burden. Structural Statistics Percent (n = )

8 Structural statistics, manufacturing industry. Response burden and completion time 2004 and 2006 PRB -indexTime Paper 2004Web 2006Paper 2004Web 2006 One establishment business = = 126 Multi establishment business = = 288