Matea Paškvan Marc Plate Statistics Austria Contact: matea.paskvan@statistik.gv.at Usability Testing of Web Surveys Key issues in the design of web questionnaires

Background Need for fast and cost efficient data collection (Potaka, 2008) Declining response rates (de Leeuw, 2005) Computer Assisted Web Interviewing (CAWI) flexibility in terms of time an place additional cognitive burden

Study Aim After implementation of an in-house integrated system for data collection (STATsurv) Evaluation of the user-friendliness of STATsurv test & improve user-friendliness

Method Field study Different qualitative and quantitative methods Heterogeneous sample (N = 18) Own device Aviso – letter  login via internet browser Different qualitative and quantitative methods Online LFS-questionnaire (video taped) Standardized debriefing Cognitive interviews

Results – Finding the webpage and login A Person, who does not find the web page, is not able to participate! „googling“ the web site font size Every second person had problems with the login Optimize Aviso-letter & Search engine optimization Present a detailed login description Shorten login-name

To see all response options one has to scroll down. Results – Dropdown To see all response options one has to scroll down. No scrolling = biased responses Respondents take the best „fitting“ response option and rather do not scroll Respondents prefer radio buttons Chose Radio Buttons Be careful about the order of responses

Results – Lookup list Every second person was surprised that a list occurred Every fourth rather types the job title instead of choosing it Give brief and clear instructions!

Results – Household Grid Household grid is essential for the whole questionnaire! Sex was overlooked Date of birth is not user-friendly Not all persons did add a line for another house member Change date format Use radio buttons for sex-question Add lines automatically based on number of household members

Take Home „Think-aloud“ during questionnaire completion? Field testing Video tapes Even the „best“ CAWI-questionnaire will profit of an usability test.   

Test your questionnaire and share  Matea Paškvan Marc Plate Statistics Austria contact: matea.paskvan@statistik.gv.at