1. Overview: Online Surveys Vasja Vehovar University of Ljubljana, Slovenia http://WebSM.org Handbook of Online Research Methods Colloquium 28-29 March 2007

2. Computer-assisted survey information collection Key methodological issues Related issues Applications Structure Introduction

3. 1930s: applications of probability sampling 1960s: expansion of telephone surveys 1970s: computer technology appears in surveys 1980s: computer-assisted surveying 1990s: Internet mediated surveys Technology and surveys Computer-assisted survey information collection

4. CATI – Computer-assisted telephone interviewing CAPI – Computer-assisted personal interviewing CASI – Computer-assisted self-interviewing Early CASIC interviewer assisted modes Computer-assisted survey information collection

5. Reduced time and costs for data input Elimination of errors during data transcription Implementation of advanced features:  automatic skips and branching  randomization of questions and response options  control of answers  inclusion of multimedia elements… CASIC benefits Computer-assisted survey information collection

6. Different modes of CSAQ:  disk-by-mail  touch-tone data entry (TDE)  interactive voice response (IVR)  e-mail surveys  web surveys Computerized self-administered questionnaires (CSAQ) Computer-assisted survey information collection

7. CASIC (e.g. CATI, CAPI, CASI, etc.) Online surveys Internet surveys Computer-assisted survey information collection Web surveys Internet surveys Online, Internet, Web, CASIC, CSAQ, CADAC

8. CASIC (e.g. CATI, CAPI, CASI, etc.) Online surveys Internet surveys Computer-assisted survey information collection Web surveys Internet surveys CSAQ Online, Internet, Web, CASIC, CSAQ, CADAC

9. CASIC (e.g. CATI, CAPI, CASI, etc.) Online surveys Internet surveys Computer-assisted survey information collection Web surveys Internet surveys CSAQ CADAC Online, Internet, Web, CASIC, CSAQ, CADAC

10. Interviewer involvement Survey mode Paper and pencil CASIC Interviewer presence Paper and pencil (face-to- face) interviewing (PAPI) CAPI, CASI, Audio/Video CASI Remote interviewer Paper assisted telephone interviewing (PATI) CATI, CAVI (computer assisted video interviewing) No interviewer Self-administered paper questionnaires (mail questionnaires) CSAQ telesurveys (web CSAQ, TDE, IVR, Virtual interviewer,...) Interviewer-less and paper-less surveys Computer-assisted survey information collection

11. Interviewer involvement Survey mode Paper and pencil CASIC Interviewer presence Paper and pencil (face-to- face) interviewing (PAPI) CAPI, CASI, Audio/Video CASI Remote interviewer Paper assisted telephone interviewing (PATI) CATI, CAVI (computer assisted video interviewing) No interviewer Self-administered paper questionnaires (mail questionnaires) CSAQ telesurveys (web CSAQ, TDE, IVR, Virtual interviewer,...) Interviewer-less and paper-less data collection Computer-assisted survey information collection

12. Asking question (INPUT) Recording of responses (OUTPUT) Manual recording Automatic voice recognition Written questions Standard CSAQ (PC, TV, PDA, Mobile,...) Visual CSAQ with AVR Audio questions Video CSAQ, Audio CSAQ, TDE IVR, Video IVR, TTS with AVR, Virtual interview CSAQ input-output technology Computer-assisted survey information collection

13. Asking question (INPUT) Recording of responses (OUTPUT) Manual recording Automatic voice recognition Written questions Standard CSAQ (PC, TV, PDA, Mobile,...) Visual CSAQ with AVR Audio questions Video CSAQ, Audio CSAQ, TDE IVR, Video IVR, TTS with AVR, Virtual interview CSAQ input-output technology Computer-assisted survey information collection

14. Developments of the Internet and related technologies Importance of broadband Internet access Possibilities of distribution across various platforms and devices Fast and easy implementation using specialized software tools Technological aspects Computer-assisted survey information collection

15. Two major problems of probability samples in Internet surveys:  non-coverage  sampling frame problems Image of Internet surveys as inherently non-probability ones 1. Probability vs. non-probability sampling Key methodological issues

16. Issue of a statistical inference: probability vs. non-probability samples Internet survey Information-communication technologies Mode of survey data collection Management of the survey process The spurious link Key methodological issues

17. Sampling type Solicitation Type of invitation Personal invitationGeneral invitation Probability samples Samples from closed population with email, Pre-recruited with mail, telephone, f2f Web intercept surveys Non-probability samples Self-recruited Internet panels, Opt-in lists of e-mail adresses General banner-ad invitation 2. Sampling and invitation Key methodological issues

18. Generally low and variable response rates to web surveys Complexity of the response process Measures to increase response rates:  incentives (e.g. using PayPal)  multiple follow-up contacts  other motivations 3. Non-response Key methodological issues

19. Target population Operational population Frame population Sampled units Absorbed units Complete respondents Partial respondents Starting units Clickers Aware units Solicitation Informed units Involvement process Key methodological issues

20. Respondent’s characteristics Social environment Researcher- respondent interaction Survey design Technological environment No researcher’s control Partially under researcher’s control Participation in web surveys Key methodological issues

21. Question types and visual elements Advanced features of computerized questionnaires Inclusion of multimedia Problems and considerations:  standardization  impact on respondent’s answers  technical problems measurement error 4. Questionnaire design Key methodological issues

22. Commonly utilized for correcting:  deviations from probability selections  non-coverage and non-response problems Importance for non-probability Internet surveys Promising approaches:  calibration methods  propensity score weighting  modelling and causal analysis 5. Post-survey adjustments Key methodological issues

23. The managerial process of survey implementation:  costs optimization  data quality  relations between data quality and costs Cost effectiveness of web surveys:  costs for increasing response rates  evaluations of costs in relation to errors 1. Costs, errors and management Related issues

24. Web surveys vs. other modes:  lower response rates (Lozar Manfreda et. al., 2007)  comparable or higher level of validity and reliability (e.g. Fricker et al., 2005; Chang & Krosnick, 2002) 2. Survey mode Related issues

25. Contact with the respondent personal, telephone, e-mail, mail,… Interview PAPI, CAPI, CATI, CASI,… Centralized data management Solicitation Mixing survey modes Related issues

26. Response rates and behaviour Modelling mixed modes Data quality requirements Willingness to participate Structure of the data (MAR) Cost models and impact of environment Optimal combination Selecting the right mixture Related issues

27. Programing of the questionaire Related issues User-friendly interface Standardized questionnaire description in QML Paper questionnaire CAPI CATI CASI Internet survey

28. Quantitative and qualitative research methods Flexible combinations of approaches (Morgan, 1998) :  qualitative preliminary  quantitative preliminary  qualitative follow-up  quantitative follow-up New possibilities with the Internet 3. Mixing research methods Related issues

29. Qualitative method Quantitative method Analysis Beginning Conclusion Flexible on-line mixed methods (FOMM) Related issues

30. New ethical dilemmas in survey research:  solicitation process, spam and privacy  data protection  children and minors  combining data… Development of new standards (e.g. ESOMAR, AoIR, MRA…) 4. Ethics, guides and standards Related issues

31. Different applications according to:  type (internet access panels, on-line voting, intercept…)  organizations (official, academic, commercial)  topics (marketing research, polling opinions, health…)  complexity (simple one-shot surveys vs. complex integrated data collection) Applications Current and future applications

32.  psychological research  online experiments  paradata and online measurement  specific research fields (usability research, HCI...)  emerging areas (e-learning, e-government…) Related areas Current and future applications

33. Expansion of CASIC modes, options and mixtures Integration of devices (TV, mobile,..) Speech recognition and touch-screens Multimedia and interactivity Integration with other CADAC (GPS, paradata, o bservations,...) Specialisation and professionalisation Trends Current and future applications