Designing Effective Surveys with Qualtrics Gareth Johns IT Skills Development Advisor 1

Course Aim 2 Use Qualtrics software to create and distribute surveys that effectively capture the data you require.

Survey 3 “A systematic collection and analysis of data relating to the attitudes, living conditions, opinions, etc. of a population” Oxford English Dictionary

Survey vs Questionnaire 4 “A formulated series of questions by which information is sought from a selected group, usually for statistical analysis; a document containing these.” Oxford English Dictionary “A formulated series of questions by which information is sought from a selected group, usually for statistical analysis; a document containing these.” Oxford English Dictionary

Qualtrics 5 Getting to know you Open your to complete survey: staff .cardiffmet.ac.uk cardiffmet.qualtrics.com

Objectives By the end of this session you should be able to: Plan the design and structure of a survey to capture data in a coherent way. Use Qualtrics to create a range of question types, including: Multiple Choice, Text Entry, Matrices, Rank Order and Sliders. Utilise survey logic to adjust survey flow, based on previous responses. Customise the Look & Feel of a survey. Distribute the survey in a range of ways, including anonymous distribution, “panels” and through social media. Adhere to good survey design practices to gather the data you require, effectively. 6

Likert scale A Likert scale can be used to measure behaviour and (especially) attitude. Comprised of several Likert items; statements that respondents evaluate and respond to – typically by agreeing or disagreeing. 7

Likert scale (2/4) Results for each Likert item can be analysed individually… 8 …or summed into a Likert scale to establish a wider measure of the attitude.

Likert scale (3/4) Advantages: Familiar – easy to answer Results approximate to an interval scale – can calculate mean, standard deviation etc. More reliable – “we should not rely on single questions when we measure attitudes; we should have sets of questions or attitude scales” (Oppenheim, 1992 p.147) 9

Likert scale (4/4) Disadvantages: Biases  Acquiescence - “yea-saying”  Central-tendency – choosing the middle option  Pattern-answering  Primacy effect – choosing earlier options (More info on bias: Brace (2005) – Ch.5, Visser et. al (2000)) Interval or ordinal scale? Is difference between each option semantically equal? 10

Recap You should now be able to: Plan the design and structure of a survey to capture data in a coherent way. Use Qualtrics to create a range of question types, including: Multiple Choice, Text Entry, Matrices, Rank Order and Sliders. Utilise survey logic to adjust survey flow, based on previous responses. Customise the Look & Feel of a survey. Distribute the survey in a range of ways, including anonymous distribution, “panels” and through social media. Adhere to good survey design practices to gather the data you require, effectively. 11

Feedback Your feedback is valuable to us. Please help us to improve the relevance, content and delivery of our IT Training courses by completing a brief questionnaire. Please check your s for survey link Thank you for attending 12

Further Study E-learning Learn Qualtrics in 5 Steps – Qualtrics Online Training: qualtrics-in-5-steps/ qualtrics-in-5-steps/ Qualtrics eBooks - resources/ebook-downloads/ resources/ebook-downloads/ Questionnaire Design for Social Surveys – Coursera.com – available until 31 st March Books Blair, J., Czaja, R., Blair, E. (2014) Designing Surveys: A Guide to Decisions and Procedures. London: Sage Publications. Oppenheim, A. (1992) Questionnaire Design, Interviewing and Attitude Measurement. New edition. New York: Cassell Brace, I. (2008) Questionnaire design: how to plan, structure and write survey material for effective market research. Kogan Page: London. Visser, P., Krosnick, J., Lavrakas, P. (2000). ‘Survey research’ in Reis, H. ; Judd, C. (eds) Handbook of research methods in social and personality psychology., (pp ). Cambridge University Press New York 13

Additional Information Session hand outs and other resources available at: tsr.cardiffmet.ac.uk/Learning/Help/Training/ Upcoming training sessions: New Summer Term schedule – Easter Break 14

Contact Us 15 Visit an IT Advisor in the Learning Centres