The basics of Social Science Research Lecture 3 Dr. Jordy Gosselt Department of Communication Science

Today Research methods and Data collection Making a questionnaire April 3th: Data analysis (making a data file in spss, calculating the reliability of your instrument, making variables, calculating Means/SD, comparing groups) Crime Science Theories (M. Junger)  Book Chapter 5, 6, 7, 9, 10 [Bha2012] Bhattacherjee, A. (2012). Social Science Research. Principles, Methods, and Practices. Tampa, Florida: University of South Florida. PDF http://scholarcommons.usf.edu/cgi/viewcontent.cgi?article=1002&context=oa_textbooks

Now: Making a questionnaire Social Science Research Training: Aim of the study?

Now: Making a questionnaire Study: The privacy paradox observed in normal behavior Theory: Users claim to be highly concerned about their privacy (=attitude) Consumer’s choice to download an app is driven by (1) popularity; (2) usability; (3) price, without much hesitation (=actual behavior) Privacy paradox? For the largest part of lay people, technical processes that run in the background are not understandable: users make use of risky online services even if they have privacy or security concerns. Knowledge gap? Users have the tendency to accept all kinds of permissions although having privacy concerns. RQ1: To what extent do people understand how permissions work? RQ2: What are the perceived concerns regarding permissions? RQ3: To what extent do people behave differently regarding different kinds of permissions and does the type of app have an influence? RQ4: To what extent is there a gap between self-reported and actual behavior?

Now: Making a questionnaire Study: The privacy paradox observed in normal behavior RQ1: To what extent do people understand how permissions work? RQ2: What are the perceived concerns regarding permissions? RQ3: To what extent do people behave differently regarding different kinds of permissions and does the type of app have an influence? RQ4: To what extent is there a gap between self-reported and actual behavior?

Now: Making a questionnaire Combination of survey, qualitative and experiment Part 1: Evaluation of participants’ general smartphone and app usage: survey Part 2: Measuring users’ general privacy sensitivity: open question and survey Part 3: Level of comfort with accepting permissions and granting access: experiment Part 4: Comprehension of permissions: quiz Part 5: Permissions and perceived concern in different risk scenarios: survey RQ1: To what extent do people understand how permissions work? (Part 4) RQ2: What are the perceived concerns regarding permissions? (Part 2 and Part 5) RQ3: To what extent do people behave differently regarding different kinds of permissions and does the type of app have an influence? (Part 3) RQ4: To what extent is there a gap between self-reported and actual behavior? (Part 1 and Part 3)

Now: Making a questionnaire Participant information (goal / relevance / motivating) Informed consent (withdrawal) Ask for demographical data: Participant information + informed consent = see CCS website Demographics: Ask as much as you need to carefully describe your sample but do not ask useless information

Now: Making a questionnaire Part 1: Evaluation of participants’ general smartphone and app usage : survey Measuring general smartphone and app usage  is there any influence of (indicated) behavior on risk perception

Now: Making a questionnaire Part 1: Evaluation of participants’ general smartphone and app usage : survey Measuring general smartphone and app usage  is there any influence of (indicated) behavior on risk perception

Now: Making a questionnaire Part 1: Evaluation of participants’ general smartphone and app usage : survey Measuring general smartphone and app usage  is there any influence of (indicated) behavior on risk perception

Now: Making a questionnaire Part 2: Measuring users’ general privacy sensitivity: open question and survey

Now: Making a questionnaire Part 2: Measuring users’ general privacy sensitivity: open question and survey Scale: Westin Privacy Index (3 items, privacy sensitivity; see Kumaraguru & Cranor, 2005) Westin Privacy Index = General privacy sensitivity Does the general privacy sensitivity has an influence on risk perception and level of comfort when accepting permissions? [Kum2005] Kumaraguru, P. & Cranor L. F., 2005. Privacy indexes : a survey of Westin's studies. Technical report. Carnegie Mellon University: Pittsburgh, Pennsylvania. http://repository.cmu.edu/cgi/viewcontent.cgi?article=1857&context=isr

Now: Making a questionnaire Part 3: Level of comfort with accepting permissions and granting access: experiment Level of comfort and granting access for 2 apps: Hedonic app (tetris) vs utilitarian app (to-do-list) Choice of apps based on a pre-test Selection of permissions (n=7) based on Taylor & Martinovic, 2016 Answers on comfort and granting access Is there a difference in level of comfort between the evaluation and allowing of permissions of hedonic vs utilitarian app? [Tay2016] Taylor, V.F., & Martinovic, I. (2016). SecuRank: Starving permission-hungry apps using contextual permission analysis. Proceedings of the 6th Workshop on Security and Privacy in Smartphones and Mobile Devices, pp. 43-52, October 24 - 24, 2016 Vienna, Austria. https://dl.acm.org/citation.cfm?id=2994474

Now: Making a questionnaire Part 4: Comprehension of permissions: quiz Adopted from Felt et al. (2012) What is your comprehension level of permissions? Does the comprehension level match with risk perception and level of comfort when granting access to permissions? Adopted from Felt, Ha, Egelman, Haney, Chin & Wagner, 2012 – Permission Comprehension Quiz to the permissions asked in the experiment. Approx. 50% of the items are reused, the rest is modified. [Fel2012] Felt, A. P., Ha, E., Egelman, S., Haney, A., Chin, E., & Wagner, D. (2012). Android permissions: User attention, comprehension, and behavior. Proceedings of the eighth symposium on usable privacy and security, 1-14. https://www.cs.ucy.ac.cy/courses/EPL682/papers/usable-1.pdf

Now: Making a questionnaire Part 5: Permissions and perceived concern in different risk scenarios: survey Ranked list of risk scenarios: Is there a match between level of comfort and granting access to permissions?

Next: Analyzing results Combination of survey, qualitative and experiment Part 1: Evaluation of participants’ general smartphone and app usage: survey Part 2: Measuring users’ general privacy sensitivity: open question and survey Part 3: Level of comfort with accepting permissions and granting access: experiment Part 4: Comprehension of permissions: quiz Part 5: Permissions and perceived concern in different risk scenarios: survey RQ1: To what extent do people understand how permissions work? (Part 4) RQ2: What are the perceived concerns regarding permissions? (Part 2 and Part 5) RQ3: To what extent do people behave differently regarding different kinds of permissions and does the type of app have an influence? (Part 3) RQ4: To what extent is there a gap between self-reported and actual behavior? (Part 1 and Part 3)

To sum up Combination of research methods Method dependent on RQ (knowledge, attitude, behavior) Use existing scales (reliable!) Do a pre-test if possible (validity!) regarding: Stimulus material Questions + answers

Data Collection Type of data collection: qualitative vs. quantitative Expected amount of collected data Online questionnaire: Between 200 and 250 respondents Hard-copy questionnaire: Between 100 and 150 respondents Experimental designs: Depending on number of conditions (min 30 resp. per condition) Interviews: Between 15 and 20 participants Focus groups: Between 4 and 6 group sessions Content analysis: Depends on the research design Observations: Depends on the research design

Data Collection: Qualtrics Data collection procedure Online survey tool “Qualtrics” is available in cases of quantitative data collection (i.e. surveys, experiments, sorting tasks, heat maps...) Easy to use and distribute Direct export to SPSS

Data Collection: Qualtrics Data collection procedure Online survey tool “Qualtrics” is available in cases of quantitative data collection (i.e. surveys, experiments, sorting tasks, heat maps...) Easy to use and distribute Direct export to SPSS

Your research proposal: Refine RQ(s) based on literature De/Refine your concept(s), construct(s) and variable(s) Determine how to examine your RQ(s) The research method must fit your RQ(s) Define your target group and sampling method Operationalize your chosen method Make or find scales / make experimental groups / make interview questions / etc. and think of validity and reliability Fill in the documents for the ethical committee Contact your student assistant if you have questions