The Basics of Social Science Research Methods Susanne Barth, MSc 08.03.2017

Structure of the lecture Research Methods and Data collection case: Social Research Training  Book Chapter 5, 6, 7, 9, 10 [Bha2012] Bhattacherjee, A. (2012). Social Science Research. Principles, Methods, and Practices. Tampa, Florida: University of South Florida. PDF http://scholarcommons.usf.edu/cgi/viewcontent.cgi?article=1002&context=oa_textbooks

Being a participant Third part of the study

Your research process Exploration: Research question(s) Literature review Research design: Operationalization Research method Sampling = Research proposal

Research method (I) Method should fit your RQ! Quantitative Survey Experimental design field vs laboratory Qualitative observations/interview/focus groups Method should fit your RQ! Survey: you may use existing scales or develop your own scale (then, you should run a pre-test), Experiment: field experiment: your are able to observe actual behavior, difficult because manipulation is almost not doable laboratory experiment: manipulation of two conditions are possible, but is this a good reflection of real behavior? Qualitative: face-to-face interview (develop questions (interview scheme) or use existing questions from other research) Focus groups: usually between 6-10 respondents that discuss a certain issue. Difficulty: participants may influence each other due to their responses. On the other hand you might get in-depth insights on the topic you wish to examine. Different methods vary with regard to these two aspects of validity. Experiments, because they tend to be structured and controlled, are often high on internal validity. However, their strength with regard to structure and control, may result in low external validity. The results may be so limited as to prevent generalizing to other situations. In contrast, observational research may have high external validity (generalizability) because it has taken place in the real world. However, the presence of so many uncontrolled variables may lead to low internal validity in that we can't be sure which variables are affecting the observed behaviors.

Research method (II): Reliability vs Validity (I) https://www.youtube.com/watch?v=tMERBdjs8Dw

Research method (III): Reliability vs Validity (II) Reliability: extent to which a measurement tool gives consistent results Validity: extent to which a measurement tool measures what it is supposed to measure In order to be valid an instrument has to be reliable Reliability: e.g. you weight yourself several times during a day and the scale does not vary much = high reliability you weight yourself several times during a day and the scale drastically varies = low reliability Validity: e.g. Intelligence test that actually measures intelligence = high validity Intelligence test that measures memory instead = low validity

Research method (IV): Reliability (I) Test-retest reliability = consistency over time  e.g. pre-test or pre and post measurement Inter-coder/observer reliability (agreement between two or more coders) = Cohens Kappa Internal consistency reliability = loading of items e.g. Cronbach’s Alpha Reliability: e.g. you weight yourself several times during a day and the scale does not vary much = high reliability you weight yourself several times during a day and the scale drastically varies = low reliability Validity: e.g. Intelligence test that actually measures intelligence = high validity Intelligence test that measures memory instead = low validity Item 1 Item 2 Item 3 Construct Item 4 Item 5 Item 6

Research method (V): Reliability (II) Manipulation: experimental and control group (only for experimental design) Elimination or inclusion of extraneous variables e.g. gender Random sampling (e.g. not only family members) Random assignment (e.g. random assignment to treatment groups) Reliability: e.g. you weight yourself several times during a day and the scale does not vary much = high reliability you weight yourself several times during a day and the scale drastically varies = low reliability Validity: e.g. Intelligence test that actually measures intelligence = high validity Intelligence test that measures memory instead = low validity

Research method (VI): Validity (I) internal validity (change in dependent variable is caused by independent variable = causality)  experimental design external validity (generalizability from sample to population)  broad and divers sample construct validity (does your instrument measure the construct)  carefully define your construct and develop valid instrument or use existing instrument Reliability: e.g. you weight yourself several times during a day and the scale does not vary much = high reliability you weight yourself several times during a day and the scale drastically varies = low reliability Validity: e.g. Intelligence test that actually measures intelligence = high validity Intelligence test that measures memory instead = low validity

Now: Social Science Research Training (I) Research execution: The Social Science Research Training Aim of the study?

Social Science Research Training (II) Study split into three parts Combination of survey, experiment and qualitative design 1. part: survey 2. part: experiment downloading app description of your downloading process short survey 3. part: review + survey

Social Science Research Training (III) First part of the study (I) Participant information Informed consent Ask for demographical data: Participant information + informed consent = see CCS website Demographics: Ask as much as you need to carefully describe your sample but do not ask useless information

Social Science Research Training (IV) First part of the study (II) Scale: Smartphone knowledge and skills (adapted from Androulidakis & Kandus, 2011, 2012) Scale smartphone knowledge and skills  no Cronbachs alpha available but quite a lare sample, more than 7000 participants Measurement of smartphones knowledge and skills  is there any influence of the level of knowledge on the decision making process to download an app? [And2011] Androulidakis, I., & Kandus, G. (2011). Mobile phone security awareness and practices of students in Budapest. Availability, Reliability and Security (ARES), The Sixth International Conference on Digital Telecommunications, Budapest, Hungary. https://www.google.nl/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&ved=0ahUKEwjambHKo8bSAhUGOhQKHVqxCZcQFghBMAA&url=https%3A%2F%2Fwww.thinkmind.org%2Fdownload.php%3Farticleid%3Dicdt_2011_1_40_20110&usg=AFQjCNHcKrrmozMQgJ2TQph6NlMU-mhXAQ&bvm=bv.148747831,d.d24

Social Science Research Training (V) First part of the study (III) Scale: Perceived usefulness (Chuah et al., 2016) Scale: Perceived ease of use (Yang, 2013) High Cronbach‘s alpha see high internal consistency reliability = items measure the constructs you aim to examine [Chu2016] Chuah, S. H-W., et al. (2016). Wearable technologies: The role of usefulness and visibility in smartwatch adoption. Computers in Human Behavior, 65, 276-284. http://www.philipprauschnabel.com/wp-content/uploads/2015/06/chuah-rauschnabel-et_al-2016-smartwatches-adoption-fashnology-CHB.pdf [Yan2013] Yang, H. C. (2013). Bon appétit for apps: Young American consumers’ acceptance of mobile applications. Journal of Computer Information Systems, 53(3), 85-96. https://www.researchgate.net/profile/Hongwei_Yang/publication/256585586_Bon_Appetit_for_Apps_Young_American_Consumers'_Acceptance_of_Mobile_Applications/links/00b7d523744cb27c2f000000.pdf

Social Science Research Training (VI) First part of the study (IV) Scale: Westin Privacy Index (see Kumaraguru & Cranor, 2005) Westin Privacy Index = General privacy sensitivity Does the general privacy sensitivity has an influence on the decision which app to download? [Kum2005] Kumaraguru, P. & Cranor L. F., 2005. Privacy indexes : a survey of Westin's studies. Technical report. Carnegie Mellon University: Pittsburgh, Pennsylvania. http://repository.cmu.edu/cgi/viewcontent.cgi?article=1857&context=isr

Social Science Research Training (VII) First part of the study (V) Scale: Factors that play a role in the selection process of downloading an app (own scale) Verification of experiment: do you mention consistent factors that play a role in the selection process of downloading an app (comparison with review and which app you actually downloaded)

Social Science Research Training (VIII) Second part of the study (I) hedonic app (game) vs utilitarian app (to-do-list), choice based on a pre-test Mellow Meadows Tower Defense (0 permission but 1,55 Euro) New Eskimo Defense (2 permissions and 0,96 Euro) Astroid Defense Classic (4 permissions, for free) Safe the cave: Tower Defense (6 permissions, for free) Tower Defense: Infinite War (7 permissions, for free) And the same for group 2 (to-do-list) Apps increasing in permissions that are not related to functionality (e.g. in-app purchases, identity, contact, photos, WIFI connection etc.)

Social Science Research Training (IX) Second part of the study (II) Verification of experiment: do you mention consistent factors that play a role in the selection process of downloading an app (comparison with the questions from the first questionnaire) Explanation of downloading process = evaluating your decision making process: Did you followed a risk-benefit calculation or not e.g. do you mention different factors after having downloaded an app?

Social Science Research Training (X) Third part of the study (I) Manipulation check Which app did you download? Verification of app store: only include data from participants that have used Google Play Store Did you buy the app or not

Social Science Research Training (XI) Third part of the study (II) Review What factors play a role? Review: combination of qualitative and quantitative data: makes the reliability of your study stronger Do you mention different or additional factors in the review compared to the factors we asked for in the questionnaire?

Social Science Research Training (XII) Third part of the study (III) Scale: Factors influencing intention to install mobile applications (Harris et al., 2016) Perceived security Application characteristics Positive reputation Familiarity Desensitization Perceived risk Consumer trust Perceived benefits Intent to install This scale is more privacy related: If we ask this prior to the experiment you might be biased in terms of privacy and probably consider different factors while selecting an app [Har2016]. Harris, M. A., Brookshire, R., & Chin, A. G. (2016). Identifying factors influencing consumers’ intent to install mobile applications. International Journal of Information Management, 36, 441–450. http://s3.amazonaws.com/academia.edu.documents/43653825/Published_Paper.pdf?AWSAccessKeyId=AKIAIWOWYYGZ2Y53UL3A&Expires=1488971234&Signature=tLGViwEP%2BC1CSqe5JMzJf8qwr6k%3D&response-content-disposition=inline%3B%20filename%3DIdentifying_factors_influencing_consumer.pdf

Social Science Research Training (XIII) Third part of the study (IV) Scale: Mobile users’ information privacy concerns (Xu et al., 2012) Perceived surveillance Perceived intrusion Secondary use of personal information Prior Privacy Experience Behavioral Intention This scale is more privacy related: If we ask this prior to the experiment you might be biased in terms of privacy and probably consider different factors while selecting an app [Xu2012]. Xu, H., Gupta, S., Rosson, M. B., & Carroll, J. M. (2012). Measuring mobile users’ concerns for information privacy. Thirty Third International Conference on Information Systems, Orlando, 1-16. http://s3.amazonaws.com/academia.edu.documents/46247454/viewcontent.pdf?AWSAccessKeyId=AKIAIWOWYYGZ2Y53UL3A&Expires=1488971444&Signature=1nYuWzo9gnZ7nEVsR4kNMhwXk04%3D&response-content-disposition=inline%3B%20filename%3DMeasuring_Mobile_Users_Concerns_for_Info.pdf

To sum up: Social Science Research Training Develop an experiment costs you time (and money) Do a pre-test if possible (validity!) Use existing scales (reliable!) Combination of research methods

Your research proposal: Refine RQ(s) based on literature De/Refine your concept(s), construct(s) and variable(s) Determine how to examine your RQ(s) The research method must fit your RQ(s) Define your target group and sampling method Operationalize your chosen method Make or find scales / make experimental groups / make interview questions / etc. and think of validity and reliability Fill in the documents for the ethical committee Contact your student assistant if you have questions

© by WingedWolf