1. The Basics of Social Science Research Methods
Susanne Barth, MSc

2. Structure of the lecture
Research Methods and Data collection
case: Social Research Training
 Book Chapter 5, 6, 7, 9, 10
[Bha2012] Bhattacherjee, A. (2012). Social Science Research. Principles, Methods, and Practices. Tampa, Florida: University of South Florida. PDF

3. Being a participant
Third part of the study

4. Your research process Exploration: Research question(s)
Literature review
Research design:
Operationalization
Research method
Sampling
= Research proposal

5. Research method (I) Method should fit your RQ! Quantitative
Survey
Experimental design
field vs laboratory
Qualitative
observations/interview/focus groups
Method should fit your RQ!
Survey: you may use existing scales or develop your own scale (then, you should run a pre-test),
Experiment: field experiment: your are able to observe actual behavior, difficult because manipulation is almost not doable
laboratory experiment: manipulation of two conditions are possible, but is this a good reflection of real behavior?
Qualitative: face-to-face interview (develop questions (interview scheme) or use existing questions from other research)
Focus groups: usually between 6-10 respondents that discuss a certain issue. Difficulty: participants may influence each other due to their responses. On the other hand you might get in-depth insights on the topic you wish to examine.
Different methods vary with regard to these two aspects of validity. Experiments, because they tend to be structured and controlled, are often high on internal validity. However, their strength with regard to structure and control, may result in low external validity. The results may be so limited as to prevent generalizing to other situations.
In contrast, observational research may have high external validity (generalizability) because it has taken place in the real world. However, the presence of so many uncontrolled variables may lead to low internal validity in that we can't be sure which variables are affecting the observed behaviors.

6. Research method (II): Reliability vs Validity (I)

7. Research method (III): Reliability vs Validity (II)
Reliability: extent to which a measurement tool gives
consistent results
Validity: extent to which a measurement tool
measures what it is supposed to measure
In order to be valid an instrument
has to be reliable
Reliability: e.g. you weight yourself several times during a day and the scale does not vary much = high reliability
you weight yourself several times during a day and the scale drastically varies = low reliability
Validity: e.g. Intelligence test that actually measures intelligence = high validity Intelligence test that measures memory instead = low validity

8. Research method (IV): Reliability (I)
Test-retest reliability
= consistency over time
 e.g. pre-test or
pre and post measurement
Inter-coder/observer reliability
(agreement between two or more coders)
= Cohens Kappa
Internal consistency reliability
= loading of items
e.g. Cronbach’s Alpha
Reliability: e.g. you weight yourself several times during a day and the scale does not vary much = high reliability
you weight yourself several times during a day and the scale drastically varies = low reliability
Validity: e.g. Intelligence test that actually measures intelligence = high validity Intelligence test that measures memory instead = low validity
Item 1
Item 2
Item 3
Construct
Item 4
Item 5
Item 6

9. Research method (V): Reliability (II)
Manipulation: experimental and control group (only for experimental design)
Elimination or inclusion of extraneous variables e.g. gender
Random sampling (e.g. not only family members)
Random assignment (e.g. random assignment to treatment groups)
Reliability: e.g. you weight yourself several times during a day and the scale does not vary much = high reliability
you weight yourself several times during a day and the scale drastically varies = low reliability
Validity: e.g. Intelligence test that actually measures intelligence = high validity Intelligence test that measures memory instead = low validity

10. Research method (VI): Validity (I)
internal validity (change in dependent variable is caused by independent variable = causality)
 experimental design
external validity (generalizability from sample to population)
 broad and divers sample
construct validity (does your instrument measure the construct)
 carefully define your construct and develop valid instrument or use existing instrument
Reliability: e.g. you weight yourself several times during a day and the scale does not vary much = high reliability
you weight yourself several times during a day and the scale drastically varies = low reliability
Validity: e.g. Intelligence test that actually measures intelligence = high validity Intelligence test that measures memory instead = low validity

11. Now: Social Science Research Training (I)
Research execution: The Social Science Research Training
Aim of the study?

12. Social Science Research Training (II)
Study split into three parts
Combination of survey, experiment and qualitative design
1. part: survey
2. part: experiment
downloading app
description of your downloading process
short survey
3. part: review + survey

13. Social Science Research Training (III) First part of the study (I)
Participant information
Informed consent
Ask for demographical data:
Participant information + informed consent = see CCS website
Demographics: Ask as much as you need to carefully describe your sample but do not ask useless information

14. Social Science Research Training (IV) First part of the study (II)
Scale: Smartphone knowledge and skills (adapted from Androulidakis & Kandus, 2011, 2012)
Scale smartphone knowledge and skills  no Cronbachs alpha available but quite a lare sample, more than 7000 participants
Measurement of smartphones knowledge and skills  is there any influence of the level of knowledge on the decision making process to download an app?
[And2011] Androulidakis, I., & Kandus, G. (2011). Mobile phone security awareness and practices of students in Budapest. Availability, Reliability and Security (ARES), The Sixth International Conference on Digital Telecommunications, Budapest, Hungary.

15. Social Science Research Training (V) First part of the study (III)
Scale: Perceived usefulness (Chuah et al., 2016)
Scale: Perceived ease of use (Yang, 2013)
High Cronbach‘s alpha see high internal consistency reliability = items measure the constructs you aim to examine
[Chu2016] Chuah, S. H-W., et al. (2016). Wearable technologies: The role of usefulness and visibility in smartwatch adoption. Computers in Human Behavior, 65,
[Yan2013] Yang, H. C. (2013). Bon appétit for apps: Young American consumers’ acceptance of mobile applications. Journal of Computer Information Systems, 53(3),

16. Social Science Research Training (VI) First part of the study (IV)
Scale: Westin Privacy Index (see Kumaraguru & Cranor, 2005)
Westin Privacy Index = General privacy sensitivity
Does the general privacy sensitivity has an influence on the decision which app to download?
[Kum2005] Kumaraguru, P. & Cranor L. F., Privacy indexes : a survey of Westin's studies. Technical report. Carnegie Mellon University: Pittsburgh, Pennsylvania.

17. Social Science Research Training (VII) First part of the study (V)
Scale: Factors that play a role in the selection process of downloading an app (own scale)
Verification of experiment: do you mention consistent factors that play a role in the selection process of downloading an app (comparison with review and which app you actually downloaded)

18. Social Science Research Training (VIII) Second part of the study (I)
hedonic app (game) vs utilitarian app (to-do-list), choice based on a pre-test
Mellow Meadows Tower Defense (0 permission but 1,55 Euro)
New Eskimo Defense (2 permissions and 0,96 Euro)
Astroid Defense Classic (4 permissions, for free)
Safe the cave: Tower Defense (6 permissions, for free)
Tower Defense: Infinite War (7 permissions, for free)
And the same for group 2 (to-do-list)
Apps increasing in permissions that are not related to functionality (e.g. in-app purchases, identity, contact, photos, WIFI connection etc.)

19. Social Science Research Training (IX) Second part of the study (II)
Verification of experiment: do you mention consistent factors that play a role in the selection process of downloading an app (comparison with the questions from the first questionnaire)
Explanation of downloading process = evaluating your decision making process: Did you followed a risk-benefit calculation or not e.g. do you mention different factors after having downloaded an app?

20. Social Science Research Training (X) Third part of the study (I)
Manipulation check
Which app did you download?
Verification of app store: only include data from participants that have used Google Play Store
Did you buy the app or not

21. Social Science Research Training (XI) Third part of the study (II)
Review
What factors play a role?
Review: combination of qualitative and quantitative data: makes the reliability of your study stronger
Do you mention different or additional factors in the review compared to the factors we asked for in the questionnaire?

22. Social Science Research Training (XII) Third part of the study (III)
Scale: Factors influencing intention to install mobile applications (Harris et al., 2016)
Perceived security
Application characteristics
Positive reputation
Familiarity
Desensitization
Perceived risk
Consumer trust
Perceived benefits
Intent to install
This scale is more privacy related: If we ask this prior to the experiment you might be biased in terms of privacy and probably consider different factors while selecting an app
[Har2016]. Harris, M. A., Brookshire, R., & Chin, A. G. (2016). Identifying factors influencing consumers’ intent to install mobile applications. International Journal of Information Management, 36, 441–450.

23. Social Science Research Training (XIII) Third part of the study (IV)
Scale: Mobile users’ information privacy concerns (Xu et al., 2012)
Perceived surveillance
Perceived intrusion
Secondary use of personal information
Prior Privacy Experience
Behavioral Intention
This scale is more privacy related: If we ask this prior to the experiment you might be biased in terms of privacy and probably consider different factors while selecting an app
[Xu2012]. Xu, H., Gupta, S., Rosson, M. B., & Carroll, J. M. (2012). Measuring mobile users’ concerns for information privacy. Thirty Third International Conference on Information Systems, Orlando,

24. To sum up: Social Science Research Training
Develop an experiment costs you time (and money)
Do a pre-test if possible (validity!)
Use existing scales (reliable!)
Combination of research methods

25. Your research proposal:
Refine RQ(s) based on literature
De/Refine your concept(s), construct(s) and variable(s)
Determine how to examine your RQ(s)
The research method must fit your RQ(s)
Define your target group and sampling method
Operationalize your chosen method
Make or find scales / make experimental groups / make interview questions / etc. and think of validity and reliability
Fill in the documents for the ethical committee
Contact your student assistant if you have questions

26. © by WingedWolf