1. The basics of Social Science Research Lecture 4
Dr. Jordy Gosselt
Department of Communication Science

2. Today Pilot test Data collection Data analysis Self-study
 Book Chapter 9, 10, 14 & 15
[Bha2012] Bhattacherjee, A. (2012). Social Science Research. Principles, Methods, and Practices. Tampa, Florida: University of South Florida. PDF

3. Your research process
Research method: Validity/reliability & Pilot Testing
Research execution:
Data collection
Data Analysis
Make your data file
Check reliability of your variables
Calculate frequencies, Means, SD
Compare groups

4. Research method: Reliability/Validity & Pilot Testing
Reliability: extent to which a measurement tool gives consistent results
Validity: extent to which a measurement tool measures what it is supposed to measure
Example: “To what extent do media ratings instigate a forbidden fruit effect”
Media ratings (=stimulus material): What games to select + What media ratings?
=> pretest
Forbidden fruit effect: How to measure this? Existing scales / New scale / Adjusted scale?
Reliability: e.g. you weight yourself several times during a day and the scale does not vary much = high reliability
you weight yourself several times during a day and the scale drastically varies = low reliability
Validity: e.g. Intelligence test that actually measures intelligence = high validity Intelligence test that measures memory instead = low validity

5. Research method: Reliability/Validity & Pilot Testing
Report about validity and reliability; what did you do to establish sufficient validity and reliability for your research:
Have you used existing measurement scales?
Have you performed a pre-study (pilot test) to test stimulus materials?
Is the wording of the questions correct?
Do you measure what you intent to measure? (validity)
Gives your measurement consistent results? (internal consistency reliability)
 Cronbach’s Alpha Calculation
How did you select participants?
Take into account possible threats regarding validity and reliability and address those threats in your methods section.
Reliability: e.g. you weight yourself several times during a day and the scale does not vary much = high reliability
you weight yourself several times during a day and the scale drastically varies = low reliability
Validity: e.g. Intelligence test that actually measures intelligence = high validity Intelligence test that measures memory instead = low validity

6. Your research process
Research method: Validity/reliability & Pilot Testing
Research execution:
Data collection
Data Analysis
Make your data file
Check reliability of your variables
Calculate frequencies, Means, SD
Compare groups

7. Data Collection Type of data collection: qualitative vs. quantitative
Expected amount of collected data
Online questionnaire: Between 200 and 250 respondents
Hard-copy questionnaire: Between 100 and 150 respondents
Experimental designs: Depending on number of conditions (min 30 resp. per condition)
Interviews: Between 15 and 20 participants
Focus groups: Between 4 and 6 group sessions
Content analysis: Depends on the research design
Observations: Depends on the research design

8. Data Collection: Qualtrics
Data collection procedure
Online survey tool “Qualtrics” is available in cases of quantitative data collection (i.e. surveys, experiments, sorting tasks, heat maps...)
Easy to use and distribute
Direct export to SPSS

9. Data Collection: Qualtrics

10. Data Collection: Qualtrics

11. Data Collection: Qualtrics

12. Data Collection: Qualtrics
Many features and options: see video’s on YouTube for instruction and on Qualtrics’ website
Information to register for an account can be found at:

13. Your research process
Research method: Validity/reliability & Pilot Testing
Research execution:
Data collection
Data Analysis
Make your data file
Check reliability of your variables
Calculate frequencies, Means, SD
Compare groups

14. Make your data file

15. Data analysis: Make your data file
Export your data from Qualtrics to SPSS:

16. Data analysis: Make your data file
Data and variable view in SPSS
1=male
2=female
Age in numbers
1=School A
2=School B

17. Data analysis: Make your data file
Make labels (=questions)
Fill in the values (=answer options)
Data analysis: Make your data file
Data and variable view in SPSS
1=male; 2=female
Age in numbers (no value)
1=School A; 2=School B

18. Data analysis: Make your data file
Data and variable view in SPSS

19. Data analysis: Make your data file
Clean your data:
Make your data anonymous (delete names, ID‘s)
Complete?
Time duration realistic?
Only include the variables you need for your analysis in your final data file

20. Data analysis: Make your data file
Re-coding of variables
Translate open answers into quantitative codes (manually)

21. Data analysis: Make your data file
Re-coding of variables
Re-coding of “smartphone ownership“(manually)
So, better option: Answering categories from the start!
1= 0-1 yrs
2= 1-2 yrs
3= 2-3 yrs
etc,.

22. Data analysis: Make your data file
Re-coding of variables
Re-coding of “age in years“ into “age categories“ (automatically)

23. Data analysis: Make your data file
Re-coding of variables
Re-coding of “age in years“ into “age categories“ (automatically)

24. Data analysis: Make your data file
Re-coding of variables
Re-coding of “age in years“ into “age categories“ (automatically)

25. Data analysis: Make your data file
Re-coding of variables
Re-coding negative item (automatically)
Emotional Appeal
I have a good feeling about the company.
I admire and respect the company.
I do not trust this company.
Products and Services
Stands behind its products and services.
Develops innovative products and services.
Offers low quality products and services.
Offers products and services that are a good value for money.
Vision and Leadership
Has excellent leadership.
Has a clear vision for its future.
Recognizes and takes advantage of market opportunities.
Workplace and Environment
Is well managed.
Looks like a bad company to work for.
Looks like a company that would have good employees.
Social and Environmental Responsibility
Supports good causes.
Is an environmentally responsible company.
Maintains high standards in the way it treats people.
Financial Performance
Has a strong record on profitability.
Looks like a low risk investment.
Tends to outperform its competitors.
Looks like a company with strong prospects for future growth.

26. Data analysis: Make your data file
SPSS discriminates between:
Scale
Nominal
Ordinal measurement

27. Data analysis: Make your data file
Scale: ordered categories, has a „true zero“ point e.g. age (if open question)

28. Data analysis: Make your data file
Nominal: e.g. region, Zip-code, operating system, gender, yes/no (numbers that are simply used as identifiers)

29. Data analysis: Make your data file
Ordinal: ranking (e.g. Likert Scale)

30. Data analysis: Missing value

31. Data analysis: Check reliability of variables
Gives your measurement consistent results? (internal consistency reliability)
 Calculate Cronbachs Alpha

32. Data analysis: Check reliability of variables

33. Data analysis: Check reliability of variables

34. Data analysis: Check reliability of variables
Cronbach‘s Alpha for the construct Perceived Security (PS) is .72
In the original scale .82
Look at „if item deleted“
Cronbach's alpha
Internal consistency
α ≥ 0.9
Excellent
0.9 > α ≥ 0.8
Good
0.8 > α ≥ 0.7
Acceptable
0.7 > α ≥ 0.6
Questionable
0.6 > α ≥ 0.5
Poor
0.5 > α
Unacceptable

35. Data analysis: Check reliability of variables
Might consider to delete 4th item

36. Check Cronbach‘s alpha for all variables. Then: Make variables in spss

37. Data analysis: Making your variables

38. Data analysis: Making your variables

39. Data analysis: Making your variables

40. Dataset is finished, time for analysis: Calculate frequencies, Means, SD

41. Data analysis: Calculate frequencies

42. Data analysis: Calculate frequencies
Gender: 82.9 % male and 17.1 % female
Study program:
82.9 % Computer Science
8.6 % Electrical Engineering
etc. …
Describe all the characteristics of your sample that you consider as important to mention!
Be as specific as possible but do not report nonsense…

43. Data analysis: Calculate Means + SD

44. Data analysis: Calculate Means + SD
Age: (SD 6.14 yrs)
Average time s.o. has owned a smartphone (in months): (SD 36.15)
Average time s.o. uses a smartphone (in years): 6.66 (SD 3.01)
etc. .
Standard Deviation = amount of variation within a given data set.
A low standard deviation indicates that the data points tend to be close to the mean (also called the expected value) of the set, while a high standard deviation indicates that the data points are spread out over a wider range of values.

45. Data analysis: Calculate Means + SD

46. Data analysis: Calculate Means + SD

47. Sub groups in your sample. Or: different conditions
Sub groups in your sample? Or: different conditions? Compare the answers of groups of respondents

48. Data analysis: Compare Means
Example:
To what extent do females score differently on perceived security compared to males?

49. Data analysis: Compare Means
Or: condition (high tech campaign vs green campaign)

50. Data analysis: Compare Means
The difference between gender (female vs male) and the mean score on Perceived Security is not statistically significant (p = .642)
(so there is no difference!)
We usually consider results as statistically significant at a threshold of .05 (or below) (=95% certainty)

51. Data analysis: Compare Means
Fundamentalists = are most protective and concerned about their privacy
Pragmatists = weigh the potential pros and cons of sharing personal data, but are less concerned than fundamentalists
Perceived Security of the app store the individual usually uses, measured on a Likert Scale going from 1 = completely disagree to 7 = completely agree
Conclusion: Fundamentalists perceive the security of the app store they usually use as significantly (p<.05) less secure (M=3.71) than pragmatists (M=4.83).

52. Data analysis And much more tests… Depends on your RQ‘s
Read Andy Fields‘ Book
Have a look at
Have a look at
Go to methodology shop (methodologie winkel BMS)
Ask the supervisor team if you need help
[Fie2009] Field, A. (2009). Discovering statistics using SPSS. LA: Sage.

53. Now: Self-study: developing/finalizing your questionnaire
developing an analysis strategy
working on your study design or experimental setting/manipulation
etc…
What you eventually will be working on depends on the status of your project.