1. Experimental Clinical Psychology Session I
Eiko Fried
Department of Clinical Psychology
Leiden University

2. This course is scary!

3. This course is scary! “Statistics is scary” “Methodology is scary”
”You are scary”
“I wanna go home”

6. This course is not relevant for me!

7. This course is not relevant for me!
“Statistics is super boring”
“Methodology is lame”
“You are lame”
“I wanna go home”

9. This course is not relevant for me!

10. Ground rules Assignments Every assignments = 80 points
Please read the assignment very carefully
Assignment deadline Monday 5pm; else -10% points
Grades by Thursday 5pm to encourage discussions
OK to collaborate on assignment, but write own answers
Plagiarism checks
Keep up with reading, only 1 week between last assignment & exam
Every assignments = 80 points
5 assignments = 400 points
Final grade; 40% assignments, 15% participation, 45% exam

11. Ground rules New Kazdin book; corrections to Kazdin
Bring printout (laptops?)
Go to lectures, really go to seminars
Attendance & participation
English

12. Introduction round
Name, background, BA, expectation of course, fun fact

15. Internal validity

16. History
70%

17. Maturation
30%

18. Testing
10%

19. Instrumentation
10%

20. Statistical regression
Regression to the mean
10%

21. Selection bias
20%

22. Attrition
30%

23. Diffusion/imitation 50%
“Problematic. If the prayers are considered the treatment, it is possible that some patients from the control group also received prayers if they happen to have the same first name with one of the 500+ patients from the experimental group. The chance is not low given that some English names are fairly common.”

24. Construct validity

25. Attention/contact
30%

26. Narrow stimulus sampling
10%

27. Experimenter expectancies
10%

28. Der kluge Hans (clever Hans)

29. Der kluge Hans (clever Hans)

30. Demand characteristics
20%

31. Low power
40%

32. Brief excurse to NHST

33. Sample & population
URL |

35. Basics
Alpha, type I error: H0 is true, probability to falsely reject H0. False positive rate.
Beta, type II error: H1 is true, probability to falsely reject H1. False negative rate.
The larger Alpha, the smaller Beta, and vice versa
Power: 1 - Beta. High power: very low probability to falsely reject H1, i.e. high probability to correctly identify H1.

36. Basics
Alpha, type I error: H0 is true, probability to falsely reject H0. False positive rate.
Beta, type II error: H1 is true, probability to falsely reject H1. False negative rate.
The larger Alpha, the smaller Beta, and vice versa
Power: 1 - Beta. High power: very low probability to falsely reject H1, i.e. high probability to correctly identify H1.
P-value of 0.03: probability to obtain this finding by chance if you run the same test many times.
Significant p-value does not confirm H1, it rejects H0

37. Power calculation Power: function of Sample size: function of
Alpha
Sample size
Effect size (m1 – m2 / SD )
Sample size: function of
Power
Why and how is SD relevant?

38. Increase power How can you increase the power of your own study?
Larger mean difference
Repeated measures reduces error term
Decrease variability / error
What does it mean if a study has a power of 0.99?
1% probability to falsely reject H1, i.e. 99% probability to correctly identify H1

39. Examples
Study in which you want a strict alpha, i.e. you want to minimize the risk for …
false positives
Study in which you want a strict beta, i.e. you want to minimize the risk for …
false negatives

40. Trends

42. Subject heterogeneity
30%

43. Variability in procedures
40%

44. Unreliability of measures
50%

45. Restricted range of measures
30%

46. Errors in data recording, analysis, and reporting
50%

47. Multiple comparisons

49. Multiple comparisons 0%

50. Multiple tests

51. Multiple comparisons Probability of chance findings:
1-(1-0.05)^1 = 0.05; p-value; number of tests
1-(1-0.05)^3 = 0.14
1-( )^34 = 0.16
1-(1-0.05)^ ( )^34 = 0.30

52. Misreading or misinterpreting the data analysis
50%

53. Why did we choose this paper?
Would you recommend praying?

54. But the trial didn’t do any harm !
URL |

55. What really happened …

56. Threats to external validity
Sample characteristics
Narrow stimulus sampling
Reactivity to experiment
Reactivity of assessment
Test sensitization
Multiple treatment interference
Novelty effects
Generality across measures

57. DOI | http://dx.doi.org/10.1016/j.jad.2016.10.019

58. Managing threats to external validity

59. Managing threats to external validity

60. Bayesian framework

61. Bayes

62. Bayes NHST tests tell us whether we can reject H0 given the data
Bayesian analysis compares H0 and H1 and says which one is more likely given the data
Bayes factor: probability of observed data given H1 / probability of observed data given H2
P(D|H1) / P(D|H1)
BF=10 indicates data are 10x more likely under H1 than H0
Prior probability  data collection  posterior probability

64. Lower alpha vs justify your alpha
DOI | /s x
DOI | /s z

66. Break