1. Meta-Analysis: Synthesizing evidence

2. Reminders
Exam #3 (next class): Not cumulative, standard 40- question multiple choice
Make-up LA and Paper rewrites (optional)
Make-up Exam (optional):
Short essays (60pts)
Pick 2 of 4: Broad topics relevant to an entire unit (e.g., clinical research) or across units (e.g., validity)
Short answer (100pts)
Pick 6 of 12: Representative narrower topics (e.g., key considerations in designing RCTs)

3. Take-Home Messages of the Course
Understand the value of research and experience a greater desire to engage in research in some capacity
Understand p-values
Understand the difference between a true “experiment” and a non-experiment
Understand the importance of effect size (r, d)

4. Take-Home Messages of the Course
Gain a greater tolerance for ambiguity
Improve scientific writing and statistical skills
Understand different methodologies as tools that can improve the quality of research
Gain practical skills/accomplishments to foster academic and career goals
Live a more science-driven life

5. Today’s Overview What does it mean to synthesize evidence?
Why would we want to synthesize evidence?
What is a meta-analysis?

6. Weighing and Synthesizing Evidence
Any problem in life can be informed by scientific evidence
Interpersonal dynamics, parenting, studying, worker satisfaction, psychotherapy, medical treatments, educational interventions, traffic safety, health
A so-called science-driven life would use scientific evidence to make key life decisions
Problem: Studies conflict (size/direction of effect,, confidence/p-value), and any particular study has strengths and weaknesses (the many reliabilities and validities)

7. Examples
If you’re willing to shock your world view, read these articles:
Djulbegovic, M., Beyth, R. J., Neuberger, M. M., Stoffs, T. L., Vieweg, J., Djulbegovic, B., & Dahm, P. (2010). Screening for prostate cancer: systematic review and meta-analysis of randomised controlled trials. BMJ, 341, c4543.
Moyer, V. A. (2012). Screening for prostate cancer: US Preventive Services Task Force recommendation statement. Annals of Internal Medicine, 157(2),
Nelson, H. D., Fu, R., Cantor, A., Pappas, M., Daeges, M., & Humphrey, L. (2016). Effectiveness of breast cancer screening: systematic review and meta-analysis to update the 2009 US Preventive Services Task Force recommendation. Annals of Internal Medicine, 164(4),

8. Weighing and Synthesizing Evidence
Literature review
Systematic literature review
Meta-analysis
Statistical approach for combining results from multiple studies
Typically requires a systematic literature review to avoid bias
Produces an average effect size (e.g., r, d, %)
Bigger N  More power  More precision (e.g., better p-value, narrower confidence intervals)

9. Levels of Evidence (high to low)
Meta-analysis of RCTs
Experimental studies and RCTs
Longitudinal observational studies and quasi-experimental studies
Cross-sectional observational studies
Single-subject designs
Case series
Case study
Note. Other aspects of study quality could move a particular study up or down the list

10. Examples Mike’s current meta-analysis (separate file)
Smith and Glass (1977). Meta-analysis of Psycho- therapy Outcomes.

11. Hauch et al. (2016). Does training improve the detection of deception?

12. Roberts et al. (2006). Patterns of mean-level change in personality traits across the life course.

18. Strengths and Limitations
Excellent for synthesizing evidence, providing a starting point in a literature review, guiding policy
Limitations
Concern about publication bias and other biases in the systematic review process
Concern about combining studies, samples, and interventions that are too heterogeneous