1. Reproducibility Project: Psychology A Discussion
Nina Strohminger
Duke University

2. Why we are here As scientists, we want to know what is true.
Systematic replications are a tool of scientific progress:
How reliable is the data published in journals?
What practices would lead to a higher level of replicability?

3. Basic Findings
Approximately 2/3rds of studies did not replicate using a p-value cutoff
(Cancer Biology: 75–89%)
When combined in meta- analysis with original studies, 1/3rd no longer have sufficient evidence for existing

4. Basic Findings
No original study with a p > replicated

5. Basic Findings
Almost all replications had a smaller effect size than originals

6. Explanations for replication failures
Original studies:
Emphasis on innovation
Emphasis on positive results
Low power
Flexibility in data analysis/collection
Focus on clean and tidy results
Replication studies:
False negatives
Sampling error
Low fidelity

7. Moderators Moderators will give us a handle on this mystery
Preliminary evidence from replicability by journal
Design practice
Statistical practices
Journal protocol

8. Caveat
A replication attempt is no more gospel than the original. The necessity of repeated replications and meta-analyses

9. Implications As consumers of psychological science:
Published results are not gospel
Skepticism, particularly for large effect sizes from small samples, marginal p-values
As producers of psychological science:
Adopt incentive structures that reduce research bias, such as pre-registration
Publish null results online
Be skeptical of your own results
ABR (Always Be Replicating)

10. Using science to fix science
Reproducibility is not just about psychology, but about repairing a weakness in the current scientific system Current results point to an undesirable rate of replication, but the existence of the project is itself a source of hope