Sara Bowman Center for Open Science | Promoting, Supporting, and Incentivizing Openness in Scientific Research

Mission: Improve openness, integrity, and reproducibility of scientific research

Technology to enable change Training to enact change Incentives to embrace change

InfrastructureMetascience Community

Transparency and Openness Promotion (TOP) Guidelines

TOP Design Principles Low barrier to entry Modular Agnostic to discipline

TOP Guidelines 1.Data citation 2.Design transparency 3.Research materials transparency 4.Data transparency 5.Analytic methods (code) transparency 6.Preregistration of studies 7.Preregistration of analysis plans 8.Replication

Data Transparency: Level 1 Summary: Article states whether data are available, and, if so, where to access them The policy of the ___________ is to publish papers where authors indicate whether the data, methods used in the analysis, and materials used to conduct the research will be made available to any researcher for purposes of reproducing the results or replicating the procedure. Authors must, in acknowledgments or the first footnote, indicate if they will or will not make their data, analytic methods, and study materials available to other researchers. If an author agrees to make materials available, the author must specify where that material will be available.

Data Transparency: Level 2 Data must be posted to a trusted repository. Exceptions must be identified at article submission. The policy of the ___________ is to publish papers only if the data used to conduct the research are clearly and precisely documented and are maximally available to any researcher for purposes of reproducing the results or replicating the procedure. Details of: What must be shared Legal and Ethical Exceptions – Disclosure at onset of review Using trusted repositories

Data Transparency: Level 3 Data must be posted to a trusted repository, and reported analyses will be reproduced independently prior to publication.... are maximally available to any researcher for purposes of reproducing the results or replicating the procedure. All materials supporting the claims made by the author must be made available to the journal prior to publication. The journal, or an entity acting on behalf of the journal, will verify that the findings are replicable using the author’s data and methods of analysis. Failure to replicate at this stage may result in the paper not being published.

500+ journals 45+ Organizations TOP Signatories Journals, Organizations, Funders

Signals: Making Behaviors Visible Promotes Adoption

Case Study: Psychological Science 2014

Data Availability in Psychological Science

Metascience

Appearing in Science August 28

Infrastructure

Open Science Framework

Open Science Framework

Collaboration Documentation Archiving

Version Control

Merges Public-Private Workflows

Incentives for Openness

Persistent Citable Identifiers

Registration

Connects Services Researchers Use

OSF

What are your barriers?

Thank you! Questions? Slides: Upcoming OSF Webinars: tinyurl.com/OSF101

Norms Communality Open sharing Universalism Evaluate research on own merit Disinterestedness Motivated by knowledge and discovery Organized skepticism Consider all new evidence, even against one’s prior work Quality Counternorms Secrecy Closed Particularlism Evaluate research by reputation Self-interestedness Treat science as a competition Organized dogmatism Invest career promoting one’s own theories, findings Quantity

Anderson, Martinson, & DeVries, 2007

Prereg Prize Endorse TOP Guidelines Badges for Open Practices Registered Reports

Design Collect & Analyze Report Publish PEER REVIEW PEER REVIEW

Registered Reports

Who Publishes Registered Reports? (just to name a few) See the full list and compare features: osf.io/8mpji