Open Science Framework: Supporting the research worflow Brian Nosek University of Virginia

Improving science with openness Top-down – fast, but narrow – Funders: levers only for grantee projects – Journals: only at point of publication Bottom-up – slow, but comprehensive – Researcher’s entire workflow part of the cumulative record – Leverage norms rather than requirements

Opening the research workflow Registration is one piece

Open Science Framework (OSF) Web app and tools for collaborating, documenting, archiving, sharing, and registering science Respect workflow Robust to different workflows Integrate private and public workflows Incentivize openness Support top-down and bottom-up incentivization

Registration Existing solutions are: – Discipline specific – Appended to the workflow – Constrained to a point in time – Content restrictive OSF registration is none of those. Instead: – Templating – Integrated in the workflow – Any time, any frequency – Registers everything

OSF Demo Collaborator and Lead Developer: Jeff Spies Open Science Framework

Open Science Framework (OSF) For collaboration, documentation, archiving, sharing, registration Integrates components of workflow Replaces ad hoc archiving with shared solution Merges private and public workflows Integrates top-down and bottom-up solutions Breaks disciplinary silos Connects infrastructure

Set up your OSF account

The Sell How can you get credit for doing good science? Stick versus Carrot

TODO Metadata System Commenting Dropbox-like File Syncing Finer-Grained Collaborator Information and Access Control Exploring the Scientific/Collaborator Network Application Programming Interface (API)

Improving Science 1.Open data, materials, and workflow 2.Registration 3.Structured replication process 4.Shifting incentives – By journals, funders, and infrastructure

3. Structured replications Design – Peer-review prior to data collection – Publicly registered materials and analysis plan – Simultaneous replications on same methods Features – Incentives for doing replications and reporting well – Avoid researchers’ degrees of freedom – Minimize methodology and quality drift – Sharing and communication See Call for Proposals for Special Issue of Social PsychologyCall for Proposals for Special Issue of Social Psychology

4. Shifting Incentives Journals and funders require disclosure Journals credit openness and registration – Badges Publishing replications Gaining reputation with openness – Indicators of use; Functional citations

1. Openness Barriers – Extra work – Incentives Emerging infrastructure options – Many open data – Some open materials – Little open workflow – No integration

Collaboration Tools Materials search and acquisition Data Repository Registration Study Documenting Archiving Clinicaltrials.gov Dataverse file sharingFolders on desktop authors/Recreate