1. Disrupting Scholarly Communication
Brian Nosek
University of Virginia -- Center for Open Science

2. Metascience to encourage change Technology to enable change Training to enact change Incentives to embrace change
Improving scientific ecosystem

3. The State of Scholarly Communication
Origins and Operation
The research lifecycle – doing research, submitting to journal, ad hoc review, serial search through journals until arriving at one

4. Idea Design Collection Analysis Write Submit Publish
Review
Idea Design Collection Analysis Write Submit Publish
Review
Idea Design Collection Analysis Write Submit Publish
Review
Idea Design Collection Analysis Write Submit Publish
Review

5. What’s wrong?
Conflates actual aim (evaluation) with concrete indicator (publication)
Slow
Ineffective (reliability and validity)
Inaccessible (pay walls or file-drawer)
Inflexible (e.g., errors hard to correct)
Reduces reproducibility

6. Solution Decouple publication and evaluation
Foster open marketplace for scholarly communication innovation

7. Fix 1: Preprints

8. 721 days Benefits Faster More accessible More impact Risks
Idea Design Collection Analysis Write Submit Publish
Review
721 days
Review
Idea Design Collection Analysis Write Publish
Benefits
Faster
More accessible
More impact
Risks
No evaluation
Information overload

9. Fix 2: Versioning Corrections are fast and discoverable
Version history captures impact of evaluation
Idea Design Collection Analysis Write Publish Publish
Review

10. Fix 3: Diversify Evaluation
Pre- vs. Post-publication
Open vs. Closed
Editor-selected vs. Open Commenting
Review
Idea Design Collection Analysis Write Publish Publish

11. “journals” are not gatekeepers, they are evaluators and amplifiers
Fix 4: Article:Journal
Change 1:1 to 1:Many
“journals” are not gatekeepers, they are evaluators and amplifiers

12. Future CV
Axt, J., Nguyen, H., & Nosek, B. A. (2017, January 18). The Judgment Bias Task: A reliable, flexible method for assessing individual differences in social judgment biases. Retrieved from osf.io/bhzz6.
Evaluations: Accepted at Science; JPSP 92/100; Rated by Cog Measures, Rand Lab, EJ’s Faves
Review service aggregate: 89/100 (7 evaluations)
Community aggregate: 85/100 (108 evaluations)

13. Fix 5: Registered Reports++ http://cos.io/rr
Independent expertise throughout process
Clarity of confirmatory vs. exploratory
Solves file-drawer
Review
Review of intro and methods prior to data collection; published regardless of outcome
Beauty vs. accuracy of reporting
Publishing negative results
Conducting replications
Peer review focuses on quality of methods
Idea Design Collection Analysis Write Publish Publish

14. Marketplace of paper services

15. Services
Ecosystem
Data

16. OpenSesame

17. OpenSesame

18. Community
Interfaces
Interfaces
Services
Ecosystem
Data
OSF

25. Getting it done Open marketplace of evaluation services
Publishing, Versioning, Commenting, Review
Seed with demonstration cases
Foster diversification in evaluation practices
Continuous evaluation
33% Complete

26. Outcome
An open marketplace to accelerate innovation in scholarly communication, evaluation, and research

27. COS Strategic Plan https://osf. io/x2w9h These slides https://osf
COS Strategic Plan These slides Nosek & Bar-Anan (2012)
My general substantive interest in the gap between values and practices. The work that I am discussing today is a practical application of this interest to the gap between scientific values and practices. In particular, how can I best advance knowledge and my career at the same time?
Challenges I face when working to advance scientific knowledge and my career at the same time. And, how my scientific practices can be adapted to meet my scientific values.
Take a picture