1. Shifting the research culture toward openness and reproducibility
Brian Nosek
University of Virginia -- Center for Open Science

2. Incentives for individual success are focused on getting it published, not getting it right
Nosek, Spies, & Motyl, 2012

3. Norms Counternorms Communality Universalism Disinterestedness
Open sharing
Universalism
Evaluate research on own merit
Disinterestedness
Motivated by knowledge and discovery
Organized skepticism
Consider all new evidence, even against one’s prior work
Quality
Secrecy
Closed
Particularlism
Evaluate research by reputation
Self-interestedness
Treat science as a competition
Organized dogmatism
Invest career promoting one’s own theories, findings
Quantity
Communality – open sharing with colleagues; Secrecy
Universalism – research evaluated only on its merit; Particularism – research evaluated by reputation/past productivity
Disinterestedness – scientists motivated by knowledge and discovery, not by personal gain; self-interestedness – treat science as a competition with other scientists
Organized skepticism – consider all new evidence, theory, data, even if it contradicts one’s prior work/point-of-view; organized dogmatism – invest career in promoting one’s own most important findings, theories, innovations
Quality – seek quality contributions; Quantity – seek high volume
Anderson, Martinson, & DeVries, 2007

4. Anderson, Martinson, & DeVries, 2007

5. What is good for me, the scientist versus What is good for science
Psychological COI
What is good for me, the scientist
versus
What is good for science

6. Signals: Making Behaviors Visible Promotes Adoption
Badges
Open Data
Open Materials
Preregistration
Psychological Science (Jan 2014)
Kidwell et al., 2016, PLOS Biology

7. 40%
30%
% Articles reporting that data was available
20%
10% 0%

8. Options with Badges Adopt! Collaborate on Randomized Trial
100 journals
Randomly assign to offer badges or not
Assess after a year

10. TOP Guidelines Data citation Design transparency
Research materials transparency
Data transparency
Analytic methods (code) transparency
Preregistration of studies
Preregistration of analysis plans
Replication

11. Data sharing 1 2 3
Article states whether data are available, and, if so, where to access them
Data must be posted to a trusted repository. Exceptions must be identified at article submission.
Data must be posted to a trusted repository, and reported analyses will be reproduced independently prior to publication.

12. Some TOP Signatory Organizations
AAAS/Science
American Academy of Neurology
American Geophysical Union
American Heart Association
American Meterological Society
American Society for Cell Biology
Association for Psychological Science
Association for Research in Personality
Association of Research Libraries
Behavioral Science and Policy Association
BioMed Central
Committee on Publication Ethics
Electrochemical Society
Frontiers
MDPI
PeerJ
Pensoft Publishers
Public Library of Science
The Royal Society
Springer Nature
Society for Personality and Social Psychology
Society for a Science of Clinical Psychology
Ubiquity Press
Wiley

13. Options with TOP Become a signatory!
Collaborate on education/adoption campaign with Editors
Randomized trial

14. Context of Justification
Confirmation
Data independent
Hypothesis testing
Context of Discovery Exploration Data contingent Hypothesis generating
p-values interpretable
PREREGISTRATION
p-values NOT interpretable
Presenting exploratory as confirmatory increases publishability of results at the cost of credibility of results

15. Preregistration Challenge http://cos.io/prereg/

16. Options with Prereg Challenge
Make more Elsevier journals eligible
Co-marketing to promote engagement with Challenge at eligible journals

17. Registered Reports http://cos.io/rr
Design
Collect & Analyze
Report
Publish
PEER
REVIEW
Review of intro and methods prior to data collection; published regardless of outcome
Beauty vs. accuracy of reporting
Publishing negative results
Conducting replications
Peer review focuses on quality of methods
Committee Chair: Chris Chambers

18. Options with Registered Reports
Adopt
Special issues as pilots
Randomized trial (grant application pending)

19. The Kindergartener’s Guide to Improving Research

20. 1. Show your work
2. Share

22. OpenSesame

23. OpenSesame

24. Will be added soon:
And interest from 50+ research universities and organizations

25. What can you do? OSF: http://osf.io/
Prereg Challenge:
Training:
Adopt TOP Guidelines:
Hackathon July Community meeting: July  
Themes: 
1) Pairing automatic enhancement with expert curation and the creation of tools to support these efforts
2) Pedagogy to develop expert curation of local data and technical skills to get SHARE data into local services (and then give back to SHARE)
3) Accessing (meta)data across the research workflow
See below:
I hope you'll consider attending SHARE's 2016 Community Meeting during the week of July 11, 2016 in Charlottesville, VA at the Center for Open Science (COS). If you're unfamiliar with SHARE or the free, open dataset we are creating, you can read more at share-research.org. 
The meeting will include a hackathon and a working meeting and you are welcome to register for one or both of these components. We welcome a diversity of skills, skill levels, backgrounds, and interests at the hackathon. This diversity is not only welcome, but will result in a better, more impactful event. Please do consider attending both events, whether or not you are an (experienced) programmer.
There is no registration fee for this meeting, but we are asking participants to cover their own travel and hotel costs in Charlottesville. We have a room block at the Omni Charlottesville with a rate of $149/night if booked by May 27. You can make your reservation by calling the Omni directly and mentioning the Center for Open Science. We do have a limited budget for travel support if needed.
Please fill out the registration form by Friday, April 15:
Thank you and please let me know if you have questions or concerns.
These slides are shared at:
[take a picture] or