1. Transitioning to a more open psychological science Creating a customized workflow to maximize productivity, security, and transparency
Good afternoon everyone! My name is Kaitlyn Werner and I am an ambassador for the Center for Open Science. Today I am excited to be here to talk to you about how to transition to a more open psychological science. As I am sure many of you know, this has been quite a hot topic within our field over the past few years, and so my goal here today is to give you a fresh perspective on what the Center for Open Science has to offer, as well as some ways in which I have personally adopted open science research practices to increase my own productivity.
Kaitlyn M. Werner, MSc.
PhD Candidate, Carleton University
Ambassador, Center for Open Science

2. Outline Open Science Practices What is the Center for Open Science?
OSF Tutorial
(brief)
Open Science Practices
(registration)
Security and Transparency

3. Openness. Integrity. Reproducibility.
Before we jump into the specifics, I thought I’d take a quick moment to talk about what the Center for Open Science actually is. At its core, our mission is to provide the necessary tools and support to help researchers maintain an efficient workflow, while at the same time increasing openness, and enhancing integrity, and reproducibility. You may have seen a lot of information circulating in the literature and on social media about the need to “fix” our field, and while that is part of what COS does, as an ambassador part of my job is to focus on researchers’ individual needs and help figure out strategies that support that COS mission while at the same time benefiting individual researchers.
Openness. Integrity. Reproducibility.

4. What does COS offer? Infrastructure Community Metascience
COS has developed a variety of software tools and workflows based on our free Open Science Framework to help researchers manage and archive their research, privately or publicly.
That said, the Center for Open Science has three core elements – Infrastructure, Community, and Meta Science.
Community
COS helps connect and build open science communities of researchers, their affiliated institutions, their funders, and the publishers of research outcomes.
Metascience
COS supports and conducts research on scientific practices. Our team members are thought leaders on open science and our tools serve as critical research platforms.

5. Tutorial
A very brief, but incredibly simple tutorial on how to create a project and descriptions of various pre-registration options

6. The Open Science Framework
osf.io
Project management
Easily share files
Document project changes
Project analytics
Archive data
Facilitate collaboration
Research efficiency
Registration

7. The Open Science Framework
osf.io

8. The Open Science Framework
osf.io/r6yhz
You can think about a project as being a filing cabinet for your research. It starts off completely empty and it’s up to you to fill it out and organize the files in a way that works best for you. For example, you can have one project that houses the information for a series of studies that go into a single manuscript, or you can create a project for individual studies and link them later on.
In OSF terms, you can create these folders by creating a new component. These components are completely customizable and are stored in a similar fashion as dropbox. Once a component is created, you can drag and drop and move it around, and you can edit it however you want.

9. The Open Science Framework
osf.io/r6yhz

10. The Open Science Framework
osf.io/r6yhz

11. Connect with services you already use!
…and more!

12. Open Science Practices
What exactly can you do to increase transparency?

13. What are open science practices?
Practices that promote transparent research from start to finish
Explain p-hacking; not intentional
Explain decision rules (e.g., autonomous and controlled – data driven with a cutoff)
100% p-hacking free
Here’s how you can replicate our work
Here, check our numbers
What does this really mean?

14. The Open Science Framework
osf.io/r6yhz

15. Registration with the Open Science Framework
Registration creates a frozen version of the project that can never be edited or deleted but can be withdrawn. Your original project remains editable but will now have the registration linked to it.
Open-Ended Registration
AsPredicted Pre-registration
OSF Standard Registration
Special Registration Types
Provide a narrative summary of what is contained in the registration, or how it differs from prior registrations.
Eight (8) questions based on the criteria of aspredicted.org
Data collected? (Y/N)
Question or hypothesis
Describe key variables
Number of participants
Primary analyses plan
Secondary analyses
Sample size determination
Additional (e.g., exclusions)
Basic registration; Similar to open-ended registration with two extra questions (3 total)
Data collected? (Y/N)
Looked at data? (Y/N)
Additional comments
Pre-registration Challenge
Replication recipe (pre and post) (Brandt et al., 2013)
Pre-registration in social psychology (van’t Veer & Giner-Sorolla, 2013)
“Soft pre-registration”

16. Documenting Your Research Workflow
“Official” Pre-Registration
“Soft” Pre-Registration
100% customizable
Can include relevant background details and variable lists for greater accessibility
Can be kept private indefinitely – researcher retains full control of who has access while still keeping full documentation of the research process
Documentation is more fluid (just like the research process!)
Can never be edited or deleted
Automatically becomes public
Embargo up to 4 years
Withdrawing a registration removes the content, but leaves a log showing when it was created and when it was withdrawn
Limited to the templates provided (although some are really broad)

17. Documenting Your Workflow
Pull in example
Background
List a few bullet points explaining your idea; Include key references
Research Questions
What are the main questions that your research is trying to answer?
Hypotheses
Operationalize your research questions and list your specific predictions
Exploratory Questions
List any questions that novel, you are unsure of, and/or don’t have any specific predictions for
Method
Describe your method in as much detail as possible; If using secondary data, describe the sample and variables of interest
Analytic Strategy
Describe exclusion criteria, cutoffs, etc. Then, describe exactly how you plan to analyze your hypotheses. Exploratory analyses optional

18. How to maximize transparency while keeping your information secure
Security
How to maximize transparency while keeping your information secure

19. So what can we do in these scenarios?
Common Concerns
Large datasets
Sensitive or identifying information
Do not have ethics approval (e.g., not included in consent form) …
So what can we do in these scenarios?

20. Common Concerns – Potential Solutions
Only publish variables relevant to the study (Simine Vazire)
Can’t publish your data, materials, etc.? Then DON’T.
Provide a rationale for why this is the case
However, when possible, share data when requested individually (not sharing anything EVER is never a good idea)
e.g., Protocol at Psychological Science:
There are circumstances in which it is not possible or advisable to share any or all data, materials, or a research plan publicly. For example, there are cases in which sharing participants’ data could violate confidentiality. If you would like your article to include an explanation of such circumstances and/or provide links to any data or materials you have made available—even if not under conditions eligible to earn a badge—you may write an alternative note that will be published in the Open Practices note in the article. Please check this box if you would like your article to include an alternative note and provide the text of the note below:
Source: Open Sciences Disclosure Form (

21. The Open Science Framework – Privacy Controls
osf.io/r6yhz
Can create a link for people to access your project without editing (i.e., it’s read only)
Can create an anonymized version for peer review – just make sure all documents are also anonymous!

22. Why Be More Transparent?
Organization
Having a detailed plan helps stay organized, which helps with creating conference presentations, facilitates collaboration (both public and private), and helps reduce errors made when juggling multiple projects.
As of now, most open practices for journals are suggestions, and therefore completely voluntary
Exceptions – PSPB and SPPS requires you to submit materials with your manuscript; Collabra & PPS replications require complete transparency, including data sharing
Journals
Journals in psychology (and abroad) are beginning to request materials and more detailed protocols for submitted manuscripts
Psychological Science
Personality and Social Psychology Bulletin (PSPB)
Social Psychological and Personality Science (SPPS)
Journal of Experimental Social Psychology (JESP)
New journals aimed at publishing transparent research that are methodologically and statistically rigorous, while putting less emphasis on “sexy” findings
Collabra
Perspectives on Psychological Science

23. Take Home Messages Flexibility Security Be honest
Do what works best for you! Don’t feel as though you are tied to a specific protocol, analysis, etc. – just keep track of and report any changes that you make. Journals are pretty cool about it, too.
Security
Open science does not mean no security! There are ways to adapt your open science practices to protect sensitive data and/or cater to your preferences in data sharing. Let it all out or hold a tighter ship – it’s up to you!
Be honest
No matter what you do or what your preferences are, just be honest and transparent about it! Online supplements (either through a specific journal or OSF) are your friend.

24. kaitlynwerner.wix.com/research
Additional Resources
Center for Open Science
OSF Preprints
Pre-registration challenge ($1000 prize!)
Collaborative replications and education project (CREP)
Statistical consulting
Training workshops and webinars
Journal badges (e.g., Psych Sci.)
Berkeley Initiative for Transparency in the Social Sciences (BITSS)
Annual Institute – June 7-9, 2017
SSMART Grants
Contact Information
@kaitlynmwerner
osf.io/ftk25
kaitlynwerner.wix.com/research
Open Science Framework: Meeting Researchers Where They Are
Megan Potterbusch, COS Ambassador |