1. Session 4 Open Workflow: OSF and Pre-registration exercise
Good afternoon everyone! My name is Kaitlyn Werner and I am an ambassador for the Center for Open Science. Today I am excited to be here to talk to you about some of the services that the Center for Open Science has to offer, more specifically with respect to the Open Science Framework. As I am sure you all know, this idea of transitioning to a more open psychological science has been quite a hot topic within many fields over the past few years, and so my goal here today is to show you how the Open Science Framework can help you become more productive and efficient researchers.

2. RT2 Roadmap Design Conduct Dissemination Archiving Motivating Issues
Researchers degrees of freedom
Scientific misconduct
Publication bias
Failure to replicate
To achieve
Open materials, data, code, & access
Transparent reporting & disclosure
Reproducible & replicable results
Cumulative meta-analyses
Organized Workflow and File Management (OSF, Github)
Design
Pre-Registration
Pre-Analysis Plans
Power Planning
Conduct
Data Management
Version Control
Open Notebooks (Jupyter/Docker)
Dissemination
Transparent Reporting & Disclosure
Preprints
Open Access
Archiving
Data Repositories
Dynamic Documents

3. Kaitlyn Werner, Carleton University: I work primarily with multi-level longitudinal data in social, personality, and health psychology.

4. Outline What is the Center for Open Science? OSF Tutorial
Open Science Practices
(registration)
FAQs

5. Openness. Integrity. Reproducibility.

6. What does COS offer? Infrastructure Community Metascience
COS has developed a variety of software tools and workflows based on our free Open Science Framework to help researchers manage and archive their research, privately or publicly.
Community
COS helps connect and build open science communities of researchers, their affiliated institutions, their funders, and the publishers of research outcomes.
Metascience
COS supports and conducts research on scientific practices. Our team members are thought leaders on open science and our tools serve as critical research platforms.

7. The Open Science Framework
osf.io
Project management
Easily share files
Document project changes
Project analytics
Archive data
Facilitate collaboration
Research efficiency
Registration

8. Tutorial
A brief, but simple tutorial on how to create a project, and manage files and components

9. The Open Science Framework
osf.io

10. The Open Science Framework
osf.io

11. OSF: Creating a Project
osf.io

12. OSF: Organizing a Project
osf.io/r6yhz

13. OSF: Uploading Documents
osf.io/r6yhz

14. OSF: Uploading Documents
osf.io/r6yhz

15. The Open Science Framework – Privacy Controls
osf.io/r6yhz
Can create a link for people to access your project without editing (i.e., it’s read only)
Can create an anonymized version for peer review – just make sure all documents are also anonymous!

16. OSF: General Navigation
osf.io

17. Connect with services you already use!
…and more!

18. Open Science Practices
What exactly can you do to increase transparency? A guide to various registration formats on OSF

19. What are open science practices?
Practices that promote transparent research from start to finish
100% p-hacking free
Here’s how you can replicate our work
Here, check our numbers
What does this really mean?

20. Registration with the Open Science Framework
Registration creates a frozen version of the project that can never be edited or deleted but can be withdrawn. Your original project remains editable but will now have the registration linked to it.
Open-Ended Registration
AsPredicted Pre-registration
OSF Standard Registration
Special Registration Types
Provide a narrative summary of what is contained in the registration, or how it differs from prior registrations.
Eight (8) questions based on the criteria of aspredicted.org
Data collected? (Y/N)
Question or hypothesis
Describe key variables
Number of participants
Primary analyses plan
Secondary analyses
Sample size determination
Additional (e.g., exclusions)
Basic registration; Similar to open-ended registration with two extra questions (3 total)
Data collected? (Y/N)
Looked at data? (Y/N)
Additional comments
Pre-registration Challenge
Replication recipe (pre and post) (Brandt et al., 2013)
Pre-registration in social psychology (van’t Veer & Giner-Sorolla, 2013)
Dynamic Document: Tracking the Research Process and its Development

21. Registration Options
osf.io/r6yhz

22. Register: Open-Ended Registration
osf.io/r6yhz

23. Register: As Predicted
osf.io/r6yhz

24. Register: Open-Ended Registration
osf.io/r6yhz

25. Pre-Registration Challenge
Pull in example

26. EXAMPLE: As Predicted Registration
osf.io/r6yhz

27. Dynamic Documents: Documenting Your Workflow As it Happens
Create a word document with any updates as they happen – “lab notebook”
Use your registration as a template and document changes using bullet points, another colour, etc.
The more detailed you are in the registration, the more time you save later
Report any deviations from your registration in your manuscript – refer to dynamic document
Pull in example
Background
List a few bullet points explaining your idea; Include key references
Research Questions
What are the main questions that your research is trying to answer?
Hypotheses
Operationalize your research questions and list your specific predictions
Method
Describe your method in as much
detail as possible; If using secondary data, describe the sample and variables of interest
Exploratory Questions
List any questions that novel, you are unsure of, and/or don’t have any specific predictions for
Analytic Strategy
Describe exclusion criteria, cutoffs, etc. Then, describe exactly how you plan to analyze your hypotheses. Exploratory analyses optional

28. FAQ
Demystifying common misconceptions about open science and pre-registration

29. Common Concerns Do I have to make my data, materials, etc. public?
No. Your registration can be embargoed for up to 4 years, but components can be kept private and shared among (assigned) project contributors
What if I have sensitive data? (e.g., special populations, linked data)
Consider sharing the non-sensitive portions of the data (e.g., demographics) with a list of missing variables made upon request
What if I have a large dataset that I’m not finished using?
Consider sharing the variables used in the analyses along with a detailed, complete list of all study variables

30. Common Concerns
What if I can’t share my data, materials, etc.? (e.g., not included in original consent form)
Then DON’T. Provide a rationale for why this is the case
However, when possible, share data when requested individually (not sharing anything EVER is never a good idea)
e.g., Protocol at Psychological Science:
There are circumstances in which it is not possible or advisable to share any or all data, materials, or a research plan publicly. For example, there are cases in which sharing participants’ data could violate confidentiality. If you would like your article to include an explanation of such circumstances and/or provide links to any data or materials you have made available—even if not under conditions eligible to earn a badge—you may write an alternative note that will be published in the Open Practices note in the article. Please check this box if you would like your article to include an alternative note and provide the text of the note below:

31. Take Home Messages Flexibility Security Be honest
Do what works best for you! Don’t feel as though you are tied to a specific protocol, analysis, etc. – just keep track of and report any changes that you make. Journals are pretty cool about it, too.
Security
Open science does not mean no security! There are ways to adapt your open science practices to protect sensitive data and/or cater to your preferences in data sharing. Let it all out or hold a tighter ship – it’s up to you! Just don’t keep anything locked away in your file drawer
Be honest
No matter what you do or what your preferences are, just be honest and transparent about it! Online supplements (either through a specific journal or OSF) are your friend.

32. kaitlynwerner.wix.com/research
THANK YOU!
@kmwerner
osf.io/ftk25
kaitlynwerner.wix.com/research
carleton.ca/goallab