1. A Framework for Managing and Sharing Research Workflow
Jeffrey Spies
Center for Open Science
University of Virginia

2. Non-profit tech company in Charlottesville, VA
Openness, integrity, and reproducibility
$25 since 2013
I’ll introduce you to the team as I tell you more

3. If I disconnect email jeff@cos
If I disconnect Please do not let me talk to myself for 40 minutes.

4. Mission Increase openness, integrity, and reproducibility of scientific research.

5. Problem The gap between scholarly values and practices.

6. Adelson, 1995

7. Local contract
Shadows have soft edges
x junctions
Adelson, 1995

8. Incentives for individual success are focused on getting it published, not getting it right.
Novel, positive,
clean stories with
confirmatory successful narratives
Now how science works--arrogant
Nosek, Spies, & Motyl, 2012

9. Key incentive = Publication
What is published?
What is not?
Novel results Positive results Clean results
Replications Negative/Nulls Mixed evidence
Arrogant

10. Norms
Counternorms
Communality
Open sharing
Secrecy
Closed

11. Norms Counternorms Communality Secrecy Particularlism Universalism
Open sharing
Universalism
Evaluate research on own merit
Secrecy
Closed
Particularlism
Evaluate research by reputation

12. Norms Counternorms Communality Secrecy Particularlism Universalism
Open sharing
Universalism
Evaluate research on own merit
Disinterestedness
Motivated by knowledge and discovery
Secrecy
Closed
Particularlism
Evaluate research by reputation
Self-interestedness
Treat science as a competition

13. Norms Counternorms Communality Secrecy Particularlism Universalism
Open sharing
Universalism
Evaluate research on own merit
Disinterestedness
Motivated by knowledge and discovery
Organized skepticism
Consider all new evidence, even against one’s prior work
Secrecy
Closed
Particularlism
Evaluate research by reputation
Self-interestedness
Treat science as a competition
Organized dogmatism
Invest career promoting one’s own theories, findings

14. Norms Counternorms Communality Secrecy Particularlism Universalism
Open sharing
Universalism
Evaluate research on own merit
Disinterestedness
Motivated by knowledge and discovery
Organized skepticism
Consider all new evidence, even against one’s prior work
Quality
Secrecy
Closed
Particularlism
Evaluate research by reputation
Self-interestedness
Treat science as a competition
Organized dogmatism
Invest career promoting one’s own theories, findings
Quantity

15. Anderson, Martinson, & DeVries, 2007

16. Anderson, Martinson, & DeVries, 2007

17. Anderson, Martinson, & DeVries, 2007

18. Problem The gap between scholarly values and practices.

19. Solution Openness.

20. Openness is a means to increase research quality and efficiency.

21. Solution Technology can operationalize and incentivize scholarly values.

22. Let experts be experts.

23. Openness is the solution; Workflow makes it practical.

24. Publish Report
Search/Discovery
Develop Idea
Design Study
Collect Data
Store Data
Analyze Data
Write Report

25. Publish Report
Search/Discovery
Develop Idea
Design Study
Collect Data
Store Data
Analyze Data
Write Report

26. Scholars should focus on scholarship.

27. OSF Publish Report Search/Discovery Develop Idea Design Study
Collect Data
Store Data
Analyze Data
Write Report
OSF

28. OSF Application Framework
Workflow
Authentication
Permissions
File Storage
File Rendering
Meta-database
Persistence
Integrations
Search
SHARE
osf.io
journals
registries
preprint servers
grants management
university systems

29. Free, login ->

30. Collaboration
Documentation
Archiving

31. Projects, Collaborators, sharing, Wikis
Provenance with logs
Curating content
Makes collaboration easier— based management is messy

32. Version Control

33. Provenance is important
Most people have some form
Cumbersome error prone
Automatic – easy backup

34. Connects Services Researchers Use Getting close to users
Meet them where they are
Don’t want to reinvent when we don’t have to
Flexible to many workflows

35. Drag and drop
Collaborative editor

36. OSF Now 9 Publish Report Search/Discovery Develop Idea Design Study
Collect Data
Store Data
Analyze Data
Write Report
OSF 9
Now

37. Publish Report
Search/Discovery
Develop Idea
Design Study
Collect Data
Store Data
Analyze Data
Write Report
OSF 9
OpenSesame
Soon
29 grants to develop open tools and services:

38. Coming soon

41. Let experts be experts.

42. OSF Application Framework
Workflow
Authentication
Permissions
File Storage
File Rendering
Meta-database
Persistence
Integrations
Search
SHARE
osf.io
journals
registries
preprint servers
grants management
university systems

44. Adam Summers aka fish guy

46. <menu>
<osf-search /> <osf-login />
</menu>
<osf-file>
<osf-file title />
<osf-file authors />
<osf-file date />
<osf-file renderer />
<osf-file analytics />
<osf-file comments />
<osf-file download />
<osf-file summary />
<osf-file tags />
</osf-file>

48. <menu>
<osf-search /> <osf-login />
</menu>
<osf-file>
<osf-file title />
<osf-file authors />
<osf-file date />
<osf-file renderer />
<osf-file analytics />
<osf-file comments />
<osf-file download />
<osf-file summary />
<osf-file tags />
</osf-file>

50. <menu>
<osf-search /> <osf-login />
</menu>
<osf-file>
<osf-file title />
<osf-file authors />
<osf-file date />
<osf-file renderer />
<osf-file analytics />
<osf-file comments />
<osf-file download />
<osf-file summary />
<osf-file tags />
</osf-file>

52. <menu>
<osf-search /> <osf-login />
</menu>
<osf-file>
<osf-file title />
<osf-file authors />
<osf-file date />
<osf-file renderer />
<osf-file analytics />
<osf-file comments />
<osf-file download />
<osf-file summary />
<osf-file tags />
</osf-file>

54. <menu>
<osf-search /> <osf-login />
</menu>
<osf-file>
<osf-file title />
<osf-file authors />
<osf-file date />
<osf-file renderer />
<osf-file analytics />
<osf-file comments />
<osf-file download />
<osf-file summary />
<osf-file tags />
</osf-file>

56. <menu>
<osf-search /> <osf-login />
</menu>
<osf-file>
<osf-file title />
<osf-file authors />
<osf-file date />
<osf-file renderer />
<osf-file analytics />
<osf-file comments />
<osf-file download />
<osf-file summary />
<osf-file tags />
</osf-file>

57. OSF Application Framework
Workflow
Authentication
Permissions
File Storage
File Rendering
Meta-database
Persistence
Integrations
Search
SHARE
osf.io
journals
registries
preprint servers
grants management
university systems

58. http://osf.io/prereg Authenticate Started a new prereg
Motor control and autism

61. Go further in the process, asks for scripts

62. Login, projects, users, metadata, files
Remove branding

63. Society for the Study of Autism Registry

64. <menu>
<title>
Society for the Study of...
</title>
<osf-search /> <osf-login />
</menu>
<society-questionnaire>
Author: <osf-user name />
Question 1: <osf-metadata />
Analysis code:
<osf-file upload />
</society-questionnaire>

65. OSF Application Framework
Workflow
Authentication
Permissions
File Storage
File Rendering
Meta-database
Persistence
Integrations
Search
SHARE
osf.io
journals
registries
preprint servers
grants management
university systems

66. Awaiting Instruction…

67. Awaiting Instruction…

68. Let experts be experts.

69. OSF Application Framework
Workflow
Authentication
Permissions
File Storage
File Rendering
Meta-database
Persistence
Integrations
Search
SHARE
registries

70. registries

71. OSF Application Framework
Workflow
Authentication
Permissions
File Storage
File Rendering
Meta-database
Persistence
Integrations
Search
SHARE
osf.io
journals
registries
preprint servers
grants management
university systems

72. Substantive experts shouldn’t deal with…
Workflow integration
Authentication
Permissions
File storage
File rendering
Database
Metadata/Annotations/Commenting
Persistence
External service integrations
Search
SHARE Data

73. …or scaling these: Workflow integration Authentication Permissions
File storage
File rendering
Database
Metadata/Annotations/Commenting
Persistence
External service integrations
Search
SHARE Data

74. @osframework, @jeffspies jeff@cos.io
Find this presentation at
_iwsg_spies.pptx
A Framework for Managing and Sharing Research Workflow