Open Peer Review: a way towards a more transparent scholarly publishing system Edit Görögh University of Göttingen Hungarian National Workshop on Open Science 2017 Debrecen, November 7
Changing scholary communication discourse 2017-11-07
dissemination of research results Our mission pening UP new methods, indicators and tools for… peer review dissemination of research results impact measurement Topics . Methodology Analysis of available methods User centered evaluation Defining require-ments Involving stakeholders Arts and Humanities Social sciences Life sciences Energy Use cases OpenUp aims to… within the Open Science ecosystem. 2017-11-07
Aims and activities Peer review landscape scan: Map out the alternative review tools and services Examine peer review in context of research flow and in different disciplinary settings Develop a framework for evidece-based research on peer review. Produce information resources. Produce policy recommendations. OpenUp hub (beta version) https://www.openuphub.eu/review 2017-11-07
Aims and activities 2. Contributing to the developing open science discourse Create ties with other EU projects – aligning efforts in researching open peer review and open science practices. Sharing taxonomies (FOSTER), Building on previous research (OpenAIRE), developing collaborations. Open science advocacy work: organizing workshops and webinars. 2017-11-07
Challenges 1. Complicated publishing scene Tennant, J. 2017. doi: 10.12688/f1000research.12037.1 2. Competing definitions of (open) peer review 2017-11-07 FORCE 2017, Berlin
Peer review re-defined Quality assurance mechanism where scholarly works are scrutinised by peers/experts, whose feedback are used to improve the works Manuscript IN OUT Objective: Mapping & promoting (Open) peer review mechanisms Anonymous Closed/Opaque Selective (participation) 2017-11-07 FORCE 2017, Berlin
Established review system Lack of accountabbility Lack of incentives Quality control Checking validitiy Assessing originality and significance Lengthy Costly Bias Lack of standards Abuse Tansparency Motivation Democratization Digital gap Problems of open ID SWOT Wasted efforts Inconsistencies Open Peer Review 2017-11-07
Defining open peer review authors and reviewers are aware of each other's identity review reports are published alonside the relevant article wider community contributes to the review process de-coupled from publishing: facilitated by a different organizational entity than the venue of publication direct reciprocal discussion between author(s) and reviewers, between reviewers Open identity Open Peer Review encompasses diverse constellations of many distinct aspects: 122 definitions collected and analysed 22 distinct configurations of 7 traits identified Open report Open participation Open platform Open interaction Open pre-review manuscripts manuscripts are made immediately available in advance of any formal peer review procedures Objective: Mapping & promoting (Open) peer review mechanisms Open final-version commenting review or commenting on final “version of record” publications Ross-Hellauer, 2017, doi: 10.12688/f1000research.11369.2 2017-11-07 FORCE 2017, Berlin
Alternative review services & platforms Publishers Publishing platforms Independent review services Repository based review platforms & tools Review/Annotation applications 2017-11-07
Publishing platforms Interactive peer review Collaborative peer review Post-publication peer review 2017-11-07
Decoupled review services 2017-11-07
Preprint based publishing Should researchers publish their findings before peer review? By IVAN ORANSKY and ADAM MARCUS MAY 27, 2016 2017-11-07
Changing discourse - Redefining roles Changing role of editors Growing responsibility of authors Proactive reviewer stance Involvement of peers 2017-11-07
Growing demands Transparency 2017-11-07 OpenUp Survey 2017 Reasons behind the reservations with the established peer-review system and proportions of respondents rating then as ‘very important’ or ‘somewhat important’ Note: Responses to question ‘2.1b - How important are the below reasons behind your reservations with the established peer review system?’ N=[253 – 256]. The percentages show a share of respondents who chose ‘very important’ and ‘somewhat important’ answer categories. OpenUp Survey 2017 2017-11-07
Growing demands 3. Training young scholars Incentives to review Crediting peer review Publons, Peerage of Science Peer review in academic promotion- recommendation of the OSI workgroup: Address incentives and motivations to participate in peer review, not only in the context of rewards or credits for individuals but also in terms of the importance of peer review for promotion and tenure. (Acreman 2016) 3. Training young scholars 2017-11-07
Reluctance to participate Reluctance governed by FEAR: Ideas being stolen Not being credited Public humiliation Abuse of power dynamics and intimidation Empowerment of bad actors Marginalization Less honesty and criticism. Source: Jon Tennant https://www.slideshare.net/OSFair/osfair2017-workshop-fear-and-loathing-in-open-peer-review 2017-11-07
Guidance Incentives Rewards Solutions Goal: Lack of clarity over assessment of outputs and activities Guidance Lack of professional incentives for being open Incentives Hiring, promotions fail to account for oprn science activities Rewards Cultural shift in scholarly research/publishing Evidence-based policies Shifting power dynamics Goal: build a global community of Open Science based on sharing and collaborations Source: Jon Tennant https://www.slideshare.net/OSFair/osfair2017-barriers-to-open-science-for-junior-researchers 2018.12.02
Collaborative writing tools Advance Open Science practices Collaborative writing tools Publishing platforms Repositories Altmetrics Open ID Source: Jeoren Bosman and Bianca Kramer, https://101innovations.wordpress.com/ 2017-11-07
Advance Open Science practices Open peer review oath Principle 1: I will sign my name to my review Principle 2: I will review with integrity Principle 3: I will treat the review as a discourse with you; in particular, I will provide constructive criticism Principle 4: I will be an ambassador for the practice of open science 2017-11-07
Data should be made publicly available. In case some data or materials are not open, clear reasons (e.g., legal, ethical constraints, or severe impracticality) should be given why. Documents containing details for interpreting any files or code, and how to compile and run any software programs should be made available with the above items. The location of all of these files should be advertised in the manuscript, and all files should be hosted by a reliable third party. 2017-11-07
Move toward greater inclusiveness by encouraging wider participation. Move toward greater transparency to improve accountability and minimize bias. Move toward greater inclusiveness by encouraging wider participation. Identify new approaches that lessen rather than increase the burden of reviewing and decrease the waste of reviewer’s time. Conduct more evidence-based analyses of different forms of peer review. Address incentives and motivations to participate OSI2016 Peer Review workgroup 2017-11-07
Source: Tony Ross-Hellauer https://www. slideshare 2017-11-07
Thank you! Thanks to Tony Ross-Hellauer and Jon Tennant for slides More information: http://openup-h2020.eu/ https://www.openuphub.eu/ @ProjectOpenUP Edit Görögh goeroegh@sub.uni-goettingen.de
References Novel Models for Open Peer Review. 2017. OpenAIRE2020 report. Open Science Monitor. 2017. EC Research and Innovation. Accessed on May 30, 2017: http://ec.europa.eu/research/openscience/index.cfm?section=monitor&pg=scholarlycomm#1 Peer Review Survey 2009. http://www.senseaboutscience.org/news.php/87/peer-review-survey-2009. (follow-up study of PRC 2008) Ross-Hellauer, 2017, “What is open peer review? A systematic review”, F1000Research. DOI: 10.12688/f1000research.11369.2 Ross-Hellauer, T. 2016. Disambiguating post-publication peer review. OpenAIRE blog. Accessed on Sept. 14, 2016: https://blogs.openaire.eu/?p=1205 Stančiauskas, V. and Banelytė, V. (2017). OpenUP survey on researchers' current perceptions and practices in peer review, impact measurement and dissemination of research results. Accessed on May 3, 2017. DOI: 10.5281/zenodo.556157 Tennant JP, Dugan JM, Graziotin D et al. A multi-disciplinary perspective on emergent and future innovations in peer review [version 1; referees: 2 approved with reservations]. F1000Research 2017, 6:1151. DOI: 10.12688/f1000research.12037.1