Open peer review as educational resource for science PhD students Eva Amsen, PhD Outreach Director, F1000Research Open Education Conference, 20 November 2014 eva.amsen@f1000.com http://f1000research.com @F1000Research Signup sheet

Review reports may be published with the article Explain what F1000Research is. Open peer review is an alternative peer review method used by (mainly) scientific journals to solve issues with the traditional peer review process. Reviewers are named Review reports may be published with the article As a side effect, some aspects of the open peer review model also have benefits for teaching graduate students about scientific publishing and peer review.

Outline Why teach science students about peer review? Undergraduate Graduate (PhD) From traditional peer review to open peer review How can open peer review be used to teach students about peer review in science? F1000Research’s involvement in peer review education

teaching undergraduate students about peer review “The term “peer reviewed journal” is so familiar to us [scientists] that it’s often hard to imagine that the importance of the process is not immediately apparent to our students.” Ros Gleadow (Monash University) Without training, students take on the bad habits of their supervisors and of the people who reviewed their papers.

Teaching undergraduate students about peer review SCI2010 ‘Practice and application of science’ course at Monash University (Melbourne and Malaysia campuses) Compulsory for all second year science students Teach students about peer review to introduce its importance in scientific practice. Use peer review in classroom to Manage large class size Let students experience peer review first hand.

Undergraduate vs graduate Undergraduate (and graduate) students read articles that have been peer reviewed. Graduate students also write articles that are being peer reviewed. Will soon be asked to be a peer reviewer.

teaching Graduate students about peer review “Good early training of graduate students and postdocs is needed to prevent them turning into future generations of manuscript-savaging reviewers.” Virginia Walbot (Stanford University) From: Are we training pit bulls to review our manuscripts?, Journal of Biology 2009, 8:24  doi:10.1186/jbiol125 Without training, students take on the bad habits of their supervisors and of the people who reviewed their papers.

A brief history of peer review in the sciences First scientific journals were not peer reviewed. Peer review was introduced in mid-20th century, to help editors cope with increasingly specialised fields and high numbers of submissions. Peer review developed as a method to select what is fit to print in limited available space. Journals as gatekeepers. First scientific journal 1665. Not yet peer reviewed. Goal: get information out there. More science -> more journals Not enough space to print all, so peer review to select. Over the course of the next three centuries, peer review then became a method to select “what is fit to print”, and journals became a sort of gatekeepers. By the mid-twentieth century, peer review as we know it was institutionalized and formalized.

How most peer review works at scientific journals Journal receives manuscript Editors decide whether the manuscript is something they want to publish Editors send the manuscript to peer reviewers. Reviewers give their feedback to the editor. Editor passes reviewer comments and editorial decision back to authors, without reviewer names. If article is accepted for publication (usually after a few rounds of revisions), it will be published without reviewer comments.

Science journals with open peer review 2014: The BMJ, Science Open 2012: F1000Research , eLife, PeerJ, GigaScience 2011 BMJ Open 2010 EMBO journal 2007 Frontiers journals 2006 Biology Direct 2001 Atmospheric Chemistry and Physics 2000 Medical journals in the BMC series Open peer review variations: All referees are named Optional referee names Reports shown Editorial decision public F1000Research: All referees are named Reports shown Take-home: Popular recently, but has been around for a while in several journals, in many different formats. (These examples all use invited reviewers, and show parts of the review process to readers of the papers.)

Why Open peer review? Discourage reviewers from holding back competitors’ work. Hold reviewers accountable for their words. Make publishing process more transparent. Step beyond open access ...Erin will tell you more!

F1000Research publication process F1000Research is making it possible to get an article online within days, by using post-publication peer review. When a manuscript is submitted to the journal, the in-house editorial team will run a pre-refereeing check. This thorough check includes making sure the language is readable, doing a plagiarism check, checking figures, making sure all data are included, and checking for ethical concerns. Once the paper is ready to be sent to referees, it’s also published online, and the data are in suitable repositories. Referees are invited based on expertise, much like at other journals. Authors can suggest referees, but the editorial team will make the final call whether an editor is suitable or not. When a referee submits a referee report, that is published with the article, and the reviewers are named. Registered users can also leave comments on the article at any time, but that is independent of the refereeing process. If an author needs to make revisions, they can send in a new version, and that will be peer reviewed again. All versions of an article and all referee reports are public. When an article has passed peer review, it becomes “indexed”, because it will now be sent to external databases such as PubMed. F1000Research publishes articles online before peer review Can see peer review as it happens

F1000Research article This is what an article looks like on F1000Research after it has received referee reports. Minimum required to pass review is two, but several papers have three or more. Can read all opinions.

F1000Research article This is what an article looks like on F1000Research after it has received referee reports. Minimum required to pass review is two, but several papers have three or more. Can read all opinions.

Article revisions and their review are all public Versions Article revisions and their review are all public When an author updates their article in response to reviewer comments, they upload a new version.

REFEREE reports are public Referee names are visible. View count shows how many people read the referee report This is what a referee report looks like. Referee reports on all papers are visible to anyone reading the article, and include the referee name. Referee reports can be cited independently of the article, and each have their own DOI. Author responses are visible as well. Referee reports and author comments are visible to anyone. Referee reports are citable with a DOI.

ALL our articles have ALL their peer review visible ALL our articles have ALL their peer review visible. We implemented this to make research more transparent, and to make our authors and reviewers happy, but having all those reports is an incredible resource for educators who are teaching students about peer review.

Peer review education page

Peer review education page

Current collaborations

Current collaborations NYU Neuroscience Institute workshop: Three sessions, led by NYU faculty. NYU contact: Nina Gray Goals: Show students what the peer review process looks like Encourage students to contribute to the scientific community by participating in peer review Develop critical thinking and analysis skills in students that will help them understand the literature in their own disciplines better. Provide tips on how to write journal articles that reviewers will find easy to approach and understand.

Current collaborations NYU Neuroscience Institute workshop: Incorporation of F1000Research articles in workshop: Step-by-step walkthrough of publicly reviewed article from F1000Research to show critical steps, rounds of revision, and steps taken by reviewers. Let students look at a not-yet-reviewed article, carry out their own review and later compare with the reviews published on the article. ONLY possible at F1000Research. Other journals that include full peer review history only do so after the article has already passed peer review.

Future collaborations... Interested in collaborating with F1000Research? We’re looking for people who would benefit from examples of open peer review reports to... ... teach (graduate) students how to do scientific peer review ... teach undergraduates about the scientific publishing process ... run workshops for scientists about open science

Links and contact information F1000Research peer review examples: http://f1000research.com/peer-reviewing-tips For information about collaboration: Eva Amsen – Outreach Director F1000Research eva.amsen@f1000.com Cesar Berrios-Otero – Outreach Director F1000Research Americas cesar.berrios-otero@f1000.com Or @F1000Research on Twitter