Archived File The file below has been archived for historical reference purposes only. The content and links are no longer maintained and may be outdated. See the OER Public Archive Home Page for more details about archived files.archived OER Public Archive Home Page

CSR Best Practices Committee Cheryl A. Kitt, Ph.D. Deputy Director National Institutes of Health U.S. Department of Health and Human Services December 3, 2007

CSR’s Mission Ensure NIH grant applications receive fair, independent, expert, and timely reviews To ensure that the process continue to operate at a high level, the CSR Best Practices Committee was formed.

Charge to the CSR Best Practices Committee: Goals: Review existing policies and procedures Ensure alignment with FACA and other regulations Ensure consistency of practices and policies across study sections Promote the sharing of best practices across sections at CSR

The Committee Authored Five CSR Peer Review Best Practices Guidelines: 1.Criteria for Selecting Reviewers and Assembling Rosters 2.Conduct of Study Section Meetings – Roles and Responsibilities 3.Roles and Responsibilities for Summary Statement Production 4.Telephone Reviewers at Face-to-Face Review Meetings 5.Mail Reviews * These documents are companion pieces to the SRA Handbook, a critical compendium of peer review policies and practices

1. Criteria for Selecting Reviewers and Assembling Rosters Core Values Core Values: The quality of scientific evaluation of a grant application is a reflection of the quality of the reviewers on the roster. SROs must ensure that the study section remains responsive to emerging areas of science and shifting scientific boundaries, and maintain a balanced membership with respect to geographic, gender, and minority representation. Balance is needed between recruiting generalists for their breadth and specialists for specific niche research areas.

Reviewer Selection General Considerations: Independent and Established Investigator Respected by peers (e.g., department chair, journal editor, clinical status, plenary lectures, keynote speaker, etc.) Quality of research accomplishments – Impact on a field Independent publications in peer-reviewed journals Honors (e.g., Awards, Memberships, etc.) Research Support (e.g., NIH, VA, NSF, foundations, industry, foreign, etc.)

For Fellowship Committees: All of the above Broad perspective Reviewer’s track record of mentoring Reviewer’s track record of mentoring: e.g., students, pre- and/or postdoctoral fellows; institutional or unit (departmental, division; lab) mentoring- years and resources For Small Business Committees: If from academia, criteria listed above Rank in the small business concern (e.g., CSO, section head, directorship, group/project leader) SBIR/STTR funding Publication record in peer reviewed, high visibility journals or Patents Recognized as prominent practitioner in a professional field (e.g., surgeon, engineer)

Sources for Reviewer Recruitment: Research databases (e.g. CRISP, PubMed, PsychInfo, etc.) Professional societies Scientific community consultation NIH Program Staff Conference attendance Study section chairs Study section members

Maintaining Reasonable Application to Reviewer Workload Best Practice Statement: 7 assignmentsFor reviewers attending the meeting, the goal is 7 assignments per reviewer on average, which includes reader assignments (see exceptions below). A workload of 7 gives the SRO flexibility to bring in a limited number of reviewers with light loads (e.g., field leaders, reviewers for a small group of specialized applications), balanced by the rest of the study section who are assigned a more substantial number of applications to review.*Exceptions: For small numbers of temporary members, member SEPs, re- review SEPs, multidisciplinary clinical trials, etc., that must be approved by the IRG Chief.

2. Conduct of Study Section Meetings Goal: conduct before and at study section meetings To provide consistent and clear guidance to SROs regarding the expectations of conduct before and at study section meetings, and to provide guidance for continued oversight of this critically important function. An effective peer review process requires an SRO to be active at the meeting, as well as in the weeks before the meeting. *An effective peer review process requires an SRO to be active at the meeting, as well as in the weeks before the meeting.

Conduct of Study Section Meetings How: Establish contact with reviewers and stay in touch with themEstablish contact with reviewers and stay in touch with them during the weeks leading up to the meeting. ongoing interactions between SRO, IRG Chief, and Division DirectorFoster a team environment with ongoing interactions between SRO, IRG Chief, and Division Director to enhance the quality of peer review. Use Pre- and Post-Meeting ChecklistsUse Pre- and Post-Meeting Checklists

3. Summary Statement Production Best Practices High quality summary statements depend both on the reviewer and SRO. Reviewers effectively communicate in writing the critique of an application. SRO provides orientation and oversight before and during meetings to avoid discrepancies and misleading statements. SRO monitors critiques posted in IAR and provides an opportunity to detect and correct problems before the meeting, enhancing the review process and subsequent summary statement. The SRO uses written critiques and input from study section discussions to assemble summary statements that are concordant with the scores.

4. Telephone Reviewers at Face-to-face Review Meetings Limit number of telephone reviewers initially to 10% and not to exceed 20% for regular face-to-face meetings, barring unforeseen emergencies. Telephone reviewers should have prior review experience. SROs must focus on handling the meeting and taking notes. Thus, they should be minimally involved in communication logistics. Telephone reviewers who are not regular members preferably should not lead off the discussion. Telephone reviewers submit their scores in a secure manner, preferably via IAR or Meeting One.

5. MAIL REVIEWS In balancing expert coverage, SROs will on occasion have an application where a small part of the proposed project involves a special expertise or technique not covered by the existing members of the panel. If the size of the need for the expertise does not warrant addition of another member to the meeting then a mail review may be appropriate.

Parking Lot Issues Conflict of Interest Streamlining Appeals/Rebuttals Deferral for Re-Review Selection of Study Section Chairs Training of Reviewers and Chairs Role of Program Staff at Meetings

Committee Members Shirley, Mariela (NIAAA-Program Director) Bradley, Eileen (CSR-IRG Chief) Byrnes, Noni (CSR-IRG Chief) Cooper, Cathleen (CSR-SRO) Edwards, Emmeline (NINDS-Dep. Dir. DEA) Etcheberrigaray, Rene (CSR-IRG Chief) Fisher, Suzanne (CSR-Division Director) Gibson, Joy (CSR-IRG Chief) Khan, Mushtaq (CSR-IRG Chief) Krishnan, Krish (CSR-SRO) Panniers, Richard (CSR-IRG Chief) Pyper, Joanna (CSR-Dep. IRG Chief) Rigas, Marc (CSR-SRO) Schneider, Donald (CSR-Division Director) Kitt, Cheryl (CSR), Chair