Archived File The file below has been archived for historical reference purposes only. The content and links are no longer maintained and may be outdated. See the OER Public Archive Home Page for more details about archived files.archivedOER Public Archive Home Page

Update on NIH Peer Review Enhancements Elias Zerhouni, MD Director, National Institutes of Health Peer Review Advisory Committee Meeting April 30, 2008

Reviewing Peer Review  Reality: First-rate peer review is a cornerstone of NIH  Emerging Reality: Increasing breadth, complexity, interdisciplinary nature of biomedical science are creating new challenges for peer review  Funding trends aggravate the stress on peer-review  NIH Response: Reviewing – and enhancing – peer review The Continuing Charge: “Fund the best science, by the best scientists, with the least administrative burden…” And the Added Challenge: … but recognize that “best” is dependent on many factors, including scientific quality; public health impact; mission of Institute or Center; existing NIH portfolio

Reviewing Peer Review: Project Phases Diagnostic Evaluate Actions Develop New NIH Policies Begin Phased Implementation of Selected Actions Jul 07 - Feb 08 Mar 08 - April 08 June 08 Design Implementation Plan

5 DRAFT Report of Challenges & Recommended Actions 1. Reducing Administrative Burden of Applicants, Reviewers and NIH Staff 2. Enhancing the Rating System 3. Enhancing Review & Reviewer Quality 4. Optimizing Support at Different Career Stages 5. Optimizing Support for Different Types and Approaches of Science 6. Reducing Stress on the Support System of Science 7. Meeting the Need for Continuous Review of Peer Review

Reviewing Peer Review: Project Phases Diagnostic Evaluate Actions Develop New NIH Policies Begin Phased Implementation of Selected Actions Jul 07 - Feb 08 Mar 08 - April 08 June 08 Design Implementation Plan

Develop Draft Implementation Plan  Release Draft Report Formal public comment period Outreach to professional organizations and other stakeholders  SCWG  SC Peer Review Implementation Groups 3 Working Groups  Cluster 1: Applications, review and ratings  Cluster 2: Quality of peer reviews  Cluster 3: Support of different career stages Position papers from EPMC, RPC, GMAC, PLC, TAC and eRA Broad input from NIH staff Town meeting with Scientific Review Officers and Program Staff Cross-cutting committee: provide integrated recommendations to NIH Director 7

Rocks Sand Pebbles Guided by several principles: 1.Do no harm 2.Continue to maximize the freedom of scientists to explore 3.Focus on the changes that are most likely to add significant value at a reasonable cost/benefit ratio Granularity of the Discussion: We need to tackle the big challenges

9 Core Theme: Excellence of Reviewers  The excellence of peer review is directly correlated to our ability to recruit, retain and motivate the most accomplished, broad minded and creative scientists to serve on study section Reduce burden of review Flexibility Training strategy Recognize and Reward distinguished service

10 Core Theme: The Fairness and Clarity of Peer Review  Consistently identify application’s relative merit, potential for scientific and/or public health impact, and feasibility  Applicants and NIH program officers deserve clear and purposeful review feedback, including informative summary statements and a rating system comparable across study sections and fields of science Summary statement structure aligned with specific criteria Pilot ranking strategies Pilot and implement shorter applications

Core Theme: Support Scientists at Different Stages of Their Careers  Peer review should fairly evaluate proposals from all scientists, regardless of their career stage or discipline, and avoid bias towards more conservative and proven approaches at the expense of innovation and originality Early stage investigators – reduce bias (number of fully discussed proposals, cluster discussion of ESI within panel, percentile separately) Reduce need for multiple applications Established investigators retrospective and prospective

Comparison of the Age of NIH PIs and Medical School Faculty 1980 Age Distribution of NIH RPG Investigators and Medical School Faculty Sources: IMPAC II Current and History Files and AAMC Faculty Roster System

1981 Age Distribution of NIH RPG Investigators and Medical School Faculty Sources: IMPAC II Current and History Files and AAMC Faculty Roster System Comparison of the Age of NIH PIs and Medical School Faculty

1982 Age Distribution of NIH RPG Investigators and Medical School Faculty Sources: IMPAC II Current and History Files and AAMC Faculty Roster System Comparison of the Age of NIH PIs and Medical School Faculty

1983 Age Distribution of NIH RPG Investigators and Medical School Faculty Sources: IMPAC II Current and History Files and AAMC Faculty Roster System Comparison of the Age of NIH PIs and Medical School Faculty

1984 Age Distribution of NIH RPG Investigators and Medical School Faculty Sources: IMPAC II Current and History Files and AAMC Faculty Roster System Comparison of the Age of NIH PIs and Medical School Faculty

1985 Age Distribution of NIH RPG Investigators and Medical School Faculty Sources: IMPAC II Current and History Files and AAMC Faculty Roster System Comparison of the Age of NIH PIs and Medical School Faculty

1986 Age Distribution of NIH RPG Investigators and Medical School Faculty Sources: IMPAC II Current and History Files and AAMC Faculty Roster System Comparison of the Age of NIH PIs and Medical School Faculty

1987 Age Distribution of NIH RPG Investigators and Medical School Faculty Sources: IMPAC II Current and History Files and AAMC Faculty Roster System Comparison of the Age of NIH PIs and Medical School Faculty

1988 Age Distribution of NIH RPG Investigators and Medical School Faculty Sources: IMPAC II Current and History Files and AAMC Faculty Roster System Comparison of the Age of NIH PIs and Medical School Faculty

1989 Age Distribution of NIH RPG Investigators and Medical School Faculty Sources: IMPAC II Current and History Files and AAMC Faculty Roster System Comparison of the Age of NIH PIs and Medical School Faculty

1990 Age Distribution of NIH RPG Investigators and Medical School Faculty Sources: IMPAC II Current and History Files and AAMC Faculty Roster System Comparison of the Age of NIH PIs and Medical School Faculty

1991 Age Distribution of NIH RPG Investigators and Medical School Faculty Sources: IMPAC II Current and History Files and AAMC Faculty Roster System Comparison of the Age of NIH PIs and Medical School Faculty

1992 Age Distribution of NIH RPG Investigators and Medical School Faculty Sources: IMPAC II Current and History Files and AAMC Faculty Roster System Comparison of the Age of NIH PIs and Medical School Faculty

1993 Age Distribution of NIH RPG Investigators and Medical School Faculty Sources: IMPAC II Current and History Files and AAMC Faculty Roster System Comparison of the Age of NIH PIs and Medical School Faculty

1994 Age Distribution of NIH RPG Investigators and Medical School Faculty Sources: IMPAC II Current and History Files and AAMC Faculty Roster System Comparison of the Age of NIH PIs and Medical School Faculty

1995 Age Distribution of NIH RPG Investigators and Medical School Faculty Sources: IMPAC II Current and History Files and AAMC Faculty Roster System Comparison of the Age of NIH PIs and Medical School Faculty

1996 Age Distribution of NIH RPG Investigators and Medical School Faculty Sources: IMPAC II Current and History Files and AAMC Faculty Roster System Comparison of the Age of NIH PIs and Medical School Faculty

1997 Age Distribution of NIH RPG Investigators and Medical School Faculty Sources: IMPAC II Current and History Files and AAMC Faculty Roster System Comparison of the Age of NIH PIs and Medical School Faculty

1998 Age Distribution of NIH RPG Investigators and Medical School Faculty Sources: IMPAC II Current and History Files and AAMC Faculty Roster System Comparison of the Age of NIH PIs and Medical School Faculty

1999 Age Distribution of NIH RPG Investigators and Medical School Faculty Sources: IMPAC II Current and History Files and AAMC Faculty Roster System Comparison of the Age of NIH PIs and Medical School Faculty

2000 Age Distribution of NIH RPG Investigators and Medical School Faculty Sources: IMPAC II Current and History Files and AAMC Faculty Roster System Comparison of the Age of NIH PIs and Medical School Faculty

2001 Age Distribution of NIH RPG Investigators and Medical School Faculty Sources: IMPAC II Current and History Files and AAMC Faculty Roster System Comparison of the Age of NIH PIs and Medical School Faculty

2002 Age Distribution of NIH RPG Investigators and Medical School Faculty Sources: IMPAC II Current and History Files and AAMC Faculty Roster System Comparison of the Age of NIH PIs and Medical School Faculty

2003 Age Distribution of NIH RPG Investigators and Medical School Faculty Sources: IMPAC II Current and History Files and AAMC Faculty Roster System Comparison of the Age of NIH PIs and Medical School Faculty

2004 Age Distribution of NIH RPG Investigators and Medical School Faculty Sources: IMPAC II Current and History Files and AAMC Faculty Roster System Comparison of the Age of NIH PIs and Medical School Faculty

2005 Age Distribution of NIH RPG Investigators and Medical School Faculty Sources: IMPAC II Current and History Files and AAMC Faculty Roster System Comparison of the Age of NIH PIs and Medical School Faculty

2006 Age Distribution of NIH RPG Investigators and Medical School Faculty Sources: IMPAC II Current and History Files and AAMC Faculty Roster System Comparison of the Age of NIH PIs and Medical School Faculty

2006 Age Distribution of NIH RPG Investigators and Medical School Faculty Sources: IMPAC II Current and History Files and AAMC Faculty Roster System 1980 Aging of Both NIH Grantees and Medical School Faculty Cohorts Since 1980

Age Distribution of PIs 2007 (Projection)

Age Distribution of PIs 2008

Age Distribution of PIs 2009

Age Distribution of PIs 2010

Age Distribution of PIs 2011

Age Distribution of PIs 2012

Age Distribution of PIs 2013

Age Distribution of PIs 2014

Age Distribution of PIs 2015

Age Distribution of PIs 2016

Age Distribution of PIs 2017

Age Distribution of PIs 2018

Age Distribution of PIs 2019

Age Distribution of PIs 2020

% 12.57% 3.35% 14.97% 6.96% 9.00% 7.18% 7.71% 7.99% 3.00% 5.41% 0.00% 3.79% 4.55% 6.97% 6.65% 12.53% 12.18% 13.16% 11.52% 13.28% 4.84% 1.85% 0.59% ,000 1,500 2, Fiscal Year Number of R01 First Time Investigators 0% 2% 4% 6% 8% 10% 12% 14% 16% Budget % Growth First time R01 EBudget History Budget History First Time R01, Budget History

Handling the Growing Number of Applications YearFirst-timeEstablishedTotal Fully Reviewed Applications  Streamlined Applications 

New Investigators Encounter Lower Success Rates at Every Submission Stage for Type 1 R01s

Percent R01-equivalent Awards and Amendment Status

Establish a continuous quality control and improvement process for peer review based on rigorous and independent prospective evaluations that favor rather than discourage adaptive and innovative approaches to review and program management Core Theme: Continuous quality control and improvement of peer review

Reviewing Peer Review: Project Phases Diagnostic Evaluate Actions Develop New NIH Policies Begin Phased Implementation of Selected Actions Jul 07 - Feb 08 Mar 08 - April 08 June 08 Design Implementation Plan Implement Communication Plan with Stakeholders

PRAC Input Needed  Advise on proposed implementation steps What makes a reviewer “distinguished”? What criteria could be used to evaluate reviewers? How do we continuously measure and maintain quality and responsiveness of Peer review process Other recommendations

NIH Transforming medicine and health through discovery