1. Archived File The file below has been archived for historical reference purposes only. The content and links are no longer maintained and may be outdated. See the OER Public Archive Home Page for more details about archived files.archivedOER Public Archive Home Page

2. Update on NIH Peer Review Enhancements Elias Zerhouni, MD Director, National Institutes of Health Peer Review Advisory Committee Meeting April 30, 2008

3. Reviewing Peer Review  Reality: First-rate peer review is a cornerstone of NIH  Emerging Reality: Increasing breadth, complexity, interdisciplinary nature of biomedical science are creating new challenges for peer review  Funding trends aggravate the stress on peer-review  NIH Response: Reviewing – and enhancing – peer review The Continuing Charge: “Fund the best science, by the best scientists, with the least administrative burden…” And the Added Challenge: … but recognize that “best” is dependent on many factors, including scientific quality; public health impact; mission of Institute or Center; existing NIH portfolio

4. Reviewing Peer Review: Project Phases Diagnostic Evaluate Actions Develop New NIH Policies Begin Phased Implementation of Selected Actions Jul 07 - Feb 08 Mar 08 - April 08 June 08 Design Implementation Plan

5. 5 DRAFT Report of Challenges & Recommended Actions 1. Reducing Administrative Burden of Applicants, Reviewers and NIH Staff 2. Enhancing the Rating System 3. Enhancing Review & Reviewer Quality 4. Optimizing Support at Different Career Stages 5. Optimizing Support for Different Types and Approaches of Science 6. Reducing Stress on the Support System of Science 7. Meeting the Need for Continuous Review of Peer Review http://enhancing-peer-review.nih.gov/

6. Reviewing Peer Review: Project Phases Diagnostic Evaluate Actions Develop New NIH Policies Begin Phased Implementation of Selected Actions Jul 07 - Feb 08 Mar 08 - April 08 June 08 Design Implementation Plan

7. Develop Draft Implementation Plan  Release Draft Report Formal public comment period Outreach to professional organizations and other stakeholders  SCWG  SC Peer Review Implementation Groups 3 Working Groups  Cluster 1: Applications, review and ratings  Cluster 2: Quality of peer reviews  Cluster 3: Support of different career stages Position papers from EPMC, RPC, GMAC, PLC, TAC and eRA Broad input from NIH staff Town meeting with Scientific Review Officers and Program Staff Cross-cutting committee: provide integrated recommendations to NIH Director 7

8. Rocks Sand Pebbles Guided by several principles: 1.Do no harm 2.Continue to maximize the freedom of scientists to explore 3.Focus on the changes that are most likely to add significant value at a reasonable cost/benefit ratio Granularity of the Discussion: We need to tackle the big challenges

9. 9 Core Theme: Excellence of Reviewers  The excellence of peer review is directly correlated to our ability to recruit, retain and motivate the most accomplished, broad minded and creative scientists to serve on study section Reduce burden of review Flexibility Training strategy Recognize and Reward distinguished service

10. 10 Core Theme: The Fairness and Clarity of Peer Review  Consistently identify application’s relative merit, potential for scientific and/or public health impact, and feasibility  Applicants and NIH program officers deserve clear and purposeful review feedback, including informative summary statements and a rating system comparable across study sections and fields of science Summary statement structure aligned with specific criteria Pilot ranking strategies Pilot and implement shorter applications

11. Core Theme: Support Scientists at Different Stages of Their Careers  Peer review should fairly evaluate proposals from all scientists, regardless of their career stage or discipline, and avoid bias towards more conservative and proven approaches at the expense of innovation and originality Early stage investigators – reduce bias (number of fully discussed proposals, cluster discussion of ESI within panel, percentile separately) Reduce need for multiple applications Established investigators retrospective and prospective

12. Comparison of the Age of NIH PIs and Medical School Faculty 1980 Age Distribution of NIH RPG Investigators and Medical School Faculty Sources: IMPAC II Current and History Files and AAMC Faculty Roster System

13. 1981 Age Distribution of NIH RPG Investigators and Medical School Faculty Sources: IMPAC II Current and History Files and AAMC Faculty Roster System Comparison of the Age of NIH PIs and Medical School Faculty

14. 1982 Age Distribution of NIH RPG Investigators and Medical School Faculty Sources: IMPAC II Current and History Files and AAMC Faculty Roster System Comparison of the Age of NIH PIs and Medical School Faculty

15. 1983 Age Distribution of NIH RPG Investigators and Medical School Faculty Sources: IMPAC II Current and History Files and AAMC Faculty Roster System Comparison of the Age of NIH PIs and Medical School Faculty

16. 1984 Age Distribution of NIH RPG Investigators and Medical School Faculty Sources: IMPAC II Current and History Files and AAMC Faculty Roster System Comparison of the Age of NIH PIs and Medical School Faculty

17. 1985 Age Distribution of NIH RPG Investigators and Medical School Faculty Sources: IMPAC II Current and History Files and AAMC Faculty Roster System Comparison of the Age of NIH PIs and Medical School Faculty

18. 1986 Age Distribution of NIH RPG Investigators and Medical School Faculty Sources: IMPAC II Current and History Files and AAMC Faculty Roster System Comparison of the Age of NIH PIs and Medical School Faculty

19. 1987 Age Distribution of NIH RPG Investigators and Medical School Faculty Sources: IMPAC II Current and History Files and AAMC Faculty Roster System Comparison of the Age of NIH PIs and Medical School Faculty

20. 1988 Age Distribution of NIH RPG Investigators and Medical School Faculty Sources: IMPAC II Current and History Files and AAMC Faculty Roster System Comparison of the Age of NIH PIs and Medical School Faculty

21. 1989 Age Distribution of NIH RPG Investigators and Medical School Faculty Sources: IMPAC II Current and History Files and AAMC Faculty Roster System Comparison of the Age of NIH PIs and Medical School Faculty

22. 1990 Age Distribution of NIH RPG Investigators and Medical School Faculty Sources: IMPAC II Current and History Files and AAMC Faculty Roster System Comparison of the Age of NIH PIs and Medical School Faculty

23. 1991 Age Distribution of NIH RPG Investigators and Medical School Faculty Sources: IMPAC II Current and History Files and AAMC Faculty Roster System Comparison of the Age of NIH PIs and Medical School Faculty

24. 1992 Age Distribution of NIH RPG Investigators and Medical School Faculty Sources: IMPAC II Current and History Files and AAMC Faculty Roster System Comparison of the Age of NIH PIs and Medical School Faculty

25. 1993 Age Distribution of NIH RPG Investigators and Medical School Faculty Sources: IMPAC II Current and History Files and AAMC Faculty Roster System Comparison of the Age of NIH PIs and Medical School Faculty

26. 1994 Age Distribution of NIH RPG Investigators and Medical School Faculty Sources: IMPAC II Current and History Files and AAMC Faculty Roster System Comparison of the Age of NIH PIs and Medical School Faculty

27. 1995 Age Distribution of NIH RPG Investigators and Medical School Faculty Sources: IMPAC II Current and History Files and AAMC Faculty Roster System Comparison of the Age of NIH PIs and Medical School Faculty

28. 1996 Age Distribution of NIH RPG Investigators and Medical School Faculty Sources: IMPAC II Current and History Files and AAMC Faculty Roster System Comparison of the Age of NIH PIs and Medical School Faculty

29. 1997 Age Distribution of NIH RPG Investigators and Medical School Faculty Sources: IMPAC II Current and History Files and AAMC Faculty Roster System Comparison of the Age of NIH PIs and Medical School Faculty

30. 1998 Age Distribution of NIH RPG Investigators and Medical School Faculty Sources: IMPAC II Current and History Files and AAMC Faculty Roster System Comparison of the Age of NIH PIs and Medical School Faculty

31. 1999 Age Distribution of NIH RPG Investigators and Medical School Faculty Sources: IMPAC II Current and History Files and AAMC Faculty Roster System Comparison of the Age of NIH PIs and Medical School Faculty

32. 2000 Age Distribution of NIH RPG Investigators and Medical School Faculty Sources: IMPAC II Current and History Files and AAMC Faculty Roster System Comparison of the Age of NIH PIs and Medical School Faculty

33. 2001 Age Distribution of NIH RPG Investigators and Medical School Faculty Sources: IMPAC II Current and History Files and AAMC Faculty Roster System Comparison of the Age of NIH PIs and Medical School Faculty

34. 2002 Age Distribution of NIH RPG Investigators and Medical School Faculty Sources: IMPAC II Current and History Files and AAMC Faculty Roster System Comparison of the Age of NIH PIs and Medical School Faculty

35. 2003 Age Distribution of NIH RPG Investigators and Medical School Faculty Sources: IMPAC II Current and History Files and AAMC Faculty Roster System Comparison of the Age of NIH PIs and Medical School Faculty

36. 2004 Age Distribution of NIH RPG Investigators and Medical School Faculty Sources: IMPAC II Current and History Files and AAMC Faculty Roster System Comparison of the Age of NIH PIs and Medical School Faculty

37. 2005 Age Distribution of NIH RPG Investigators and Medical School Faculty Sources: IMPAC II Current and History Files and AAMC Faculty Roster System Comparison of the Age of NIH PIs and Medical School Faculty

38. 2006 Age Distribution of NIH RPG Investigators and Medical School Faculty Sources: IMPAC II Current and History Files and AAMC Faculty Roster System Comparison of the Age of NIH PIs and Medical School Faculty

39. 2006 Age Distribution of NIH RPG Investigators and Medical School Faculty Sources: IMPAC II Current and History Files and AAMC Faculty Roster System 1980 Aging of Both NIH Grantees and Medical School Faculty Cohorts Since 1980

40. Age Distribution of PIs 2007 (Projection)

41. Age Distribution of PIs 2008

42. Age Distribution of PIs 2009

43. Age Distribution of PIs 2010

44. Age Distribution of PIs 2011

45. Age Distribution of PIs 2012

46. Age Distribution of PIs 2013

47. Age Distribution of PIs 2014

48. Age Distribution of PIs 2015

49. Age Distribution of PIs 2016

50. Age Distribution of PIs 2017

51. Age Distribution of PIs 2018

52. Age Distribution of PIs 2019

53. Age Distribution of PIs 2020

55. 1348 10.42% 12.57% 3.35% 14.97% 6.96% 9.00% 7.18% 7.71% 7.99% 3.00% 5.41% 0.00% 3.79% 4.55% 6.97% 6.65% 12.53% 12.18% 13.16% 11.52% 13.28% 4.84% 1.85% 0.59% - 500 1,000 1,500 2,000 1983198419851986198719881989199019911992199319941995199619971998199920002001200220032004200520062007 Fiscal Year Number of R01 First Time Investigators 0% 2% 4% 6% 8% 10% 12% 14% 16% Budget % Growth First time R01 EBudget History 1363 15961683 Budget History First Time R01, Budget History

56. Handling the Growing Number of Applications YearFirst-timeEstablishedTotal Fully Reviewed Applications 20024131615610287 20073612649210104  -519+336-183 Streamlined Applications 2002335141187469 20075154729312447  +1803+3175+4978

57. New Investigators Encounter Lower Success Rates at Every Submission Stage for Type 1 R01s

58. Percent R01-equivalent Awards and Amendment Status

59. Establish a continuous quality control and improvement process for peer review based on rigorous and independent prospective evaluations that favor rather than discourage adaptive and innovative approaches to review and program management Core Theme: Continuous quality control and improvement of peer review

60. Reviewing Peer Review: Project Phases Diagnostic Evaluate Actions Develop New NIH Policies Begin Phased Implementation of Selected Actions Jul 07 - Feb 08 Mar 08 - April 08 June 08 Design Implementation Plan Implement Communication Plan with Stakeholders

61. PRAC Input Needed  Advise on proposed implementation steps What makes a reviewer “distinguished”? What criteria could be used to evaluate reviewers? How do we continuously measure and maintain quality and responsiveness of Peer review process Other recommendations

62. NIH Transforming medicine and health through discovery