1. Center for Scientific Review National Institutes of Health Department of Health and Human Services Toni Scarpa NIH Peer Review: Continuity and Change NIDA Council Bethesda, Feb 8, 2006

2. Peer Review: An N.I.H. “Conception” Is the heart and soul of NIH Has produced an effective partnership between the federal government and research institutions Has created the best academic medical centers, the best biomedical/behavioral research and biotechnology Has made possible the best cures and the best prevention Has been admired and imitated here and abroad Has protected NIH against outside influence

3. Center for Scientific Review

4. This Is Not Amazon.com This is CSR

5. Applications Received for all of NIH and Applications Referred for CSR Review, FY 1998 - 2005 NIH Applications Applications for CSR Review

6. CSR Mission Statement To see that NIH grant applications receive fair, independent, expert, and timely reviews -- free from inappropriate influences -- so NIH can fund the most promising research.

7. NIDA Applications Reviewed by CSR in FY 2002 and 2005 20022005 Number of applications reviewed by CSR (for all ICs) 33,46951,690 Number of NIDA applications reviewed by CSR Percent of total reviewed by CSR 969 (2.9%) 1,480 (2.9%) Number of CSR study section meetings 1,3871,798 Number of CSR study sections meetings that reviewed NIDA applications (Percent of total) 225 (16.2%) 304 16.9% Fiscal Year of Review

8. Review Outcomes for NIDA Applications in 2002 and 2005 R01 Applications Only FYN % Unscored Median Priority Scores % Scoring at the 10 th percentile or better % Scoring at the 20 th percentile or better 200265432.4%244.07.8%18.6% 200594841.1%*239.09.8% 20.5% * FY comparison, p <.001

9. Time Complexity and Impact CSR Operations Current Systems New Systems? Necessary Changes in CSR Peer Review Operations

10. Increase communications between CSR, the ICs, our reviewers and applicants Increase uniformity Increase efficiency Facilitate work of IC program staff Changes in CSR Operations

11. Increased Communication and Transparency Within CSR With NIH and other Agencies With the Scientific Community Changes in CSR Operations 1

12. Increase uniformity Slate Nomination Summary Statements Posting all within one month of Study Section meetings Posting Summary Statements of new investigators within one week Producing more complete and structured resumes Unscoring Common practice Unscoring 50% Changes in CSR Operations 2

13. Increase Efficiency Electronic Submission Text Fingerprinting, Artificial Intelligence Software Changes in CSR Operations 3

14. Potential of Knowledge Management Tools for Peer Review Collexis Software or Others Knowledge management solutions Fingerprinting and text retrieving Disease coding Benefits for Peer Review Assigning applications to Integrated Review Groups or Study Sections Selecting reviewers (one application, multiple applications) Nine pilots are underway to begin to assess these benefits

15. Facilitate work of IC program staff Possible Changes in CSR Operations

16. Study Section Realignment Review of one IRG every month Total review every 2 years

17. Required Changes in Current Systems Shorten the review cycle

18. This is Not an Ford Assembly Line ReceiptRefer Evaluate Scientific Merit of Applications EnterpriseArchitecture@mail.nih.gov

19. Shortening the NIH Review Cycle, Initial Steps For most research grants, we are posting summary statements within one month after the study section meeting instead of two to three months after the meeting (effective Oct 05) We are conducting a pilot study to speed the review process for new investigators so they may revise and resubmit for the very next review cycle 4 months earlier than before (effective Feb 06)

20. Possible Changes in Current Systems Shorten the review cycle Address concern that clinical research is not properly evaluated Improve the assessment of innovative, high- risk/high-reward research Do more to recruit and retain more high-quality reviewers

21. Expanding Peer Review’s Platforms Electronic Reviews Telephone Enhanced Discussions Video Enhanced Discussions Asynchronous Electronic Discussions Study Sections Necessity ● Clinical reviewers Preference ● Physicists, computational biologists New Opportunities ● Fogarty, International Reviewers

22. Applications Received for All of NIH FY 1998-2004 0 20,000 40,000 60,000 80,000 1998200020022004 Fiscal year Number of applications

23. Number of Research Grant Applications/Applicant

24. CSR Applications Reviewed, Regular and SEP May Council Only

25. Study Section Application/Reviewer Ratio October Council Only

26. CENTER FOR SCIENTIFIC REVIEW FY 2004-2006 Non-Discretionary vs. Discretionary Spending ■ Non-Discretionary ■ Discretionary FY 2004FY 2005 FY 2006

27. If we didn’t have any peer-review system and we had to design one from scratch, what would it look like? Possible New Systems

28. This is CSR