1. Archived File The file below has been archived for historical reference purposes only. The content and links are no longer maintained and may be outdated. See the OER Public Archive Home Page for more details about archived files.archived OER Public Archive Home Page

2. Improving Peer Review CSR Initiatives toni scarpa PRAC, April 30, 2008 National Institutes of Health U.S. Department of Health and Human Services

3.  New Data  Improving Peer Review: CSR Initiatives Changes In CSR Organization Realigning CSR Peer Review Peer Review at CSR

4. New Data

5. Number of Applications Submitted Historical Growth

6. SBIR Applications 7000 6000 5000 4000 3000 2000 1000 0 199920002001200220032004200520062007 INST CSR

7. Success Rates of Competing SBIR Applications 60 0 20002001200220032004200520062007 Fiscal Year 50 40 30 10 19981999 Percent Success Rate 20 SBIR Phase I SBIR Phase II

8. R01 Applications Reviewed at CSR vs. Other ICs

9. R21 Applications Reviewed at CSR vs. Other ICs

10. % of RO1 Application Submitted, Type 1

11. % of RO1 Application Submitted, Type 2

12. Reviewer’s Load Applications Per Reviewer Oct Council Rounds

13. Colleagues Reviewing for CSR

14. CSR Budget for 2008 CSR Budget $ 63 M SREA (Reviewers Expenses Budget)$ 39 M Review Cost, including travel for 17,000 reviewers, is less than 0.2% of the budget requested in the applications reviewed

15. SREA Budget Progression Millions

16. Travel Purchase non-refundable (restricted) coach tickets instead of unrestricted tickets Preliminary Pilot Data – June 2006 through July 2007 # of tickets issued 28,493 Changes requested 2,544 cost of changes + fees $550,274 YTD Fare Savings$14,522,877

17. Additional Savings AirfareHotelMeeting Average Cost Washington $ 7,682 $ 19,166 $ 26,848 West Coast $ 6,486 $ 8,974 $ 15,464

18. Improving Peer Review CSR Initiatives

19. Improving Peer Review Matrix A : Changes in CSR Operations Matrix B : Changes in Peer Review Matrix C : System Changes?

20. 1.CSR Reorganization of Division and IRGs 2.Review Enabling Committees 3.Increase Efficiency 4.Division Directors, Chiefs and SROs Recruitment Matrix A : Changes in CSR Operations

21. 1.CSR Reorganization of Division and IRGs Matrix A : Changes in CSR Operations

22. 1.CSR Reorganization of Division and IRGs Realign Divisions and IRGs with the changes in science Increase efficiency and effectiveness Increase consistency Recruitment of Scientific Review Officers Matrix A. Changes in CSR Operations

23. New Division Neuroscience, Aging and Development BDCN/Brain Disorders & Clinical Neuroscience (est. 11 SRAs) IFCN/Integrative, Functional, & Cognitive Neuroscience (est. 9 SRAs) MDCN/Molecular, Cellular & Developmental Neuroscience (est. 10 SRAs) ENST/Emerging Neuroscience & Training (est. 8 SRAs) BDA/Biology of Development & Aging (7 SRAs)

24. Organization of Peer Review at CSR Divisions Study Sections I.R.G.s (Integrated Review Groups)

25. New CSR Review Divisions Division A (Neuroscience, Development and Aging) Brain Disorders and Clinical Neuroscience IRG (BDCN) Molecular, Cellular and Developmental Neuroscience IRG (MDCN) Integrative, Functional and Cognitive Neuroscience IRG (IFCN) Emerging Technologies and Training in Neuroscience IRG (ETTN) Biology of Development and Aging IRG (BDA) Biobehavioral and Behavioral Processes IRG ( BBBP) Risk, Prevention and Health Behaviors IRG (RPHB) Epidemiology and Population Sciences IRG (EPS) Healthcare Delivery and Methodologies IRG (HDM) AIDS and Related Research IRG (AARR) Division B (AIDS, Behavioral and Population Sciences) Division C (Basic and Integrative Biological Sciences) Biological Chemical and Macromolecular Biophysics IRG (BCMB) Bioengineering Sciences and Technologies IRG (BST) Genes, Genomes and Genetics IRG (GGG) Oncology 1 – Basic Translational IRG (OBT) Cell Biology IRG (CB) Interdisciplinary Molecular Sciences and Training IRG (IMST) Division D (Physiological and Pathological Sciences) Endocrinology, Metabolism, Nutrition and Reproductive Sciences IRG (EMNR) Immunology IRG (IMM) Infectious Diseases and Microbiology IRG (IDM) Digestive, Kidney and Urological Systems IRG (DKUS) Division E (Translational and Clinical Sciences) Cardiovascular and Respiratory Sciences IRG (CVR) Surgical Sciences, Biomedical Imaging and Bioengineering IRG (SBIB) Musculoskeletal, Oral And Skin Sciences IRG (MOSS) Oncology 2 – Translational Clinical IRG (OTC) Vascular and Hematology IRG (VH) Scientific Review Groups= 48 Scientific Review Groups= 50 Scientific Review Groups= 43 Scientific Review Groups= 44 Scientific Review Groups= 55

26. Don Schneider

27. 1.CSR Reorganization of Division and IRGs 2.Review Enabling Committees Matrix A : Changes in CSR Operations

28. Review Enabling Committees Policy (Martin) Scientific Community Outreach (Schneider) Recruitment of SROs (Gibson) Recruitment of Reviewers (Stassi) Knowledge Management (Many) Peer Review Electronic (Many) SRA Training (Freund) Mentoring and staff development (Many) Best review practices (Kitt)

29. 1.CSR Reorganization of Division and IRGs 2.Review Enabling Committees 3.Increase Efficiency Matrix A : Changes in CSR Operations

30. Increase Efficiency How we distribute 80,000 applications? Retooled for electronic submission Applications are now submitted electronically Assign applications using text fingerprinting Implementation by August 2008

31. 1.CSR Reorganization of Division and IRGs 2.Review Enabling Committees 3.Increase Efficiency 4.Division Directors, Chiefs and SROs Recruitment Matrix A : Changes in CSR Operations

32. Recruitment 3 Division Directors First Search (Richard Nakamura, Chair) 6 Integrated Review Chiefs 40 Scientific Review Officers Matrix A : Changes in CSR Operations

33. Matrix B : Changes in Peer Review 1.Improve study section alignment and performance 2.Shorten the review cycle 3.Do more to recruit and retain more high-quality reviewers and decrease the burden on applicants and reviewers 4.Improve the identification of significant, innovative and high-impact research

34. CCSR Recent Activities in Peer Review SR Recent Activities in Peer Review 1.Improve Study Section Alignment and Performance

35. Second Round of Internal IRG Review July 24, 2008: BDCN and MDCN September 15, 2008: ETTN, BDA and IFCN January 2009: BBBP and HOP 2 (HDM) March 2009: RPHB and HOP 1 (EPS) May 2009: IMM and IDM July 2009: ONC 1-Basic and ONC 2-Clinical September 2009: BST and SBIB November 2009: AARR and EMNR January 2010: VH and CVR March 2010: MOSS and DRUS May 2010: GGG and BCMB July 2010: IMST and CB

36. Two Major Challenges Percentiling or Ranking Applications Reviewed in SEPS Locus of Review of “Orphan Applications”

37. Don Schneider Clustering of grant application for review

38. CCSR Recent Activities in Peer Review SR Recent Activities in Peer Review 1.Improve Study Section Alignment and Performance 2. Shorten the review cycle

39. Shortening the Review Cycle The Goal To provide applicants a review and score within 3 months of application submission. This will permit resubmission of applications (when doable and desirable) 4 months earlier than in the past. To permit 3 reviews within one year

40. R01 A1 Resubmissions Reviewed in the Cycle Immediately following the Original Application's Review - 74 60 72 124 29 44 30 51 0 20 40 60 80 100 120 140 160 180 200601200605200610200701200705200710 Council Cycle of Original R01 NewExperienced

41. CCSR Recent Activities in Peer Review SR Recent Activities in Peer Review 1.Improve Study Section Alignment and Performance 2. Shorten the review cycle 3.Do more to recruit and retain more high-quality reviewers and decrease the burden on applicants and reviewers

42. Colleagues Reviewing for CSR

43. Academic Rank of All CSR Reviewers

44. Near-Term Solutions for Recruiting and Retaining the Best Reviewers Remake peer review a learning experience Develop a national registry of volunteer reviewers Searchable Database with 2700 reviewers Move a meeting/year to the west coast 50 % this year, 90% next year Provide tangible rewards for reviewers No submission deadlines for permanent members of Study Sections (effective February 2008) Shorten applications Require less travel by expanding peer review platforms

45. Expanding Peer Review’s Platforms Study Sections Necessity ● Clinical reviewers Preference ● Physicists, computational biologists New Opportunities ● Fogarty, International Reviewers Goal: 15% of all reviews to be electronic in 2008 Electronic Reviews Video Enhanced Discussions Asynchronous Electronic Discussions

46. Matrix C: System Changes? The Advisory Committees of NIH Director

47. Advisory Committees to the NIH Director The Charge from Dr. Zerhouni: “Fund the best science, by the best scientists, with the least administrative burden…” with the least administrative burden…” http://enhancing-peer-review.nih.gov A Self-Study by the NIH in Partnership with the Scientific Community to Strengthen Peer Review in Changing Times

48. This is CSR