1. Archived File The file below has been archived for historical reference purposes only. The content and links are no longer maintained and may be outdated. See the OER Public Archive Home Page for more details about archived files.archived OER Public Archive Home Page

2. Improving Peer Review CSR Initiatives PRAC, August 27, 2007 National Institutes of Health U.S. Department of Health and Human Services

3.  CSR Data  CSR Recent Activities  CSR Organization Initiatives  Realigning CSR Peer Review  Initiatives Under Early Discussion Peer Review at CSR

4. CSR Data

5. Number of Applications Submitted to NIH Historical Growth

6. Changes Impacting CSR Peer Reviews 5 Years Ago Now Applications Received46,00080,000 R01s and R21s Reviewed by CSR21,00039,000 10 Years Ago Now Average Number of Applications Reviewed by Reviewers 11.66 The first problem The second problem

7. 2006: CSR Peer Review Statistics 80,000 applications received 55,000 applications reviewed 18,000 reviewers 238 Scientific Review Administrators 1,800 review meetings

8. Application Reviewed by CSR

9. Growth of R01 Applications Reviewed at CSR vs. Other ICs

10. Growth of R21 Applications Reviewed at CSR vs. Other ICs

11. CSR Recent Activities

12. 1.Increase Communication and Transparency 2.Increase Uniformity 3.Increase Efficiency 4. Improve Study Section Alignment and Performance Changes in CSR Operations

13. 1. Increase Communication Advisory Committee Division Director IRG Chief SRA 1 SRA 2 Mgmt. Services Cross Cutting Weekly Biweekly Monthly Monthly Monthly Monthly

14. 2. Increase Uniformity Appeal Advisory Committee Best Practices Committee Assessments Appropriate Rosters Conduct of Meetings Structure of Summary Statements Changes in CSR Operations

15. 1.Increase Communication and Transparency 2.Increase Uniformity 3.Increase Efficiency Changes in CSR Operations

16. Increase Efficiency How we distribute 80,000 applications? Retooled for electronic submission Applications are now submitted electronically Assign applications using text fingerprinting, artificial intelligence software Full Implementation by February 2008

17. 1.Increase Communication and Transparency 2.Increase Uniformity 3.Increase Efficiency 4.Improve Study Section Alignment and Performance Changes in CSR Operations

18. Biannual IRG Review Schedule Scheduled 2007 -- 9 IRGs Biology of Development and Aging (BDA) Infectious Diseases and Microbiology (IDM) Biobehavioral and Behavioral Processes (BBBP) Cell Biology (CB) Musculoskeletal, Oral and Skin Sciences (MOSS) Oncological Sciences (ONC) Surgical Sciences, Biomedical Imaging and Bioengineering (SBIB) Respiratory Sciences (RES) Renal and Urological Sciences (RUS) Scheduled 2006 --14 IRGs Biological Chemistry and Macromolecular Biophysics (BCMB) Cardiovascular Science (CVS) Bioengineering Sciences and Technologies (BST) AIDS and Related Research (AARR) Risk, Prevention, and Health Behavior (RPHB) Genes, Genomes and Genetics (GGG) Digestive Sciences (DIG) Endocrinology, Metabolism, Nutrition and Reproductive Sciences (EMNR) Brain Disorders and Clinical Neuroscience (BDCN) Integrative, Functional and Cognitive Neuroscience (IFCN) Molecular, Cellular and Developmental Neuroscience (MDCN) Hematology (HEME) Immunology (IMM) Health of the Population (HOP)

19. Six Open House Workshops in 2007 Mar. 2,Neurological (3): Brain Disorders and Clinical Neuroscience (BDCN); Integrative, Functional and Cognitive Neuroscience (IFCN); Molecular, Cellular and Developmental Neuroscience (MDCN) April 25Behavioral/Social (3): Biobehavioral and Behavioral Processes (BBBP); Health of the Population (HOP); Risk Prevention and Health Behavior (RPHB) Jun. 29Disease-based (4): AIDS and Related Research (AARR); Infectious Diseases and Microbiology (IDM); Oncological Sciences (ONC); Surgical Sciences, Biomedical Imaging and Bioengineering (SBIB) Aug. 24Integrated Biological (4): Digestive Sciences (DIG); Musculoskeletal, Oral and Skin Sciences (MOSS); Renal and Urological Sciences (RUS) Endocrinology, Metabolism, Nutrition and Reproductive Sciences (EMNR) Nov. 9 Integrated Biological (5): Immunology (IMM); Hematology (HEME); Cardiovascular Sciences (CVS); Respiratory Sciences (RES); Biology of Development and Aging (BDA) Dec. 18Biomolecular (4): Biological Chemistry and Macromolecular Biophysics (BCMB); Bioengineering Sciences and Technologies (BST); Cell Biology (CB); Genes, Genomes and Genetics (GGG)

20. Neuroscience Open House, March 2007

21. Neuroscience Open House Workshop Total registrants169 Study Section Chairs26 PRAC Members2 Professional Society Members From 26 different societies 52 NIH staff (non-CSR) 14 ICs represented 33 CSR staff members50 Public members6

22. CSR Organization Initiatives

23. CSR Organization

24. Review Enabling Committees Policy Scientific Community Outreach Recruitment of SRAs Recruitment of Reviewers Knowledge Management Electronic Reviews SRA Training Mentoring and staff development Best review practices

25. CSR 4 Review Divisions & 23 IRGs Division of Biologic Basis of Disease. AIDS and Related Research (AARR). Endoc, Metabolism, Nutrition, Reproductive Sciences (EMNR) Immunological Sciences (IMM) Oncological Sciences (ONC) Infectious Diseases and Microbiology (IDM). Division of Physiology and Pathology. Cardiovascular Sciences (CVS) Integrative, Functional Cognitive Neuroscience (IFCN). Renal and Urological Sciences (RUS). Hematology (HEME) Digestive Sciences IRG (DIG) Division of Clinical and Population-Based Studies Surgery, Radiology, and Bioengineering (SRB) Health of the Population (HOP) Risk, Prevention, and Health Behavior (RPHB) Brain Disorders and Clinical Neuroscience (BDCN) Behavioral & Biobehavioral Processes (BBBP) Division of Molecular & Cellular Mechanisms. Biology of Development and Aging (BDA) Biological Chemistry and Macromolecular Biophysics (BCMB) Cell Biology (CB). Bioengineering Sciences and Technologies (BST) Genes, Genomes, and Genetics (GGG). Molecular, Cellular Develop Neuroscience (MDCN). Respiratory Sciences (RES) Musculoskeletal, Oral, Skin Sciences (MOSS)

26. Organization of Peer Review at CSR Divisions Study Sections I.R.G.s (Integrated Review Groups)

27. New IRG ENST/Emerging Neuroscience & Training

28. Organization of Peer Review at CSR Divisions Study Sections I.R.G.s (Integrated Review Groups)

29. New Division Neuroscience, Aging and Development BDCN/Brain Disorders & Clinical Neuroscience (est. 11 SRAs) IFCN/Integrative, Functional, & Cognitive Neuroscience (est. 9 SRAs) MDCN/Molecular, Cellular & Developmental Neuroscience (est. 10 SRAs) ENST/Emerging Neuroscience & Training (est. 8 SRAs) BDA/Biology of Development & Aging (7 SRAs)

30. Realigning Peer Review

31. CCSR Recent Activities in Peer Review SR Recent Activities in Peer Review 1.Shorten the review cycle 2.Do more to recruit and retain more high-quality reviewers and decrease the burden on applicants and reviewers 3.Improve the identification of significant, innovative and high-impact research

32. Shortening the Review Cycle First Step: Posting Summary Statements Earlier Post new investigator within 10 days, (within 30 days for established investigators) Second Step: Pilot to Shorten the Review Cycle for New R01 Investigators Last year: About 2,000 new investigators in pilot Third Step: Shorter Review Cycles to All R01 From New Investigators Feb 2007: 3,000 June 2007: 6,000 new investigators Nov 2007: All new investigators (>9000)

33. R01 A1 Resubmissions Reviewed in the Cycle Immediately following the Original Application's Review - 74 60 72 124 29 44 30 51 0 20 40 60 80 100 120 140 160 180 200601200605200610200701200705200710 Council Cycle of Original R01 NewExperienced

34. A Vision for Peer Review 1.Shorten the review cycle 2.Do more to recruit and retain more high-quality reviewers and decrease the burden on applicants and reviewers

35. CSR’s Growing Need for Reviewers

36. Reviewers – Current Situation Far too many reviewers on study sections Broader science Decrease in reviewer load Unnecessary Too many ad hoc reviewers

37. Near-Term Solutions for Recruiting and Retaining the Best Reviewers Remaking peer review a learning experience Massive call for volunteers Rewards for reviewers Require less travel by expanding peer review platforms Shorten applications

38. CSR Recent Activities in Peer Review

39. Near-Term Solutions for Recruiting and Retaining the Best Reviewers Require less travel by using electronic review modes

40. Expanding Peer Review’s Platforms Study Sections Necessity ● Clinical reviewers Preference ● Physicists, computational biologists New Opportunities ● Fogarty, International Reviewers Our Goal: 10% of all reviews to be electronic in 2007 Electronic Reviews Telephone Enhanced Discussions Video Enhanced Discussions Asynchronous Electronic Discussions

41. A Study Section Chair Talks About Asynchronous Electronic Discussions “Finally, in spite of my initial skepticism regarding this “blog-like” review mechanism, all... [reviewers] produced comments, critiques, posed questions in a fashion that I would judge to be the BEST I’ve seen in ~30 years of chairing various review groups.” Craig M. Jackson, Ph.D. President and Principal Scientist Hemosaga Diagnostics Corp.

42. Near-Term Solutions for Recruiting and Retaining the Best Reviewers Require less travel by using electronic review modes Have Shorter Meetings Shorten Applications

43. Shorter Applications Trans NIH Committee Communication and Support from Societies, Diseases Groups, Scientific Leadership 2 Retreats of NIH Directors Approval and Support of our Advisory Committee Survey of Scientists

44. Initiatives Under Early Discussion

45. Initiatives Under Early Planning/Consideration 12 Pilots with short applications Continuous receipt of applications for chartered study section members Editorial board reviews

46. Deeper Changes? The NIH ACD Subcommittees Locus of review NIH Support for Investigator The Role of Council NIH Portfolio Management Firewall between review and program Scoring Review of deeply innovative, translational and multidisciplinary research Others?

47. This is CSR

48. Growth of R01 Applications Reviewed at CSR vs. Other ICs

49. Growth of R21 Applications Reviewed at CSR vs. Other ICs

50. Major Factors Affecting Payline 1. NIH Budget 2. NIH Budget 3. NIH Budget 4. NIH Budget

51. Major Factors Affecting Payline 1. Number of Application Submitted 2. Targeting of the Budget 3. Recycling of Funds 4. NIH Budget

52. CSR Review of Women Applications, Women Scoring 10 and 20 %

54. NIH 20%tile or Better Scoring (R01s, CSR reviews) CouncilsNew Applications New Investigators (Type 1) New Applications Established Investigators (Type 1) Renewals From Established Investigators (Type 2) May 200115%17%34% May 200215%17%34% May 200315%18%33% May 200414%18%34% May 200515%18%33% CSR New and Established Investigators Score%

55. M. Martin, CSR/NIH/DHHS 17.56 22.07

56. Significant Numbers of Clinical Grantees Are Not Submitting Renewal Applications R01 Applications 2000-2004HS+HS- Type 1 New A024.59%20.20% Type 2 A011.71%17.11% M. Martin, CSR/NIH/DHHS

57. CSR Budget

58. CSR Budget for 2007 CSR Budget $ 60 M SREA Budget $ 40 M Review Cost, including travel for 18,000 reviewers, is less than 0.2% of the budget requested in the applications reviewed

59. Travel Purchase non-refundable (restricted) coach tickets instead of unrestricted tickets Saving – June 2006 through July 2007 # of tickets issued 28,493 Changes requested 2544 cost of changes + fees $550,274 YTD Fare Savings$14,522,877