Archived File The file below has been archived for historical reference purposes only. The content and links are no longer maintained and may be outdated. See the OER Public Archive Home Page for more details about archived files.archivedOER Public Archive Home Page

Center for Scientific Review National Institutes of Health Department of Health and Human Services Toni Scarpa CSR: New Challenges and Opportunities PRAC, May 2006, 2006

CSR Welcomes-- Cheryl Kitt Cheryl Oros Deputy Director Director of Planning, Analysis, and Evaluation Director, Extramural Research, NIAMS Director, Office of Planning and Accountability USDA extramural research program

CSR Progress Report Changes that have Occurred Changes in Progress Changes Under Discussion Challenges: Present and Futures

Time Complexity and Impact CSR Operations Current Systems New Systems? Necessary Changes in CSR Peer Review Operations

Increased Communication and Transparency Within CSR With NIH and other Agencies With the Scientific Community Increase uniformity Summary Statements Posting all within one months of Study Section Posting Summary Statements of new investigators within one week More complete and structured resumes Unscoring Unscoring 50% Increase Efficiency Electronic Submission Text Fingerprinting, Artificial Intelligence Software Changes in CSR Operations

Knowledge Management Tools for Peer Review Collexis Software or Others Knowledge management solutions Fingerprinting and text retrieving Benefits for Peer Review Assigning applications to Integrated Review Groups or Study Sections Selecting reviewers (one application, multiple applications) Major Pilot to Assign Study Sections directly by October Plan to be in Operation by February

Monitoring IRGs and Study Sections 1.Every IRG is been assessed by all senior CSR staff every 2 years in addition to the current 5-year assessments involving all stakeholders. 2.Emergent problems addressed by working groups with leaders from the extramural community, and senior NIH/CSR staff. 3.Substantive issues/changes reviewed by the NIH Peer Review Advisory Committee. 4.Most study sections visited by CSR’s Director and senior staff at least once a year. 5.All retiring study section chairs called by CSR’s Director to learn about problems and possible improvements. 6.All summary statements read by CSR’s Director. 7.Multiple outreach efforts now gather fresh input.

IRG Review Schedule Scheduled IRGs Biology of Development and Aging (BDA) Infectious Diseases and Microbiology (IDM) Biobehavioral and Behavioral Processes (BBBP) Cell Biology (CB) Musculoskeletal, Oral and Skin Sciences (MOSS) Oncological Sciences (ONC) Surgical Sciences, Biomedical Imaging and Bioengineering (SBIB) Respiratory Sciences (RES) Renal and Urological Sciences (RUS) Scheduled IRGs Biological Chemistry and Macromolecular Biophysics (BCMB) Cardiovascular Science (CVS) Bioengineering Sciences and Technologies (BST) AIDS and Related Research (AARR) Risk, Prevention, and Health Behavior (RPHB) Genes, Genomes and Genetics (GGG) Digestive Sciences (DIG) Endocrinology, Metabolism, Nutrition and Reproductive Sciences (EMNR) Brain Disorders and Clinical Neuroscience (BDCN) Integrative, Functional and Cognitive Neuroscience (IFCN) Molecular, Cellular and Developmental Neuroscience (MDCN) Hematology (HEME) Immunology (IMM) Health of the Population (HOP)

Monitoring IRGs and Study Sections 1.Every IRG is been assessed by all senior CSR staff every 2 years in addition to the current 5-year assessments involving all stakeholders. 2.Emergent problems addressed by working groups with leaders from the extramural community, and senior NIH/CSR staff. 3.Substantive issues/changes reviewed by the NIH Peer Review Advisory Committee. 4.Most study sections visited by CSR’s Director and senior staff at least once a year. 5.All retiring study section chairs called to learn about problems and possible improvements. 6.All summary statements read. 7.Multiple outreach efforts now gather fresh input. 8.Open house meetings

Broad Scientific Areas Biomolecular (4): Biological Chemistry and Macromolecular Biophysics (BCMB); Biology of Development and Aging (BDA); Bioengineering Sciences and Technologies (BST); Cell Biology (CB) Control/Regulation (4): Genes, Genomes and Genetics (GGG); Immunology (IMM); Oncological Sciences (ONC); Surgical Sciences, Biomedical Imaging and Bioengineering (SBIB) Integrated Biological (5): Cardiovascular Sciences (CVS); Digestive Sciences (DIG); Musculoskeletal, Oral and Skin Sciences (MOSS); Respiratory Sciences (RES); Renal and Urological Sciences (RUS) Disease-based (5): AIDS and Related Research (AARR); Endocrinology, Metabolism, Nutrition and Reproductive Sciences (EMNR); Hematology (HEME); Infectious Diseases and Microbiology (IDM) Neurological (3): Brain Disorders and Clinical Neuroscience (BDCN); Integrative, Functional and Cognitive Neuroscience (IFCN); Molecular, Cellular and Developmental Neuroscience (MDCN) Behavioral/Social (3): Biobehavioral and Behavioral Processes (BBBP); Health of the Population (HOP); Risk Prevention and Health Behavior (RPHB)

Possible Changes in Current Systems Shorten the review cycle Address concern that clinical research is not properly evaluated Improve the assessment of innovative, high- risk/high-reward research Do more to recruit and retain more high-quality reviewers

Changes in CSR Review Shorten the review cycle

Shortening the NIH Review Cycle, Initial Steps We are conducting a pilot study to speed the review process for new investigators so they may revise and resubmit for the very next review cycle, 4 months earlier than before (effective Feb ‘06). We are posting Summary Statements within one month after the Study Section meeting, instead of two to three months after the meeting (effective Oct 05)

Changes in CSR Review Shorten the review cycle Address concern that clinical research is not properly evaluated

M. Martin, CSR/NIH/DHHS

Distribution Among Different Types of R01 Applications: All HS+ or HS- Applications – HS+HS- Type1NewA024.59%20.20% Type1NewA19.20%7.68% Type1NewA22.30%1.94% Type1ExpA028.12%27.58% Type1ExpA112.57%11.55% Type1ExpA23.62%3.40% Type2A011.71%17.11% Type2A15.95%7.90% Type2A21.96%2.65% Total100% M. Martin, CSR/NIH/DHHS

Some Observations On a percent basis, HS+ PIs are more unlikely to apply for a type 2 A0 or A1 application Funded type 1 new HS+ PIs that do not submit a type 2 are less likely to submit for another activity than HS- funded PIs NIH is loosing successful, funded, HS+ PIs at a greater rate than HS- PIs [40% vs. 30%]

Desirable Changes in CSR Review Shorten the review cycle Address concern that clinical research is not properly evaluated Improve the assessment of innovative, high- risk/high-reward research

Possible Changes in Current Systems Shorten the review cycle Address concern that clinical research is not properly evaluated Improve the assessment of innovative, high- risk/high-reward research Do more to recruit and retain more high-quality reviewers

CSR Applications Reviewed, Regular and SEP May Council Only

Number of Research Grant Applications/Applicant

Average Number of Applications Per Reviewer October Council Only

Expanding Peer Review’s Platforms Electronic Reviews Telephone Enhanced Discussions Video Enhanced Discussions Asynchronous Electronic Discussions Study Sections Necessity ● Clinical reviewers Preference ● Physicists, computational biologists New Opportunities ● Fogarty, International Reviewers 10% of all reviews by the end of the year.

Challenges to Recruiting Reviewers Decrease the number of reviewers and increase the quality Increase the number of applications reviewed without extra workload Recruit and retain the best reviewers

Possible Short Term Approaches for Increasing Efficiency for Reviewers and CSR Replace Many SEPs with Smaller Parallel Study Sections Enlarge Study Section Membership and Decrease Frequency of Participation Convene Pre-meetings to Streamline Use Various Review Platforms Use Hybrid Review Platforms Unscore 40% of the F32s (postdoctoral fellowships) Shorten Applications Create More Structured Applications and Reviews

Size of Grant Applications RO1 Will increase number of applications reviewed by reviewers Will decrease the number of reviewers in a study section May be combined with a change in format of the application, more consonant to review criteria Strong support by councils and scientific leadership May result in more innovation

Trans-NIH Committee to Shorten the Application Chairs Robert Finkelstein, Ph.D. (Co-Chair), NINDS Donald Schneider, Ph.D. (Co-Chair), CSR Members Mary Custer, Ph.D. CSR Ann A. Hagan, Ph.D. NIGMS Craig A. Jordan, Ph.D. NIDCD Sherry Mills, M.D., M.P.H. OER Philip F. Smith, Ph.D. NIDDK Betsy Wilder, Ph.D. NIDDK’ Alan L. Willard, Ph.D. NINDS

This is CSR