Archived File The file below has been archived for historical reference purposes only. The content and links are no longer maintained and may be outdated. See the OER Public Archive Home Page for more details about archived files.archived OER Public Archive Home Page

Improving Peer Review CSR Initiatives toni scarpa PRAC, April 30, 2008 National Institutes of Health U.S. Department of Health and Human Services

 New Data  Improving Peer Review: CSR Initiatives Changes In CSR Organization Realigning CSR Peer Review Peer Review at CSR

New Data

Number of Applications Submitted Historical Growth

SBIR Applications INST CSR

Success Rates of Competing SBIR Applications Fiscal Year Percent Success Rate 20 SBIR Phase I SBIR Phase II

R01 Applications Reviewed at CSR vs. Other ICs

R21 Applications Reviewed at CSR vs. Other ICs

% of RO1 Application Submitted, Type 1

% of RO1 Application Submitted, Type 2

Reviewer’s Load Applications Per Reviewer Oct Council Rounds

Colleagues Reviewing for CSR

CSR Budget for 2008 CSR Budget $ 63 M SREA (Reviewers Expenses Budget)$ 39 M Review Cost, including travel for 17,000 reviewers, is less than 0.2% of the budget requested in the applications reviewed

SREA Budget Progression Millions

Travel Purchase non-refundable (restricted) coach tickets instead of unrestricted tickets Preliminary Pilot Data – June 2006 through July 2007 # of tickets issued 28,493 Changes requested 2,544 cost of changes + fees $550,274 YTD Fare Savings$14,522,877

Additional Savings AirfareHotelMeeting Average Cost Washington $ 7,682 $ 19,166 $ 26,848 West Coast $ 6,486 $ 8,974 $ 15,464

Improving Peer Review CSR Initiatives

Improving Peer Review Matrix A : Changes in CSR Operations Matrix B : Changes in Peer Review Matrix C : System Changes?

1.CSR Reorganization of Division and IRGs 2.Review Enabling Committees 3.Increase Efficiency 4.Division Directors, Chiefs and SROs Recruitment Matrix A : Changes in CSR Operations

1.CSR Reorganization of Division and IRGs Matrix A : Changes in CSR Operations

1.CSR Reorganization of Division and IRGs Realign Divisions and IRGs with the changes in science Increase efficiency and effectiveness Increase consistency Recruitment of Scientific Review Officers Matrix A. Changes in CSR Operations

New Division Neuroscience, Aging and Development BDCN/Brain Disorders & Clinical Neuroscience (est. 11 SRAs) IFCN/Integrative, Functional, & Cognitive Neuroscience (est. 9 SRAs) MDCN/Molecular, Cellular & Developmental Neuroscience (est. 10 SRAs) ENST/Emerging Neuroscience & Training (est. 8 SRAs) BDA/Biology of Development & Aging (7 SRAs)

Organization of Peer Review at CSR Divisions Study Sections I.R.G.s (Integrated Review Groups)

New CSR Review Divisions Division A (Neuroscience, Development and Aging) Brain Disorders and Clinical Neuroscience IRG (BDCN) Molecular, Cellular and Developmental Neuroscience IRG (MDCN) Integrative, Functional and Cognitive Neuroscience IRG (IFCN) Emerging Technologies and Training in Neuroscience IRG (ETTN) Biology of Development and Aging IRG (BDA) Biobehavioral and Behavioral Processes IRG ( BBBP) Risk, Prevention and Health Behaviors IRG (RPHB) Epidemiology and Population Sciences IRG (EPS) Healthcare Delivery and Methodologies IRG (HDM) AIDS and Related Research IRG (AARR) Division B (AIDS, Behavioral and Population Sciences) Division C (Basic and Integrative Biological Sciences) Biological Chemical and Macromolecular Biophysics IRG (BCMB) Bioengineering Sciences and Technologies IRG (BST) Genes, Genomes and Genetics IRG (GGG) Oncology 1 – Basic Translational IRG (OBT) Cell Biology IRG (CB) Interdisciplinary Molecular Sciences and Training IRG (IMST) Division D (Physiological and Pathological Sciences) Endocrinology, Metabolism, Nutrition and Reproductive Sciences IRG (EMNR) Immunology IRG (IMM) Infectious Diseases and Microbiology IRG (IDM) Digestive, Kidney and Urological Systems IRG (DKUS) Division E (Translational and Clinical Sciences) Cardiovascular and Respiratory Sciences IRG (CVR) Surgical Sciences, Biomedical Imaging and Bioengineering IRG (SBIB) Musculoskeletal, Oral And Skin Sciences IRG (MOSS) Oncology 2 – Translational Clinical IRG (OTC) Vascular and Hematology IRG (VH) Scientific Review Groups= 48 Scientific Review Groups= 50 Scientific Review Groups= 43 Scientific Review Groups= 44 Scientific Review Groups= 55

Don Schneider

1.CSR Reorganization of Division and IRGs 2.Review Enabling Committees Matrix A : Changes in CSR Operations

Review Enabling Committees Policy (Martin) Scientific Community Outreach (Schneider) Recruitment of SROs (Gibson) Recruitment of Reviewers (Stassi) Knowledge Management (Many) Peer Review Electronic (Many) SRA Training (Freund) Mentoring and staff development (Many) Best review practices (Kitt)

1.CSR Reorganization of Division and IRGs 2.Review Enabling Committees 3.Increase Efficiency Matrix A : Changes in CSR Operations

Increase Efficiency How we distribute 80,000 applications? Retooled for electronic submission Applications are now submitted electronically Assign applications using text fingerprinting Implementation by August 2008

1.CSR Reorganization of Division and IRGs 2.Review Enabling Committees 3.Increase Efficiency 4.Division Directors, Chiefs and SROs Recruitment Matrix A : Changes in CSR Operations

Recruitment 3 Division Directors First Search (Richard Nakamura, Chair) 6 Integrated Review Chiefs 40 Scientific Review Officers Matrix A : Changes in CSR Operations

Matrix B : Changes in Peer Review 1.Improve study section alignment and performance 2.Shorten the review cycle 3.Do more to recruit and retain more high-quality reviewers and decrease the burden on applicants and reviewers 4.Improve the identification of significant, innovative and high-impact research

CCSR Recent Activities in Peer Review SR Recent Activities in Peer Review 1.Improve Study Section Alignment and Performance

Second Round of Internal IRG Review July 24, 2008: BDCN and MDCN September 15, 2008: ETTN, BDA and IFCN January 2009: BBBP and HOP 2 (HDM) March 2009: RPHB and HOP 1 (EPS) May 2009: IMM and IDM July 2009: ONC 1-Basic and ONC 2-Clinical September 2009: BST and SBIB November 2009: AARR and EMNR January 2010: VH and CVR March 2010: MOSS and DRUS May 2010: GGG and BCMB July 2010: IMST and CB

Two Major Challenges Percentiling or Ranking Applications Reviewed in SEPS Locus of Review of “Orphan Applications”

Don Schneider Clustering of grant application for review

CCSR Recent Activities in Peer Review SR Recent Activities in Peer Review 1.Improve Study Section Alignment and Performance 2. Shorten the review cycle

Shortening the Review Cycle The Goal To provide applicants a review and score within 3 months of application submission. This will permit resubmission of applications (when doable and desirable) 4 months earlier than in the past. To permit 3 reviews within one year

R01 A1 Resubmissions Reviewed in the Cycle Immediately following the Original Application's Review Council Cycle of Original R01 NewExperienced

CCSR Recent Activities in Peer Review SR Recent Activities in Peer Review 1.Improve Study Section Alignment and Performance 2. Shorten the review cycle 3.Do more to recruit and retain more high-quality reviewers and decrease the burden on applicants and reviewers

Colleagues Reviewing for CSR

Academic Rank of All CSR Reviewers

Near-Term Solutions for Recruiting and Retaining the Best Reviewers Remake peer review a learning experience Develop a national registry of volunteer reviewers Searchable Database with 2700 reviewers Move a meeting/year to the west coast 50 % this year, 90% next year Provide tangible rewards for reviewers No submission deadlines for permanent members of Study Sections (effective February 2008) Shorten applications Require less travel by expanding peer review platforms

Expanding Peer Review’s Platforms Study Sections Necessity ● Clinical reviewers Preference ● Physicists, computational biologists New Opportunities ● Fogarty, International Reviewers Goal: 15% of all reviews to be electronic in 2008 Electronic Reviews Video Enhanced Discussions Asynchronous Electronic Discussions

Matrix C: System Changes? The Advisory Committees of NIH Director

Advisory Committees to the NIH Director The Charge from Dr. Zerhouni: “Fund the best science, by the best scientists, with the least administrative burden…” with the least administrative burden…” A Self-Study by the NIH in Partnership with the Scientific Community to Strengthen Peer Review in Changing Times

This is CSR