Archived File The file below has been archived for historical reference purposes only. The content and links are no longer maintained and may be outdated. See the OER Public Archive Home Page for more details about archived files.archivedOER Public Archive Home Page

Improving Peer Review CSR Initiatives toni scarpa PRAC Nov. 3, 2008 National Institutes of Health U.S. Department of Health and Human Services

 New Data and the Drivers for Change  CSR’s Efforts to Enhance Peer Review  The NIH Director’s Peer Review Initiatives Improving Peer Review

New Data and the Drivers for Change

CSR Peer Review: ,000 applications received 56,000 applications reviewed 16,000 reviewers 240 Scientific Review Officers 1,800 review meetings

$13.7 $15.6 $17.8 $20.5 $23.3 $27.1 $28.0 $28.6 $29.1 $29.5 $0 $5 $10 $15 $20 $25 $ st Driver: The NIH Budget Doubling $ In Billions

2 nd Driver: Number of Applications Submitted Historical Growth

SBIR Applications INST CSR

R01 Applications Reviewed at CSR vs. Other ICs

R21 Applications Reviewed at CSR vs. Other ICs

3 rd Driver: Reviewer’s Load Applications Per Reviewer October Council Rounds

4th Driver: CSR Budget $ Millions

4th Driver: CSR Budget $ Millions

Annual Savings in Reviewers’ Expenses Budget Non-refundable tickets with one possible change $15 million 3,000 fewer reviewers $3 million 15% reviews using electronic platforms $5 million One meeting a year on the West Coast $1.8 million

Next Savings? Replace shipping of CDs with online downloading Cost of 18,000 CDs $750,000 Cost of shipping$200,000 Handling $50,000 5% more reviews using electronic platform SREA budget $1,500,000 CSR budget -$100,000 More meetings in Baltimore Savings/100 meetings$500,000 Review of fellowship applications

CSR’s Efforts to Enhance Peer Review

1.Reorganizing CSR’s Review Groups 2.Recruiting CSR Staff 3.Shortening the Review Cycle 4.Holding Out-of-Town Meetings 5.Recruiting the Best Reviewers 6.Performing a Second Cycle of IRG Reviews 7.Revising Study Section Guidelines 8.Training Chairs 9.Managing Conflicts of Interest 10.Advancing Additional Review Platforms and Processes CSR’s Efforts to Enhance Peer Review

1.Reorganizing CSR’s Review Groups

1. CSR Organization of Peer Review DivisionsScientific Review OfficersIntegrated Review Groups

1. CSR Reorganization Rene Etcheberrigaray Translational and Clinical Sciences Cardiovascular and Respiratory Sciences Surgical Sciences, Biomedical Imaging and Bioengineering Musculoskeletal, Oral And Skin Sciences Oncology 2 – Translational Clinical Vascular and Hematology Physiological and Pathological Sciences Endocrinology, Metabolism, Nutrition & Reproductive Sciences Immunology Infectious Diseases & Microbiology Digestive, Kidney & Urological Systems Neuroscience, Development and Aging Brain Disorders & Clinical Neuroscience Molecular, Cellular & Developmental Neuroscience Integrative, Functional & Cognitive Neuroscience Emerging Technologies & Training in Neuroscience Biology of Development & Biobehavioral & Behavioral Processes Risk, Prevention& Health Behaviors Epidemiology & Population Sciences Healthcare Delivery & Methodologies AIDS & Related Research AIDS, Behavioral and Population Sciences Basic and Integrative Biological Sciences Biological Chemistry & Macromolecular Biophysics Bioengineering Sciences & Technologies Genes, Genomes &Genetics Oncology 1 – Basic Translational Cell Biology Interdisciplinary Molecular & Training Don Schneider

Update on CSR Realignments Don Schneider Update on CSR Realignments Don Schneider 1. CSR Organization of Peer Review

CSR’s Efforts to Enhance Peer Review 1.Reorganizing CSR’s Review Groups 2.Recruiting CSR Staff

3 Division Directors 6 Integrated Review Chiefs 20 Scientific Review Officers

CSR’s Efforts to Enhance Peer Review 1.Reorganizing CSR’s Review Groups 2.Recruiting CSR Staff 3.Shortening the Review Cycle

3. Shortening the Review Cycle The Goal To provide applicants a review and score within 3 months of application submission. This will permit resubmission of applications (when doable and desirable) 4 months earlier than in the past.

RO1 A1 Resubmission Within 4 Months of Original Application

CSR’s Efforts to Enhance Peer Review 1.Reorganizing CSR’s Review Groups 2.Recruiting CSR Staff 3.Shortening the Review Cycle 4.Holding Out-of-Town-Meetings

4. Holding Out-of-Town Meetings To facilitate participation from colleagues on the West Coast, one meeting out of three each year will be held in the metro areas of: o Los Angeles o San Francisco o Seattle o Chicago Benefits Great satisfaction Significant savings Ranking of hotels

5. Near-Term Solutions for Recruiting and Retaining the Best Reviewers Move a meeting a year to the West Coast Develop a national registry of volunteer reviewers Searchable database with 3,500 reviewers Provide tangible rewards for reviewers No submission deadlines for chartered members of study sections (effective February 2008). 1,574 chartered members used flexible deadlines during the last 6 months Provide flexible time for reviewers Choice of 3 times/year for 4 years or 2 times/year for 6 years

CSR’s Efforts to Enhance Peer Review 1.Reorganizing CSR’s Review Groups 2.Recruiting CSR Staff 3.Shortening the Review Cycle 4.Out-of-Town Meetings 5.Recruiting the Best Reviewers 6.Performing a Second Cycle of IRG Reviews

6. Second Cycle of IRG Reviews Scheduled IRGsScheduled IRGs (Continued) Brain Disorders and Clinical Neuroscience (BDCN) and Molecular, Cellular and Developmental Neuroscience (MDCN) Bioengineering Sciences and Technologies (BST) andSurgical Sciences, Biomedical Imaging and Bioengineering (SBIB) Biology of Development and Aging (BDA), Emerging Technologies and Training in Neuroscience (ETTN) andIntegrative, Functional and Cognitive Neuroscience (IFCN) AIDS and Related Research (AARR) and Endocrinology, Metabolism, Nutrition and Reproductive Sciences (EMNR) Scheduled IRGsScheduled IRGs Biobehavioral and Behavioral Processes (BBB) and Risk, Prevention and Health Behavior (RPHB) Vascular and Hematology (VH) and Cardiovascular and Respiratory Sciences (CVR) Healthcare Delivery and Methodologies (HDM) and Epidemiology and Population Sciences (EPS) Musculoskeletal, Oral and Skin Sciences (MOSS) and Digestive, Kidney and Urological Systems (DKUS) Immunology (IMM) and Infectious Diseases and Microbiology (IDM) Genes, Genomes and Genetics (GGG) and Biological, Chemical and Macromolecular Biophysics (BCMB) Oncology 1 — Basic and Translational (OBT) and Oncology 2 — Translational Clinical (OTC) Interdisciplinary Molecular Sciences and Training (IMST) and Cell Biology (CB)

6. Performing a Second Cycle of IRG Reviews All pertinent study section chairs were invited Advice on study section and IRG Revision of study sections description/outline Training of the chair

6. Performing a Second Cycle of IRG Reviews Second Cycle of IRG Reviews Don Schneider

CSR’s Efforts to Enhance Peer Review 1.Reorganizing CSR’s Review Groups 2.Recruiting CSR Staff 3.Shortening the Review Cycle 4.Holding Out-of-Town Meetings 5.Recruiting the Best Reviewers 6.Performing a Second Cycle of IRG Reviews 7.Revising Study Section Guidelines

7. Revising Study Section Outlines Cellular Signaling and Regulatory Systems [Roster] The Cellular Signaling and Regulatory Systems (CSRS) study section reviews applications that focus on the initiation and execution of programs that control cellular homeostasis and physiology. A distinguishing characteristic of these applications is an emphasis on signaling networks and the coordination of processes related to cell proliferation, survival, and growth. Cell cycle regulation, mitosis, meiosis, checkpoint controls and regulation by ubiquitination Proteolytic mechanisms associated with cell cycle, senescence and death Programmed cell death and apoptosis, particularly their regulation in the context of stress, growth, and transformation. Proliferation and growth control by the nucleus; signaling pathways regulating transcription Integrative cell physiology, e.g., stress, clocks, cellular modeling; cell differentiation and transformation Basic studies of cytokine signaling Application of state-of-the-art technologies such as imaging and computational modeling of cellular signaling networks Study sections with most closely related areas of similar science listed in rank order are: Molecular and Integrative Signal Transduction Intercellular Interactions Membrane Biology and Protein Processing Molecular Genetics A Molecular Genetics B

CSR’s Efforts to Enhance Peer Review 1.Reorganizing CSR’s Review Groups 2.Recruiting CSR Staff 3.Shortening the Review Cycle 4.Holding Out-of-Town Meetings 5.Recruiting the Best Reviewers 6.Performing a Second Cycle of IRG Reviews 7.Revising Study Section Guidelines 8.Training Chairs

8. Training Study Section Chairs Scope and Schedule 120 new chairs every year, more if “permanent” SEPs chairs are included 5 meetings in January 2009 for chairs recently appointed 5 meetings in July 2009 for chairs appointed next June Training Program Share data Explain the new changes and the significance Share the best practice (ours and theirs) Answer questions and address concerns Make Chairs more effective stakeholders

CSR’s Efforts to Enhance Peer Review 1.Reorganizing CSR’s Review Groups 2.Recruiting CSR Staff 3.Shortening the Review Cycle 4.Holding Out-of-Town Meetings 5.Recruiting the Best Reviewers 6.Performing a Second Cycle of IRG Reviews 7.Revised Study Section Guidelines 8.Training of Chairs 9.Managing Conflicts of Interest

Report of Conflict of Interest Committee Noni Byrnes

CSR’s Efforts to Enhance Peer Review 1.Reorganizing CSR’s Review Groups 2.Recruiting CSR Staff 3.Shortening the Review Cycle 4.Holding Out-of-Town Meetings 5.Recruiting Reviewers 6.Performing a Second Cycle of IRG Review 7.Revising Study Section Guidelines 8.Training of Chairs 9.Managing Conflicts of interest 10.Advancing Additional Review Platforms and Processes

10. Advancing Additional Review Platforms and Process Video (VED) Internet (AED) Editorial Board Order of Discussion

10. Advancing Additional Review Platforms and Processes Editorial Board Review oHeavy triage based on innovation and potential science transformation by a small study section of distinguished, broad-science reviewers (the editors) oSpecific science reviewed by appropriate reviewers (the editorial board) oFinal ranking by the editors

10. Advancing Additional Review Platforms and Process New Transformative R01 Betsy Wilder

10. Advancing Additional Review Platforms and Process Results of Asynchronous Electronic Discussion Survey Andrea Kopstein

NIH Enhancement of Peer Review The Charge from Dr. Zerhouni: “Fund the best science, by the best scientists, with the least administrative burden…” with the least administrative burden…” Two advisory committees to the NIH Director

The NIH Director’s Peer Review Initiatives

Corporate NIH Peer Review Enhancement: Implementation Raynard Kington

This is CSR September 2008

Recognizing Outstanding Reviewers Going above and beyond the call of duty Having a significant impact on the quality of peer review Educating and inspiring other reviewers Setting the highest standard of peer review excellence

The First Marcy Speer Award, 2007 Marcy Speer

2008 Marcy Speer Outstanding CSR Reviewer Award Dr. David J. Sahn, M.D. Oregon Health & Science University

The 2 nd Marcy Speer Award, 2008