Archived File The file below has been archived for historical reference purposes only. The content and links are no longer maintained and may be outdated. See the OER Public Archive Home Page for more details about archived files.archived OER Public Archive Home Page

2 CSR INTEGRATED REVIEW GROUP REVIEWS [peer review of peer review] Don Schneider National Institutes of Health U.S. Department of Health and Human Services November 3, 2008

CSR is committed to peer review of peer review Need to respond to changing science Trace roots to Panel on Scientific Boundaries for Review (PSBR) Examine Integrated Review Group-specific issues Discuss general issues 3

The PSBR issued a report January 2000 outlining reorganization philosophy and expectations. URL - tionActivitiesChannel/BackgroundUpdatesandTimel ine/FinalPhase1Report.htm tionActivitiesChannel/BackgroundUpdatesandTimel ine/FinalPhase1Report.htm Be monitored continuously: assess peer review system at regular intervals, at the level of the IRG, and globally by the Advisory Committee 4

Working Groups Assess IRGs [Peer review of peer review] Peer reviewers report on study section operations Feedback from reviewers condensed by staff into a brief 5-10 page white paper report Subset of peer reviewers meet as working group to edit/confirm essence of report Report is shared with Advisory Committee and used as basis of taking action for improvements 5

IRG Working Groups, Round 1, provided baseline, Round 1 working groups – one-cycle visit of each study section, often by single pre- selected reviewer Pre-selection probably resulted in perturbations Nonetheless, useful information obtained that informed reorganization PSBR Reorganization occurred step-wise one IRG at a time from June 2003 to March

IRG Working Groups, Round 2, were initiated in 2005 and are currently in progress Eliminate perturbation –peer reviewers of peer review selected post-study section meetings Solicit input from multiple reviewers each cycle Solicit input from reviewers over multiple, at least two, cycles Share white papers with Chairs prior to reporting to Advisory Committee Provide substantial basis for future actions 7

Schedule for IRG Reviews July 2008 Brain Disorders & Clinical Neuroscience (BDCN) Molecular, Cellular & Developmental Neuroscience (MDCN) September 2008 Integrative, Functional & Cognitive Neuroscience (IFCN) Emerging Technologies & Training in Neuroscience (ETTN) Biology of Development & Aging (BDA) January 2009 Biobehavioral & Biobehavioral Processes (BBBP) Risk, Prevention & Health Behaviors (RPHB) March 2009 Healthcare Delivery & Methodologies (HDM) Epidemiology & Population Sciences (EPS) 8

Cont., Schedule for IRG Reviews May 2009 Immunology (IMM) Infectious Diseases & Microbiology (IDM) July 2009 Oncology 1 – Basic Translational (OBT) Oncology 2 – Translational Clinical (OTC) September 2009 Bioengineering Sciences & Technologies (BST) Surgical Sciences, Biomedical Imaging & Bioengineering (SBIB) November 2009 AIDS & Related Research (AARR) Endocrinology, Metabolism, Nutrition & Reproductive Sciences (EMNR) 9

Cont., Schedule for IRG Reviews January 2010 Vascular & Hematology (VH) Cardiovascular & Respiratory Sciences (CVR) March 2010 Musculoskeletal, Oral & Skin Sciences (MOSS) Digestive, Kidney & Urological Systems (DKUS) May 2010 Genes, Genomes & Genetics (GGG) Biological Chemistry & Macromolecular Biophysics (BCMB) July 2010 Interdisciplinary Molecular Sciences & Training (IMST) Cell Biology (CB) 10

Format for IRG Reviews Late afternoon Present Peer Review Issues & Current Initiatives (CSR Director) Discuss; discuss further over dinner Morning Present Working Group White papers Describe study sections and specific issues/metrics (Chiefs) Discuss and improve study section guidelines (Chairs) Forecast future issues & trends in science Afternoon Consider the critical role of the study section chair (CSR Director, in closed session with Chairs) 11

IRG specific issues and actions are noted Molecular, Cellular & Developmental Neuroscience IRG One study section split for second time Initial attempt to charter a neurotechnology panel led to PRAC requested crosscutting look and eventual formation of two study sections, as well as stimulating formation of crosscutting neuroscience IRG and contributing to momentum for new division 12

Cont. IRG-specific issues & actions Brain Disorders & Clinical Neuroscience IRG Split two study sections, each into two Resolve referral issue so that brain tumor applications focusing on neurological aspects are assigned to BDCN, with those focusing on cancer being assigned to Oncology IRG 13

Cont. IRG-specific issues & actions Integrative, Functional & Cognitive Neuroscience IRG No issues & no actions Biology of Development & Aging IRG Clustering of International & Cooperative Projects in BDA, with great breadth of topics – perhaps pilot editorial board review Emerging Technologies & Training in Neuroscience IRG First applications reviewed winter 2008, no revisions/actions yet One new study section growing rapidly, on verge of needing to split 14

General Review Issues & Actions Reviewers Seniority Ad hoc/temporary Telephone Mail Registry/College Scoring Streamlining/unscoring Score compression Scoring consistency throughout meeting 15

IRG Issues & Enhancing Peer Review Reviewers Flexible service Systematic training Shorter applications and critiques Scoring Smaller scale, 7 or perhaps 9 bins Scores for all applications Promote score spreading, ranking or order of review Emerging scientific trends Selecting appropriate reviewers Updating study section guidelines 16

The End