An Overview of Peer Review at CSR – Critical Do’s and Don’ts Joy Gibson, D.Sc. Director, Division of Translational and Clinical Sciences American Association for Thoracic Surgery March 4, 2011 National Institutes of Health U.S. Department of Health and Human Services

National Institutes of Health National Institute on Alcohol Abuse and Alcoholism National Institute on Alcohol Abuse and Alcoholism National Institute of Arthritis and Musculoskeletal and Skin Diseases National Institute of Arthritis and Musculoskeletal and Skin Diseases National Cancer Institute National Cancer Institute National Institute on Drug Abuse National Institute on Drug Abuse National Institute of Environmental Health Sciences National Institute of Environmental Health Sciences National Institute on Aging National Institute on Aging National Institute of Child Health and Human Development National Institute of Child Health and Human Development National Institute on Deafness and Other Communication Disorders National Institute on Deafness and Other Communication Disorders National Eye Institute National Eye Institute National Human Genome Research Institute National Human Genome Research Institute National Institute of Mental Health National Institute of Mental Health National Institute of Neurological Disorders and Stroke National Institute of Neurological Disorders and Stroke National Institute of General Medical Sciences National Institute of General Medical Sciences National Institute of Nursing Research National Institute of Nursing Research National Library of Medicine National Library of Medicine Center for Scientific Review Center for Scientific Review National Center for Complementary and Alternative Medicine National Center for Complementary and Alternative Medicine National Institute of Allergy and Infectious Diseases National Institute of Allergy and Infectious Diseases Fogarty International Center Fogarty International Center National Center for Research Resources National Center for Research Resources Clinical Center Clinical Center National Center on Minority Health and Health Disparities National Center on Minority Health and Health Disparities National Institute of Biomedical Imaging and Bioengineering National Institute of Biomedical Imaging and Bioengineering Office of the Director Center for Information Technology Center for Information Technology National Heart, Lung, and Blood Institute National Heart, Lung, and Blood Institute National Institute of Dental and Craniofacial Research National Institute of Dental and Craniofacial Research National Institute of Diabetes and Digestive and Kidney Diseases National Institute of Diabetes and Digestive and Kidney Diseases

National Institutes of Health Center for Scientific Review Study Section Institute Advisory Councils and Boards Institute Director Assigns to IC & I IRG/Study Section Reviews for Scientific Merit Evaluates for Relevance Recommends Action Takes Final Action Initiates Research Idea Conducts Research Allocates Funds Submits Application Review Process for a Research Grant Research Grant Application School or Research Center

Scientific Review Process Dual Review System for Grant Applications Second Level of Review NIH Institute/Center Council NIH Institute/Center Council Second Level of Review NIH Institute/Center Council NIH Institute/Center Council First Level of Review CSR or Institute Review Scientific Review Group (Study Section) First Level of Review CSR or Institute Review Scientific Review Group (Study Section)

Receives all NIH applications Refers them to NIH Institutes/Centers and to scientific review groups Reviews majority of grant applications for scientific merit Your Application Goes to the NIH Center for Scientific Review (CSR) Focal Point for Initial Review at NIH

Translational and Clinical Sciences Translational and Clinical Sciences Cardiovascular and Respiratory Sciences Cardiovascular and Respiratory Sciences Surgical Sciences, Biomedical Imaging and Bioengineering Surgical Sciences, Biomedical Imaging and Bioengineering Musculoskeletal, Oral And Skin Sciences Musculoskeletal, Oral And Skin Sciences Oncology 2 – Translational Clinical Oncology 2 – Translational Clinical Vascular and Hematology Hematology Physiological and Pathological Sciences Physiological and Pathological Sciences Endocrinology,Metabolism, Nutrition & Reproductive Sciences Reproductive Sciences Endocrinology,Metabolism, Nutrition & Reproductive Sciences Reproductive Sciences Immunology Immunology Infectious Diseases & Microbiology Infectious Diseases & Microbiology Digestive, Kidney & Urological Systems Digestive, Kidney & Urological Systems Neuroscience, Development and Aging Neuroscience, Development and Aging Brain Disorders & Clinical Neuroscience Brain Disorders & Clinical Neuroscience Molecular, Cellular & Developmental Neuroscienc Developmental Neuroscienc e Molecular, Cellular & Developmental Neuroscienc Developmental Neuroscienc e Integrative, Functional & Cognitive Neuroscience Integrative, Functional & Cognitive Neuroscience Emerging Technologies & Training in Neuroscience Emerging Technologies & Training in Neuroscience Biology of Development and Aging Biology of Development and Aging Biobehavioral & Biobehavioral & Behavioral Processes Behavioral Processes Biobehavioral & Biobehavioral & Behavioral Processes Behavioral Processes Risk, Prevention& Health Behaviors Risk, Prevention& Health Behaviors Epidemiology & Population Sciences Epidemiology & Population Sciences Healthcare Delivery & Methodologies Healthcare Delivery & Methodologies AIDS & Related Research AIDS & Related Research AIDS, Behavioral and Population Science s AIDS, Behavioral and Population Science s Basic and Integrative Biological Sciences Basic and Integrative Biological Sciences Biological Chemistry & MacromolecularBiophysics MacromolecularBiophysics Bioengineering Sciences & Technologies Bioengineering Sciences & Technologies Genes, Genomes & Genetics Genes, Genomes & Genetics Oncology 1 – Basic Translational Translational Cell Biology InterdisciplinaryMolecular & Training InterdisciplinaryMolecular & Training CSR Review Divisions

Help Get Your Application to the Right Study Section Review CSR Integrated Review Group and Scientific Review Group (Study Section) guidelines to identify a home for your application. Submit a Cover Letter!

Help Get Your Application to the Right Study Section

Help Get Your Application to the Right Study Section Integrated Review Group

Help Get Your Application to the Right Study Section Study Section

Cardiovascular and Respiratory Sciences IRG Cardiac Contractility, Hypertrophy and Failure Cardiovascular Differentiation and Development (CDD) Clinical and Integrative Cardiovascular Sciences (CICS) Electrical Signaling, Ion Transport and Arrhythmias (ESTA) Myocardial Ischemia and Metabolism (MIM) Lung Injury, Repair and Remodeling (LIRR) Lung Cellular, Molecular and Immunobiology Respiratory Integrative Biology and Translational Research (RIBT)

Surgical, Biomedical Imaging and Bioengineering IRG Clinical, Molecular Imaging and Probe Development (CMIP) Medical Imaging (MEDI) Surgery, Anesthesiology and Trauma (SAT) Bioengineering, Technology and Surgical Sciences (BTSS) Biomedical Imaging Technology A & B (BMIT)

Division of Translational and Clinical Sciences (DTCS) Trans-IRG Imaging SEP Goal - enhance review of multidisciplinary, translational research with a focus on imaging Transformative in nature (translational research) Include humans Use imaging as a measure of diagnosis, intervention or treatment success/efficacy Editorial Board (Two Stage) Review First stage – 3 mail reviews (technical focus) Second stage – 3 clinical impact reviews Crosses disciplines of cancer, cardiovascular, musculoskeletal and neuroscience

Submit a Cover Letter The cover letter should be used for a number of important purposes: Suggest Institute/Center assignment Suggest review assignment Identify individuals in conflict Identify areas of expertise needed to evaluate the application Discuss any special situations It is NOT appropriate to use the cover letter to suggest specific reviewers.

Peer Review in CSR CSR Study Sections are managed by a Scientific Review Officer (SRO) who is a doctoral-level professional, whose scientific background is close to the focus of the study section. Each CSR standing study section has regular members who are from the scientific community. Temporary members are recruited as needed. About applications are normally reviewed at each study section meeting.

The Role of Your Scientific Review Officer Designated Federal Official responsible for the overall review process

Your Study Section Meeting

Your Application Is Reviewed In Order Clustered and Ordered for Fairness New and Early Stage Investigator applications are usually reviewed first Applications in each cluster are reviewed in order of their preliminary scores

New Investigator (NI):New Investigator (NI): PD/PI who has not yet competed successfully for a substantial NIH research grant oFor multiple PD/PIs-all PD/PIs must meet requirements for NI status Early Stage Investigator (ESI):Early Stage Investigator (ESI): PD/PI who qualifies as a New Investigator AND is within 10 years of completing the terminal research degree or is within 10 years of completing medical residency (or equivalent)

Reviewers discuss about % of the applications The panel will discuss any application a reviewer wants to discuss Discussions Focus on the Best Applications

Additional Review Platforms Helps to recruit reviewers Electronic review modes reduce travel Electronic Reviews Telephone Assisted Meetings Video Assisted Meetings Telepresence Internet Assisted Meetings Editorial Board Review – multidisciplinary

Review Criteria Overall Impact: exert a sustained, powerful influence on the research field(s) involved Assessment of the likelihood for the project to exert a sustained, powerful influence on the research field(s) involved Core Criteria Significance Investigator(s) Innovation Approach Environment oReview criteria each scored from 1-9

How Your Application Is Aligned to the Review CriteriaApplication Section Significance Research Strategy a. Significance Investigator(s) Biosketch Personal Statement Innovation Research Strategy b. Innovation Approach Research Strategy c. Approach Environment Resources Environment

9-Point Scoring Scale ImpactScoreDescriptor High Impact 1Exceptional 2Outstanding 3Excellent Medium Impact 4Very Good 5Good 6Satisfactory Low Impact 7Fair 8Marginal 9Poor

Scores for each review criterion Essentially unedited critiques Administrative notes if any If your application is discussed, you also will receive: An overall impact/priority score and percentile ranking An Summary of review discussion Budget recommendations Your Summary Statement

Summary Statement New Indicator for Early Stage Investigators Percentile: 29 Percentile in whole numbers Impact/Priority Score range Program Officer

Read instructions Never assume that reviewers will know what you mean Refer to literature thoroughly State rationale of proposed investigation Include well-designed tables and figures Present an organized, lucid write-up Obtain pre-review from faculty at your institution NIH Grant Writing Tips When Preparing an Application

Additional Factors to Consider Focus on significance/impact Make it exciting Be very clear Do not assume too much Have realistic aims and timelines -- Don’t be too ambitious Be brief with things that everybody knows Note the study’s limitations Proofread the application

Who Can Answer Your Questions? Before You Submit Your Application A Program Officer at an NIH Institute or Center After You Submit Your Scientific Review Officer After Your Review Your Assigned Program Officer

Key NIH Review and Grants Web Sites NIH Center for Scientific Review NIH Office of Extramural Research

Helpful Handouts Insiders Guide What Happens to NIH Grant Application to Peer Review Your Grant Application Useful Web Links