The Grant Renewal Review Process Nywana Sizemore, PhD Scientific Review Officer Molecular Oncogenesis - MONC Oncology I - Basic Translational - OBT Integrated Review Group National Cancer Institute-NCI New Grantee Workshop Bethesda, MD October 18, 2010 National Institutes of Health U.S. Department of Health and Human Services

National Institutes of Health Much of the biomedical research in the United States is supported by the Federal Government, primarily the National Institutes of Health (NIH)

National Institutes of Health National Institute on Alcohol Abuse and Alcoholism National Institute on Alcohol Abuse and Alcoholism National Institute of Arthritis and Musculoskeletal and Skin Diseases National Institute of Arthritis and Musculoskeletal and Skin Diseases National Cancer Institute National Cancer Institute National Institute on Drug Abuse National Institute on Drug Abuse National Institute of Environmental Health Sciences National Institute of Environmental Health Sciences National Institute on Aging National Institute on Aging Eunice Kennedy Shriver National Institute of Child Health and Human Development Eunice Kennedy Shriver National Institute of Child Health and Human Development National Institute on Deafness and Other Communication Disorders National Institute on Deafness and Other Communication Disorders National Eye Institute National Eye Institute National Human Genome Research Institute National Human Genome Research Institute National Institute of Mental Health National Institute of Mental Health National Institute of Neurological Disorders and Stroke National Institute of Neurological Disorders and Stroke National Institute of General Medical Sciences National Institute of General Medical Sciences National Institute of Nursing Research National Institute of Nursing Research National Library of Medicine National Library of Medicine Center for Scientific Review Center for Scientific Review National Center for Complementary and Alternative Medicine National Center for Complementary and Alternative Medicine National Institute of Allergy and Infectious Diseases National Institute of Allergy and Infectious Diseases John E. Fogarty International Center John E. Fogarty International Center National Center for Research Resources National Center for Research Resources Clinical Center Clinical Center National Center on Minority Health and Health Disparities National Center on Minority Health and Health Disparities National Institute of Biomedical Imaging and Bioengineering National Institute of Biomedical Imaging and Bioengineering Office of the Director Center for Information Technology Center for Information Technology National Heart, Lung, and Blood Institute National Heart, Lung, and Blood Institute National Institute of Dental and Craniofacial Research National Institute of Dental and Craniofacial Research National Institute of Diabetes and Digestive and Kidney Diseases National Institute of Diabetes and Digestive and Kidney Diseases

Receives all NIH applications Refers them to NIH Institutes/Centers and to scientific review groups Reviews majority of grant applications for scientific merit Your Application Goes to the NIH Center for Scientific Review (CSR) Focal Point for Initial Review at NIH

CSR Mission Statement To see that NIH grant applications receive fair, independent, expert, and timely reviews – free from inappropriate influences – so NIH can fund the most promising research.

First Level of Review CSR or Institute Review Scientific Review Group (Study Section) First Level of Review CSR or Institute Review Scientific Review Group (Study Section) Second Level of Review NIH Institute/Center Council Second Level of Review NIH Institute/Center Council Scientific Review Process Dual Review System for Grant Applications

National Institutes of Health Center for Scientific Review Study Section Institute Advisory Councils and Boards Institute Director Assigns to IC & IRG/Study Section Reviews for Scientific Merit Evaluates for Relevance Recommends Action Takes Final Action Research Grant Application School or Other Research Center Initiates Research Idea Conducts Research Allocates Funds Submits Application Review Process for a Research Grant

JanFebMarAprMayJunJulAugSepOctNovDecJanFebMarAprMayJunJul Cycle 1 Receipt Referral Council Review Award Cycle 2 Receipt Referral Council Review Award Cycle 3 Receipt Referral Council Review Award There are three overlapping cycles per year Overall Timeframe from Submission to Award

Divisions and Integrated Review Groups (IRGs) Translational and Clinical Sciences Cardiovascular and Respiratory Sciences Surgical Sciences, Biomedical Imaging and Bioengineering Musculoskeletal, Oral & Skin Sciences Oncology 2: Translational Clinical Vascular and Hematology Physiological and Pathological Sciences Endocrinology, Metabolism, Nutrition & Reproductive Sciences Immunology Infectious Diseases & Microbiology Digestive, Kidney & Urological Systems Neuroscience, Development and Aging Brain Disorders & Clinical Neuroscience Molecular, Cellular & Developmental Neuroscience Integrative, Functional & Cognitive Neuroscience Emerging Technologies & Training in Neuroscience Biology of Development & Aging Biobehavioral & Behavioral Processes Risk, Prevention & Health Behavior Population Sciences & Epidemiology Healthcare Delivery & Methodologies AIDS & AIDS Related Research AIDS, Behavioral and Population Sciences Basic and Integrative Biological Sciences Biological Chemistry & Macromolecular Biophysics Bioengineering Sciences & Technologies Genes, Genomes & Genetics Oncology 1: Basic Translational Cell Biology Interdisciplinary Molecular Sciences & Training

Integrated Review Groups Biological Chemistry & Macromolecular Biophysics Bioengineering Sciences & Technologies Oncology 1: Basic Translational Cell Biology Genes, Genomes & Genetics Interdisciplinary Molecular Sciences & Training Division of Basic and Integrative Biological Sciences Cancer Molecular Pathobiology (CAMP) Cancer Etiology (CE) Cancer Genetics (CG) Molecular Oncogenesis (MONC) Tumor Cell Biology (TCB) Tumor Microenvironment (TME) Tumor Progression & Metastasis (TPM) CSR Study Sections

Referral Officers Professional scientists, most of whom also serve as scientific review officers of CSR study sections Applications Are Assigned by:

Institutes or Centers based on─ Overall mission and guidelines of the Institute or Center Specific programmatic mandates and interests of the Institute or Center Integrated Review Groups based on─ Specific review guidelines for each Integrated Review Group (IRG) Applications Are Assigned to:

Get Your Application to the Right Study Section! Review CSR Integrated Review Group and Scientific Review Group (Study Section) guidelines to identify a home for your application. Submit a Cover Letter!

Help Your Application Get to the Right Study Section

Cover Letter The cover letter should be used for a number of important purposes: Suggest Institute/Center assignment Suggest review assignment Identify individuals in conflict Identify areas of expertise needed to evaluate the application Discuss any special situations Required for an electronic changed/corrected submission It is NOT appropriate to use the cover letter to suggest specific reviewers.

Please assign this Phase I SBIR “Drugs for Retinoblastoma Treatment” to the following: Institutes/Centers National Cancer Institute – NCI National Eye Institute – NEI Scientific Review Group Oncology Translational and Clinical IRG - OTC Please do not assign this application to the following: Scientific Review Group Biological Chemistry and Macromolecular Biophysics - BCMB This study focuses on a new in vitro model for testing drugs for treatment of retinoblastoma, not the synthesis of new chemotherapeutic agents. Sample Cover Letter

Peer Review in CSR CSR Study Sections are managed by a Scientific Review Officer (SRO) who is a doctoral-level professional, whose scientific background is close to the focus of the study section. Each CSR standing study section has regular members who are primarily from academia. About applications are normally reviewed at each study section meeting.

Role of the Scientific Review Officer Performs administrative and technical review of applications to ensure completeness and accuracy Selects reviewers based on broad input Manages study section meetings Prepares summary statements Provides any requested information about study section recommendations to Institutes/Centers and National Advisory Councils/Boards Designated Federal Official with overall responsibility for the review process

Enhancing Peer Review Overview and Timeline January 2009 May/June 2009 Reviews January 25, 2010 Submissions Phase out A2 resubmission of applications Identification of Early Stage Investigator (ESI) R01 applications Enhanced review criteria New 9-point scoring system Criterion scoring Structured critiques Score order review Clustering of New Investigator (NI) R01 & clinical applications Discussed/Not-discussed Applications Restructured applications to align with review criteria Shortened page limits and new instructions

Where Do We Find Reviewers? CSR Registry for Volunteer Reviewers? Successful applicants Recommendations from reviewers and NIH staff NIH RePORTER ( Internet Scientific conferences

How Reviewers Are Selected for Study Section Service Demonstrated scientific expertise/research support Doctoral degree or equivalent Mature judgment Work effectively in a group context Breadth of perspective Impartiality Representation of women and minority scientists Geographic distribution

Before the Study Section Meeting Each application is assigned to 3 or more reviewers 5-6 weeks in advance Reviewer assess each application by providing: A preliminary Overall Impact score Criterion Scores for each of the 5 Core Review Criteria using the scale A written critique

At The Meeting Order of Review The average of the preliminary Overall Impact score from the assigned reviewers determines the review order Discussions start with the application with the best average preliminary Overall Impact score Clustering of Review New Investigator R01 applications are clustered Clinical applications & other mechanisms may be clustered (n ≥ 20) Not Discussed Applications o About 50% applications will be discussed o Applications unanimously judged by the review committee to be less competitive are not discussed.

Scoring Score applications on five review criteria using a scale of 1-9 Preliminary overall impact/priority score using 1-9 scale Discussed applications receive an overall score from each eligible (i.e., without conflicts of interest) panel member, and these scores will be averaged to one decimal place, and multiplied by 10. The 81 possible priority scores will thus range from All applications will receive scores: Not discussed applications will receive initial criterion scores from the three assigned reviewers.

Scores for each review criterion Critiques from assigned reviewers Administrative notes if any If your application is discussed, you also will receive: An overall impact/priority score and percentile ranking A summary of review discussion Budget recommendations Your Summary Statement

Summary Statement New Indicator for Early Stage Investigators Percentile: 29 Percentile in whole numbers Impact/Priority Score in range

A Window to Your Application: eRA Commons eRA Commons is an online interface where a grant applicant can: Check submitted grant application for errors and warnings and view final image Track review assignment, view review outcomes (score, summary statements), find contact info Update Personal Profile to ensure Early Stage Investigator eligibility is in place Submit pre-award information (just in time) View Notice of Award and other key documents And much more!

Check the Status of Your Application in NIH Commons Summary Statement Additions for Review

Preparing an Application Register with Grants.gov & eRA Commons Submit in response to PA Follow Application Guide & Instructions Submit via your organizational representative Use error correction window Use eRA Commons to track Prepare to Apply & Register Find Opportunity Prepare Application Submit, Track & View Electronic Application Process

Who Can Answer Your Questions? Before You Submit Your Application A Program Officer at an NIH Institute or Center Scientific Review Officer After You Submit Your Scientific Review Officer After Your Review Your Assigned Program Officer GrantsInfo: –

View the Videos NIH Peer Review Revealed NIH Tips for Applicants

About CSR About CSR News and Reports News and Reports Peer Review Meetings Resources for Applicants Resources for Applicants CSR Web Site:

Key NIH Review and Grants Web Sites NIH Center for Scientific Review NIH Office of Extramural Research