NIH OBSSR Summer Institute July 2012 National Institutes of Health U.S. Department of Health and Human Services Overview of the NIH Peer Review Process Kate Bent, RN, PhD Senior Advisor to the Director, Center for Scientific Review

My plan: Overview: Submission and review processes Why? The more you know about the process and your audience - the more likely you will be able to develop a solid application

NIH Peer Review System for Grant Applications First Level of Review Scientific Review Group (Study Section) First Level of Review Scientific Review Group (Study Section) Second Level of Review NIH Institute/Center Council Second Level of Review NIH Institute/Center Council

Center for Scientific Review

National Institutes of Health National Institute on Alcohol Abuse and Alcoholism National Institute on Alcohol Abuse and Alcoholism National Institute of Arthritis and Musculoskeletal and Skin Diseases National Institute of Arthritis and Musculoskeletal and Skin Diseases National Cancer Institute National Cancer Institute National Institute on Drug Abuse National Institute on Drug Abuse National Institute of Environmental Health Sciences National Institute of Environmental Health Sciences National Institute on Aging National Institute on Aging Eunice Kennedy Shriver National Institute of Child Health and Human Development Eunice Kennedy Shriver National Institute of Child Health and Human Development National Institute on Deafness and Other Communication Disorders National Institute on Deafness and Other Communication Disorders National Eye Institute National Eye Institute National Human Genome Research Institute National Human Genome Research Institute National Institute of Mental Health National Institute of Mental Health National Institute of Neurological Disorders and Stroke National Institute of Neurological Disorders and Stroke National Institute of General Medical Sciences National Institute of General Medical Sciences National Institute of Nursing Research National Institute of Nursing Research National Library of Medicine National Library of Medicine Center for Scientific Review Center for Scientific Review National Center for Complementary and Alternative Medicine National Center for Complementary and Alternative Medicine National Institute of Allergy and Infectious Diseases National Institute of Allergy and Infectious Diseases John E. Fogarty International Center John E. Fogarty International Center National Center for Advancing Translational Research National Center for Advancing Translational Research Clinical Center Clinical Center National Institute on Minority Health and Health Disparities National Institute on Minority Health and Health Disparities National Institute of Biomedical Imaging and Bioengineering National Institute of Biomedical Imaging and Bioengineering Office of the Director Center for Information Technology Center for Information Technology National Heart, Lung, and Blood Institute National Heart, Lung, and Blood Institute National Institute of Dental and Craniofacial Research National Institute of Dental and Craniofacial Research National Institute of Diabetes and Digestive and Kidney Diseases National Institute of Diabetes and Digestive and Kidney Diseases

Receives all NIH applications Refers them to NIH Institutes/Centers and to scientific review groups Reviews majority of grant applications for scientific merit Your Application Goes to the NIH Center for Scientific Review (CSR) Focal Point for Initial Review at NIH MISSION: To see that NIH grant applications receive fair, independent, expert, and timely reviews – free from inappropriate influences – so NIH can fund the most promising research.

CSR Peer Review – Fiscal Year ,000 applications received 58,000 applications reviewed 16,000 reviewers 230 Scientific Review Officers 1,465 review meetings

Institutes or Centers based on─ Overall mission and guidelines of the Institute or Center Specific programmatic mandates and interests of the Institute or Center Integrated Review Groups based on─ Specific review guidelines for each Integrated Review Group (IRG) Applications Are Assigned to:

Cover Letter The cover letter should be used for a number of important purposes: Suggest Institute/Center assignment Suggest review assignment Identify individuals in potential conflict and explain why Identify areas of expertise needed to evaluate the application Discuss any special situations It is NOT appropriate to use the cover letter to suggest specific reviewers.

The Study Section

Peer Review in CSR CSR Study Sections are managed by a Scientific Review Officer (SRO) who is a doctoral-level professional, whose scientific background is close to the focus of the study section. Each CSR standing study section has regular members who are from the scientific community. Temporary members are recruited as needed. About applications are normally reviewed at each study section meeting.

Role of the Scientific Review Officer Performs administrative and technical review of applications to ensure completeness and accuracy Selects reviewers based on broad input Manages study section meetings Prepares summary statements Provides any requested information about study section recommendations to Institutes/Centers and National Advisory Councils/Boards Designated Federal Official with overall responsibility for the review process

How Reviewers Are Selected for Study Section Service Demonstrated scientific expertise/research support Doctoral degree or equivalent Mature judgment Work effectively in a group context Breadth of perspective Impartiality Representation of women and minority scientists Geographic distribution

Before the Study Section Meeting Each application is assigned to 3 or more reviewers 5-6 weeks in advance Reviewer assess each application by providing: A preliminary Overall Impact score Criterion Scores for each of the 5 Core Review Criteria A written critique

The Study Section Meeting

At The Meeting Order of Review The average of the preliminary Overall Impact score from the assigned reviewers determines the review order Discussions start with the application with the best average preliminary Overall Impact score Clustering of Review New Investigator R01 applications are clustered Clinical applications & other mechanisms may be clustered (n ≥ 20) Not Discussed Applications o About half the applications will be discussed o Applications unanimously judged by the review committee to be in the lower half are not discussed

New Investigator (NI) PD/PI who has not yet competed successfully for a substantial NIH research grant Early Stage Investigator (ESI) PD/PI who qualifies as a New Investigator AND is within 10 years of completing the terminal research degree or is within 10 years of completing medical residency (or equivalent) New and Early Stage Investigators

Electronic reviews are used when needed to recruit the best reviewers Alternative Review Platforms Internet Assisted Meetings Telephone Assisted Meetings Video Assisted Meetings Telepresence Meetings Hybrid meetings Your Application Could Be Reviewed Electronically

Review Criteria Overall Impact Assessment of the likelihood for the project to exert a sustained, powerful influence on the research field(s) involved Core Review Criteria Significance Investigator(s) Innovation Approach Environment Review criteria each scored from 1-9

Scoring 9-point score scale is used to provide: Criterion Scores for each of the 5 core review criteria Overall Impact/Priority Score based on but not a sum of the core criterion scores plus additional criteria All applications receive scores: Not discussed applications will receive only initial criterion scores from the three assigned reviewers. Discussed applications also receive an averaged overall impact score from eligible (i.e., without conflicts of interest) panel members.

Additional Criteria Contribute to Overall Impact Scores Protections for human subjects Inclusions of women, minorities, and children Appropriate use of vertebrate animals Management of biohazards

Resource Sharing Plans: Data Model Organisms Genome Wide Association Studies Foreign Organizations Select Agents Budget Other Considerations which do Not Affect Overall Impact Scores

Scores for each review criterion Critiques from assigned reviewers Administrative notes if any If your application is discussed, you also will receive: An overall impact/priority score and percentile ranking A summary of review discussion Budget recommendations Your Summary Statement

Summary Statement New Indicator for Early Stage Investigators Percentile: 29 Percentile in whole numbers Impact/Priority Score range Program Officer

Who Can Answer Your Questions? Before You Submit Your Application A Program Officer at an NIH Institute or Center Scientific Review Officer After You Submit Your Scientific Review Officer After Your Review Your Assigned Program Officer GrantsInfo: –

“You miss 100% of the shots you never take.” Wayne Gretzky

CSR and NIH Information Sources

Announces NIH Scientific Initiatives Provides NIH Policy and Administrative Information Supplies links to application forms Available on the NIH Web Site: NIH Guide For Grants and Contracts U.S. Department of Health and Human Services

A Window to Your Application: eRA Commons eRA Commons is an online interface where a grant applicant can: Check submitted grant application for errors and warnings and view final image Track review assignment, view review outcomes (score, summary statements), find contact info Update Personal Profile to ensure Early Stage Investigator eligibility is in place Submit pre-award information (just in time) View Notice of Award and other key documents And much more!

Check the Status of Your Application in NIH Commons

View the Videos NIH Peer Review Revealed NIH Tips for Applicants What Happens to Your NIH Grant Application

National Institutes of Health: Office of Extramural Research Grants Policy Electronic Submission Center for Scientific Review: Resources for Applicants CSR Study Section Rosters Review Group Meeting Dates NIH Peer Review Information on the Web

About CSR About CSR Applicant Resources Applicant Resources Study Sections Rosters and Meetings Rosters and Meetings CSR Web Site:

Helpful Handouts Insider’s Guide What Happens to NIH Grant Application to Peer Review Your Grant Application Useful Web Links