NIH Peer Review Pedro Delgado, MD

Funded NIH Grant Awards Searchable Database: RePORT http://projectreporter.nih.gov/reporter.cfm

Funding Opportunity Announcements (FOA) Available Grant Programs http://grants.nih.gov/grants/guide/

NIH eRA Commons Electronic Connection – “For the purposes of exchanging research grants administration information”…this is how the NIH communicates with applicants and grantees [Obtain password help from the Office of Sponsored Programs (OSP), UAMS]

“R01’s and K08’s and T32’s, Oh My!” So What Do All Those Letters Mean? NIH uses a variety of funding mechanisms, including: grants, cooperative agreements & contracts. Grants & Cooperative Agreements are classified by “Activity Codes” that indicate the kind of activity that will be conducted under the project: The R-series indicates Research Project Grants, e.g., R01, R03, R21. The P-series indicates Program Project Grants, e.g., P01, P30, P50. The T-series indicates institutional NRSA training grants. The K-series indicates career development awards. The F-series indicates individual NRSA fellowships. The IMPAC Manual contains complete explanations of all activity codes and other arcane NIH classification codes. It may be found at: http://grants1.nih.gov/grants/funding/funding.htm

? Your Grant NIH or Other Funding Agency “Pink Sheet” months later.. Critique and Funding Decision

President Cabinet Departments $ HHS Annual $$$ from Congress FDA CDC NIH NHLBI NIDDK NCI CSR Office of the Secretary NIAMS NIDCR NIGMS NIBIB NCCIH NHGRI NIA NIAID NINR NCMHHD NCATS OD NINDS $

National Institutes of Health Office of the Director National Institute on Alcohol Abuse and Alcoholism of Arthritis and Musculoskeletal and Skin Diseases National Cancer Institute of Diabetes and Digestive and Kidney Diseases of Dental and Craniofacial Research on Drug Abuse of Environmental Health Sciences on Aging of Child Health and Human Development National Institute on Deafness and Other Communication Disorders National Eye National Human Genome Research National Heart, Lung, and Blood of Mental Health of Neurological Disorders and Stroke of General Medical Sciences of Nursing Research National Library of Medicine Center for Information Technology Scientific Review National Center for Complementary and Integrative Health of Allergy and Infectious Diseases for Research Resources Clinical Center National Center on Minority Health and Health Disparities National Institute of Biomedical Imaging and Bioengineering Fogarty International Center National Institutes of Health Study Sections

Understanding the NIH Grants Process NIH receives over 28,000 competing grant applications per year. The Center for Scientific Review (CSR) receives almost 10,000 applications per grant cycle. NIH funded about 17,000 applications in FY01. The success rate for competing R01 applications hovers around 30%. CSR’s mailroom on deadline day.

NIH CSR Your Grant & ~30,000 others Institute1 $ Council Institute2 $ Assignment Reviewer 1 Provide Recommendation (numerical) Study Section Assignment Reviewer 2 Reviewer 3

Relationship of Study Sections (Scientific Review) to Institutes ($$) ~200 Study Sections -- CSR ~ 20 Institutes Biochemistry Clinical Oncology Genetics Epidemiology of Cancer Lung Injury, Repair Microbial Physiology Oncological Sciences Etc. NCI NEI NIDDK NIGMS NHBL Etc. Scientific Review is Done by Study Sections Funding Decisions made by Institutes

The GUTS of your APPLICATION! PHS 398 Research Strategy Significance Innovation Approach

What will reviewers look for in your grant? Core Review Criteria (in instructions to reviewers on NIH website) Significance Investigators Innovation Approach Environment

Innovation Innovation. Does the application challenge and seek to shift current research or clinical practice paradigms by utilizing novel theoretical concepts, approaches or methodologies, instrumentation, or interventions? Are the concepts, approaches or methodologies, instrumentation, or interventions novel to one field of research or novel in a broad sense? Is a refinement, improvement, or new application of theoretical concepts, approaches or methodologies, instrumentation, or interventions proposed?

SCIENTIFIC PREMISE

SCIENTIFIC PREMISE, CONT’D

UPDATED APPLICATION INSTRUCTIONS SIGNIFICANCE (SCIENTIFIC PREMISE): Describe the scientific premise for the proposed project, including consideration of the strengths and weaknesses of published research or preliminary data crucial to the support of your application. APPROACH (RIGOR & TRANSPARENCY): Describe the experimental design and methods proposed and how they will achieve robust and unbiased results. Explain how relevant biological variables, such as sex, are factored into research designs and analyses for studies in vertebrate animals and humans. For example, strong justification from the scientific literature, preliminary data, or other relevant considerations, must be provided for applications proposing to study only one sex.

Key Resource Authentication! What is considered a key biological or chemical resource? Generally, these are resources that: May differ from laboratory to laboratory, over time Have qualities or qualifications that can influence results Are integral to the proposed research Examples: cell lines, specialty chemicals, antibodies, other biologics

Reviewers Written Review = Critique major strengths/weaknesses issues concerns primary basis for the recommended Impact Score and Summary Statement usually must be specifically addressed if application revised/resubmitted

Feasibility is Critical!! Peer Review Is Critical!!

Scored Components K Grants R Grants Overall Impact Candidate Career Development Plan Research Plan Mentor (s) Environment & Commitment to Candidate Overall Impact Significance Investigator(s) Innovation Approach Environment

Scores also Provided for Each Component 1 exceptional 2 outstanding 3 excellent 4 very good 5 good 6 satisfactory 7 fair 8 marginal 9 poor

Impact/Priority Scores A numerical rating that reflects the scientific merit of the proposed research relative to the "state of the science”.

NIH Study Section Scores 1 exceptional – exceptionally strong with no weaknesses 2 outstanding – extremely strong with negligible weaknesses 3 excellent – very strong with only some minor weaknesses 4 very good – strong but with numerous minor weaknesses 5 good – strong but with at least one moderate weakness 6 satisfactory – some strengths but also some moderate weaknesses 7 fair – some strengths but with at least one major weakness 8 marginal – a few strengths and a few major weaknesses 9 poor – very few strengths and numerous major weaknesses

How does the Study Section Operate? SS has 15-25 members (regular & ad hoc) selected for their expertise in the area of the applications the study section reviews and is overseen by a Scientific Review Officer Scientific Review Officers (SRO) oversee individual SS’s; receive applications from CSR main office and assign each application to primary & secondary reviewers before the review meeting Primary & Secondary Reviewers prepare written reviews and post in advance on NIH website All SS members meet to review all grants Chair runs meeting (not NIH employee) SRO serves as a ‘resource’ about policy & protocol Study Sections are far more diverse than one might anticipate

The primary and secondary reviewers will: read your proposal thoroughly (you hope!) and understand it (you really hope!!) evaluate it (priority score), explain it AND justify their evaluation to the study section, prepare a written summary and evaluation. They present to the full study section (~15-20 people) most of whom will not have read the complete grant. After discussion, every member gets an equal vote. 50-100 grants in 2 days – You must be clear and to the point!!

Reviewers will see new drop-down menus in IAR for entering scores for individual criteria

Edit Criterion Scores on Voter Sheet

Link to Pop-up to View All Scores

For Standing Study Sections, Priority Scores Are Often Percentile Ranked

Percentiles Represents the relative position or rank of each priority score (along a 100.0 percentile band) among the scores assigned by a particular study section.

Priority Scores and Percentile Rank Priority Scores assigned by Study Sections based on average of all reviewers scores Percentile Rank assigned afterwards to normalize scoring across study sections and over time Funding determined by percentile rank

Streamlined Review Prior to the meeting, reviewers categorize applications as either in the upper half or lower half in quality Applications in the upper half = fully discussed and scored at the study section meeting All other applications are not discussed nor scored

2) Institute Funding Decision 1) Study Section Review 2) Institute Funding Decision Approved Applications Receive Percentile Scores 1st John Doe (Biochem. SS) 2nd Your Grant (Oncology SS) 13th Stancel (Genetics SS) 27th Jane Doe (Micro. SS) Etc. “Streamlined” Not Recommended for Further Consideration National Cancer Institute (Budget from Congress) 1) Your Grant 2) John Doe 3) Stancel 4) Jane Doe Etc. $ Payline -

It’s Great Science, BUT will it really help us find The Institute’s Council may make some changes in the rankings based upon their particular goals and needs. It’s Great Science, BUT will it really help us find a cure for the plague?

Grant Contacts, Interactions, and Information in Various Phases Pre Submission: Institute Program Officials Assess Institute’s Interest in Your Proposal Provide Advice & Potential SS Assignment Submission – Review : CSR’s Scientific Review Officers (SRO’s) who oversee study sections Questions about application Submitting additional information Post Award : Assigned Program Officer at Funding Institute NEVER CONTACT A STUDY SECTION MEMBER ABOUT YOUR GRANT!!!

Help your proposal be assigned appropriately – You can Recommend Assignments -Call Institute staff (be respectful) – they want to help -Study section rosters are public information (‘regular’ members) Remember – The agency does not care about supporting you as an individual. They only want to fund you if the research you propose helps them achieve one of their objectives.

(1) Get peer review from critical colleagues: EARLY and OFTEN!! (2) Write different sections of the grant for ‘different’ reviewers (3) Grant must be crystal clear and explicit! Page limits make this critically important. (How long should one plan to write?)

Important Sources of Information NIH Website – general information about types of Grants ‘standard’ information, e.g., general instructions, receipt dates, etc. NIH Guide – weekly information about program changes, Request for Applications (RFA’s) Institute Home Pages Information about areas of interest Names and Contact information for Program Officers Center for Scientific Review Website Detailed information about preparing applications Names and Contact information for Scientific Review Officers Study Section Descriptions and Rosters Instructions for Reviewers!!!

USE THE FOLLOWING TOOLS – IN ORDER – TO PREPARE YOUR GRANT The Internet to identify Study section rosters Institute contact personnel The Institute’s goals and interests The telephone to Assess interest in your project Consider study section/institute assignments The word processor to -Actually write the grant!

HELP THE REVIEWERS – When you Write the Grant! Read and follow your instructions and know the reviewers’ Give the proposal a focus, and have colleagues help you evaluate it (write a novel, not a collection of short stories) 3) Build the application around your focus (the hypothesis or scientific question being addressed) Help the reviewer prepare his/her critique and presentation to the study section (know what he/she is looking for and write it for them - CLEARLY!) Get rigorous, critical review (before you send the grant!) from “experts” from well rounded “generalists”

Talk to People at the Funding Agency And Don’t Ever Forget….. Talk to People at the Funding Agency Get peer review from critical colleagues: Do It EARLY and OFTEN!! Never Overestimate the Reviewers