NIH Study Section Review Process Paul Jacobsen, PhD; Associate Director Healthcare Delivery Research Program, NCI Hazel Nichols, PhD; Assistant Professor, Department of Epidemiology, UNC

Shriver National Institute National Institutes of Health National Institute on Alcohol Abuse and Alcoholism of Arthritis and Musculoskeletal and Skin Diseases National Cancer Institute on Drug Abuse of Environmental Health Sciences on Aging Eunice Kennedy Shriver National Institute of Child Health and Human Development National Institute on Deafness and Other Communication Disorders National Eye National Human Genome Research of Mental Health of Neurological Disorders and Stroke of General Medical Sciences of Nursing Research National Library of Medicine Center for Scientific Review National Center for Complementary and Alternative Medicine of Allergy and Infectious Diseases John E. Fogarty International Center for Research Resources Clinical Center Minority Health and Health Disparities National Institute of Biomedical Imaging and Bioengineering Office of the Director Information Technology National Heart, Lung, and Blood of Dental and Craniofacial Research of Diabetes and Digestive and Kidney Diseases NIH is the largest source of funding for medical research in the world, creating hundreds of thousands of high-quality jobs by funding thousands of scientists in universities and research institutions in every state across America and around the globe. NIH is made up of 27 Institutes and Centers, each with a specific research agenda, often focusing on particular diseases or body systems. More than 80% of the NIH's budget goes to more than 300,000 research personnel at over 3,000 universities and research institutions. In addition, about 6,000 scientists work in NIH’s own laboratories, most of which are on the NIH main campus in Bethesda, Maryland. The main campus is also home to the NIH Clinical Center, the largest hospital in the world totally dedicated to clinical research. 2

NIH Grant Process Investigator Grantee Institution NIH Initiates Research Idea and Prepares Application Conducts Research Investigator Submits Application Manages Funds Grantee Institution NIH Center for Scientific Review (CSR) Assigns to Scientific Review Group (SRG) and Institute Institute Makes Funding Selections & Issues Grant Awards SRG Evaluates for Scientific Merit Institute Evaluates for Program Relevance and Need National Advisory Council/Board Recommends Action

NIH Funding Mechanisms Academic Career Award (K07) Mentored Clinical Scientist Short-Term Training (T35) Development Award (K08) Small Business Grants (R41, R42, R43, R44) Conference Grant (R13) Marc Fellowships (F34, F36, T34) Academic Research Enhancement Award (R15) Minority Biomedical Support Grant (S06) Biomedical Research Support Shared Instrumentation Grant (S10) Resource Grant (P40, P41, R24, R26, R28) Research Project Grant (R01) Developmental/Exploratory Grant (R21) Small Grant (R03) Program Project Grant (P01) Predoctoral Fellowship (F31) Postdoctoral Fellowship (F32) Center Grant (P30, P50, P60) Senior Fellowship (F32) Institutional Fellowship (T32) Fogarty International Center Fellowship (F05, F06) Research Scientist Development (K01) Pathway to Independence (K99/R01)

Understanding NIH Peer Review

WHAT HAPPENS IN A STUDY SECTION MEETING? Closed to the public (Program officials may observe) Orientation Conflict of interest Developments of interest to the study section Changes in policy or procedure Introduction of persons present Role of persons present Applications reviewed in ascending order of mean initial impact score (best to worst), until about 50% are scored. The rest are unscored. Application by application discussion Persons with conflicts of interest excused Assigned reviewers give preliminary scores Discussion of application’s scientific and technical merit Assigned reviewers first, then other members Range of scores Every member scores every application Assignment of gender, minority, and children codes, human subjects codes; recommended changes to budget Video

Review Criteria Significance: Does the study address an important problem? How will scientific knowledge be advanced? Investigator: Is the investigator qualified? Innovation: Are there novel concepts or approaches? Are the aims original and innovative? Approach: Are design and methods well-developed and appropriate? Are problem areas addressed? Environment: Does the scientific environment contribute to the probability of success? Are there unique features of the scientific environment? Overall Impact: Assessment of the likelihood for the project to exert a sustained, powerful influence on the research field(s) involved

Scoring Scale Happy Days are Here Again! Pop the Bubbly! Cautious Optimism Borderline. In striking distance for resubmission. Not very good. Pretty Bad. Awful. Oh, the Humanity!

New Guidelines to Enhance Rigor and Transparency Scientific premise Scientific rigor Consideration of Sex and other Biological Variables Authentication of Key Biological and/or Chemical Resources

Rigor and Transparency Scientific premise State the strengths and weaknesses of published research or preliminary data crucial to your application Scientific rigor Describe how your experimental design and methods will achieve robust and unbiased results

Rigor and Transparency Scientific premise Scientific rigor Consideration of Sex and other Biological Variables Vertebrate animal and human studies Explain how biological variables are factored into research design and provide justification if only one sex is used

Does the study Involve vertebrate animals or humans?

Rigor and Transparency Scientific premise Scientific rigor Consideration of Sex and other Biological Variables Authentication of Key Biological and/or Chemical Resources Includes, but not limited to, cell lines, specialty chemicals, antibodies, non-standard laboratory reagents. Discussed after scoring

Calculations Priority Score: Mean of all reviewers’ Overall Impact scores X 10, rounded to whole number. Percentile: Percentile rank of application compared to the pool of the last three review cycles of that study section. (Low percentiles are better.)

Summary Statement After the review meeting is finished, the results are documented by the SRA in a summary statement and forwarded to the PI and to the assigned NIH Institute. The assigned NIH Institute is responsible for making a funding decision. The summary statement contains: Overall Resume and Summary of Review Discussion Essentially Unedited Critiques of Assigned Reviewers, including their scores in the 5 categories Priority Score and Percentile Ranking Budget Recommendations Administrative Notes

What Determines Which Awards Are Made? Scientific Merit Program Considerations Availability of Funds

A little vocabulary Funding Opportunity Announcements (FOA) Program Announcement (PA) Indicates that the institute would like to fund research in this area, but you must go through the regular application process. There are two special types of PAs: PAR—applications are reviewed by an Institute or a special Center for Scientific Review committee PAS—PA with set-aside funds. Request for Applications (RFA) Indicates that the institute has set aside funds for research in this area. May have a special due date and study section. Investigator Initiated Application Not in response to specific PA or RFA. That’s OK, use general, omnibus PA.

NIH Early Career Reviewer Program Goals Develop qualified scientists without prior CSR experience into well-trained reviewers Help emerging researchers advance their career by exposing them to a peer review experience to make them more competitive applicants Enrich the pool of NIH reviewers by including those from less research-intensive institutions https://public.csr.nih.gov/reviewerresources/becomeareviewer/ecr/pages/default.aspx

NIH Early Career Reviewer Program To Qualify 2 years as a faculty member or similar researcher role. Postdocs are not eligible. Evidence of an active research program. At least 2 senior author (first, last, single, or corresponding) research papers in the last 2 years. Have not served on a CSR study section. Current funding not required. https://public.csr.nih.gov/reviewerresources/becomeareviewer/ecr/pages/default.aspx

NIH Early Career Reviewer Program To Apply Full CV in word or as pdf NIH Commons ID Select 4 CSR Study sections https://internet.csr.nih.gov/ECRV2/ECRMain.aspx

Questions? Acknowledgements Heidi Sahel Cheryl Thompson Thomas Brandon