NOTICE OF POLICY CHANGES FOR NIH GRANTS Chandan K. Sen Department of Surgery

NEW NIH RULES: KEY POINTS RIGOR, TRANSPARENCY AND REPRODUCIBILITY BIOSKETCH INCLUSION OF CHILDREN AUTHENTICATION OF KEY BIOLOGICAL AND CHEMICAL RESOURCES SIMPLIFICATION OF VERTEBRATE ANIMAL SECTION

Rigor and Transparency in Research To support the highest quality science, public accountability, and social responsibility in the conduct of science, NIH’s Rigor and Transparency efforts are intended to clarify expectations and highlight attention to four areas that may need more explicit attention by applicants and reviewers: Scientific premise Scientific rigor Consideration of relevant biological variables, such as sex Authentication of key biological and/or chemical resources Role of reviewers: Assess the scientific merit of each application according to the review criteria, which include consideration of scientific premise, rigor, and consideration of relevant biological variables, and the adequacy of the authentication of key biological and/or chemical resources as an administrative issue. Evaluations should be based on current best practices in the field. SOURCE:NIH,CSR

Scientific Premise: Guidance for Reviewers GOAL: Ensure that the underlying scientific foundation of the project—concepts, previous work, and data (when relevant)—is sound. Pertains to the underlying evidence/data for the project Address under Significance (R applications) or Research plan (Ks) Addition to the review criteria: “Is there a strong scientific premise?” Specifically, has the applicant: Provided sufficient justification for the proposed work? Cited appropriate work and/or preliminary data? Appropriately identified strengths and weaknesses in prior work in the field? Proposed to fill a significant gap in the field? OR has the applicant explained why this is not possible? The instructions for Significance already include discussing the importance of the research question, critical barriers to progress, how the proposed project will improve scientific knowledge, and how the field will change if the aims are achieved. The Research Plan criterion addresses the scientific and technical merit of the proposed research, plus the contribution of the research plan to the candidate’s career development. Scientific premise extends these instructions to include a retrospective assessment of the foundation for the project in the context of these other factors. SOURCE:NIH,CSR

SIGNIFICANCE SECTION Explicitly state the scientific premise for the proposed project. The general strengths and weaknesses of the prior research cited by the applicant, which form the basis for the proposed research Not your hypothesis. This separates the premise from the hypothesis your grant is trying to address; the premise leads to the hypothesis. Recommendation: State significance section of your grant application with a paragraph or subsection entitled “Scientific Premise”. Consider a separate section in the Significance entitled “Strengths and Weakness of Supporting Data” or, alternatively, a 1-3 sentence appraisal of the data at the end of each section where it is presented. NIH: NOT-OD-16-011

Scientific Rigor: Guidance for Reviewers (NIH: NOT-OD-16-034) GOAL: Ensure a strict application of scientific method that supports robust and unbiased design, analysis, interpretation, and reporting of results, and sufficient information for the study to be assessed and reproduced. Give careful consideration to the methods and issues that matter in your field. Pertains to the proposed research Address under Approach (R applications) or Research Plan (Ks) Addition to review criteria: Are there “strategies to ensure a robust and unbiased approach, as appropriate for the work proposed?” Possible considerations, if appropriate for the scientific field and research question, include plans for: determining group sizes analyzing anticipated results reducing bias ensuring independent and blinded measurements improving precision and reducing variability including or excluding research subjects managing missing data SOURCE:NIH,CSR

Relevant Biological Variables: Guidance for Reviewers GOAL: Ensure that the research accounts for sex and other relevant biological variables in developing research questions and study designs. The ways in which sex and other biological variables need to be accounted for will differ across research questions and fields of study. Pertains to the proposed research Applies to studies in vertebrate animals and/or human subjects Address in Approach (R applications) or Research Plan (Ks) Addition to review criteria: Are there “adequate plans to address relevant biological variables for studies in vertebrate animals or human subjects?” Consideration of sex is required in all studies involving human subjects or vertebrate animals (see next slide). Specific considerations to assess include: Applies broadly to all biological variables relevant to the research such as sex, age, source, weight, or genetic strain.) Has the applicant considered biological variables, such as sex, that are relevant to the experimental design? Will relevant biological variables be controlled or factored into the study design appropriately? Although the review criteria specify that plans to address relevant biological variables are included for studies in vertebrate animals or human subjects, reviewers may extend the same thinking to other model systems or approaches if biological variables are relevant based on the research question, current knowledge, and best practices in the field. SOURCE:NIH,CSR

Sex as a Biological Variable: Guidance for Reviewers SEX = CHROMOSOME; GENDER = BEHAVIOR Consideration of sex, included under the umbrella of “Relevant Biological Variables”, is required in all studies involving human subjects or vertebrate animals. NIH expectations for applicants: If little is known about sex differences, the application should include both sexes. Sufficient numbers should be provided to inform the presence or absence of sex differences. Statistically powered comparisons between sexes may not be warranted. Specific hypotheses about sex differences may not be possible. Findings should be reported separately by sex in progress reports and publications. If sex differences are known not to exist, a strong justification should be provided if the application proposes to study one sex. If sex differences are known, experiments should be designed with appropriate group sizes to detect sex differences. Since much of the published literature has been conducted either in one sex or sex was not reported, an important consideration for research applications will be what is known about males and females in relation to the disease, condition or question under study. If little is known about males and females, the application should address this gap by including both sexes in the proposed research project(s), in sufficient numbers to inform the possibility that sex differences may be present and to report separately by sex in their reported research. These grant applications may not be able to propose specific hypotheses about sex differences or justify sufficient numbers of males and females to conduct statistically powered comparisons between sexes, but the inclusion of both sexes and the analysis of results for each sex represent an important advancement of our knowledge in areas where this information is currently lacking. A strong justification should be provided if the application proposes to study one sex. Cost and absence of known sex differences are not valid justifications for not addressing sex as a biological variable. Hyperlink to NOT-OD-15-102 supplement: http://orwh.od.nih.gov/sexinscience/overview/pdf/NOT-OD-15-102_Guidance.pdf SOURCE:NIH,CSR

Sex as a Biological Variable: Guidance for Reviewers Consideration of sex NIH expectations for reviewers: As part of the Consideration of Relevant Biological Variables, assess whether the plans to address sex as a biological variable are adequate (for studies in vertebrate animals or human subjects). If the study involves only one sex, is this justified scientifically? Assess within the context of the research question and current scientific knowledge. SOURCE:NIH,CSR

AUTHENTICATION OF KEY BIOLOGICAL AND/OR CHEMICAL RESOURCES BRIEF DESCRIPTION OF METHODS TO ENSURE IDENTITY AND VALIDITY OF KEY REAGENTS USED IN PROPOSED STUDIES Key biological resources include: Those that differ from lab to lab Qualities or qualifications that influence results Integral to proposed research Include: cell lines, specialty chemicals (homegrown/ developed in-house) antibodies, other biologics Focus only on authentication/validation; other methods and preliminary data will be within research strategy ***IF APPLICATIONS ARE NON-COMPLIANT THEY WILL NOT BE REVIEWED*** NIH: NOT-OD-16-011

AUTHENTICATION OF KEY BIOLOGICAL AND/OR CHEMICAL RESOURCES Transparent reporting of what was done to authenticate key resources One description for multiple resources in same category: example – authenticating cell lines Not necessary to show actual data on authentication If key resource is being made as part of project or under development – include in research strategy Could include company/manufacturer’s information/criteria for validation Save this information in a single PDF file named “Authentication of Key Resources Plan,” and attach it on the R&R other project information NO PAGE LIMITS NIH: NOT-OD-16-011

Plan for Resource Authentication: Guidance for Reviewers NIH: NOT-OD-16-011 GOAL: Ensure processes are in place to identify and regularly validate key resources used in their research and avoid unreliable research as a result of misidentified or contaminated resources. Researchers are expected to authenticate key biological and/or chemical resources used in their research, to ensure that the resources are genuine. New Additional Review Consideration Authentication of Key Biological and/or Chemical Resources: For projects involving key biological and/or chemical resources, reviewers will comment on the brief plans proposed for identifying and ensuring the validity of those resources. Rate as acceptable/unacceptable (provide brief explanation if unacceptable) Does not affect criterion scores or overall impact score If reviewers have serious concerns about the authenticity of resources and the feasibility of the research given the resources to be used, these concerns can be addressed in the approach and affect the overall impact score; concerns about the plan to authenticate resources should be noted as unacceptable but should not affect the approach or overall impact score. SOURCE:NIH,CSR

Related review issues: Different research fields may have different best practices for and reach different conclusions about scientific premise and rigor. Assess based on best practices in the field. Page limits have not changed. Be alert for page limit violations (e.g. inappropriate use of appendices or other application sections). Alert the SRO if you see a potential issue. Page limits, cost and time are not valid reasons to disregard attention to these issues. Investigators address rigor and transparency differently (e.g. in labeled sections vs. throughout the research plan). Focus your evaluation on the likely outcome, not grant writing preferences. Rigor and transparency considerations apply to R03 (small grant) and R21 (exploratory/developmental) applications. However, preliminary data are not required and the extent to which approach details can be provided may differ. Reviewers should evaluate the scientific merit of these applications, including rigor and transparency, in light of the goals and reviewer guidelines for these activities.

Reviewing Rigor and Transparency of Research: RPG Applications Applies to which applications? Where will I find it in the application? Where do I include it in my critique? Addition to review criteria Affect overall impact score? Scientific Premise All Research Strategy (Significance) Significance Is there a strong scientific premise for the project? Yes Scientific Rigor Research Strategy (Approach) Approach Are there strategies to ensure a robust and unbiased approach? Consideration of Relevant Biological Variables, Such as Sex Projects with vertebrate animals and/or human subjects Are adequate plans to address relevant biological variables, such as sex, included for studies in vertebrate animals or human subjects? Authentication of Key Biological and/or Chemical Resources Project involving key biological and/or chemical resources New Attachment Additional review considerations Comment on plans for identifying and ensuring validity of resources. No Rigor and transparency do not apply to all applications.  See List of Eligible Activity Codes: https://nih-extramural-intranet.od.nih.gov/d/sites/default/files/RigorActivityCodes-20151006.pdf.  Also, certain Funding Opportunity Announcements are exempt from rigor, by request from the ICs. SOURCE:NIH,CSR

Reviewing Rigor and Transparency of Research: Mentored Career Development Applications Applies to which applications? Where will I find it in the application? Where do I include it in my critique? What should I consider? Affect overall impact score? Scientific Premise All Research Strategy Research Plan Is there a strong scientific premise for the project? Yes Scientific Rigor Are there strategies to ensure a robust and unbiased approach? Consideration of Relevant Biological Variables, Such as Sex Projects with vertebrate animals and/or human subjects Are adequate plans to address relevant biological variables, such as sex, included for studies in vertebrate animals or human subjects? Authentication of Key Biological and/or Chemical Resources Projects involving key biological and/or chemical resources New Attachment Additional review considerations Comment on plans for identifying and ensuring validity of resources. No Rigor and transparency do not apply to all applications.  See List of Eligible Activity Codes: https://nih-extramural-intranet.od.nih.gov/d/sites/default/files/RigorActivityCodes-20151006.pdf.  Also, certain Funding Opportunity Announcements are exempt from rigor, by request from the ICs. SOURCE:NIH,CSR

NIH BIOSKETCH (NIH: NOT-OD-16-080) KEY: HIGHLIGHTING SCIENTIFIC CONTRIBUTIONS BE OBJECTIVE: DON’T OVERSELL OR UNDERSELL YOURSELF CLAIMS TO BE BACKED UP BY PUBLICATIONS Provide subheader to each of 5 thematic areas DATA (GRAPHS/FIGURE/TABLES) DO NOT BELONG HERE CONFERENCE PROCEEDINGS – MEETING ABSTRACTS, POSTERS AND OTHER PRESENTATIONS CAN BE INCLUDED MANUSCRIPTS UNDER DEVELOPMENT CAN BE MENTIONED IN NARRATIVE BUT CANNOT BE CITED Optional: PROVIDE LINKS TO PUBLICATIONS VIA SciENcv or MyBibliography Examples: Google Scholar page: https://scholar.google.com/citations?user=wiCTwdYAAAAJ&hl=en Link to all publications: http://www.ncbi.nlm.nih.gov/pubmed?term=sen%20ck Complete List of Published Work in MY Bibliography: http://www.ncbi.nlm.nih.gov/myncbi/browse/collection/43912766/?sort=date&direction=descending  

INCLUSION OF CHILDREN IN CLINICAL RESEARCH: CHANGE IN DEFINITION NIH’s long-standing policy has been that children must be included in all human subjects’ research, conducted or supported by the NIH, unless there are scientific and ethical reasons not to include them.  The policy was developed because medical treatments applied to children are often based upon testing done only in adults, and scientifically evaluated treatments are less available to children due to barriers to their inclusion in research studies.  Therefore, applicants/offerors conducting human subjects’ research must include a description of plans for including children.  If children (or a subset of children) will be excluded from the research, the application or proposal must present an acceptable justification.     What’s Changing:  Age: individuals under 18 years old (previously under 21 years) Expected to justify the age range of the proposed participants in their clinical research, with particular attention paid to addressing the inclusion (or exclusion) of children (or subsets of children).  NIH: NOT-OD-16-010

SIMPLIFICATION OF VERTEBRATE ANIMAL SECTION DESCRIPTION OF PROCEDURES: Concise Identify species, strains, ages, sex, number of animals For studies involving dogs/cats – source of animals JUSTIFICATIONS: Why must the proposed model be used? MINIMIZATION OF PAIN: Description of interventions to minimize discomfort or distress EUTHANASIA: American Veterinary Medical Association (AVMA) Guidelines must be followed – just need to state it. If not, describe method and provide scientific justification NO LONGER NEED: DESCRIPTION OF VETERINARY CARE OR JUSTIFICATION OF NUMBER OF ANIMALS NIH: NOT-OD-16-006

Additional resources Rigor and Reproducibility in grant applications (OER site): http://grants.nih.gov/reproducibility/index.htm NIH presentation of background and goals of Rigor and Transparency (video) https://grants.nih.gov/reproducibility/module_1/presentation.html Reviewer Guidance on Rigor and Transparency: http://grants.nih.gov/grants/peer/guidelines_general/Reviewer_Guidance_on_Rigo r_and_Transparency.pdf Consideration of Sex as a Biological Variable in NIH-funded Research http://orwh.od.nih.gov/sexinscience/overview/pdf/NOT-OD-15-102_Guidance.pdf Rigor and transparency do not apply to all applications.  See List of Eligible Activity Codes: https://nih-extramural- intranet.od.nih.gov/d/sites/default/files/RigorActivityCodes-20151006.pdf.  Also, certain Funding Opportunity Announcements are exempt from Rigor and Transparency, by request from the ICs. Questions about the NIH policy should be directed to reproducibility@nih.gov