NIH Grant Writing Workshop 4/11/2002 NIH Grant Writing Workshop Susan McHale Professor of Human Development; Director, SSRI Douglas M. Teti Professor of Human Development, Psychology, and Pediatrics; Associate Director, SSRI Rhonda BeLue Associate Professor of Health Policy and Administration Kristin Buss Professor of Psychology, College of the Liberal Arts Michelle Frisco Associate Professor of Sociology and Demography, College of the Liberal Arts NIH Fundamentals

Workshop Outline NIH Organization NIH Funding Mechanisms The Grant Writing Process Focus on the R01 The NIH Review Process Overview of Review Meeting The Scoring Process A Penn State example Workshop Evaluation

4/11/2002 Thanks to: Lori Francis, Associate Professor of Biobehavioral Health and Center for Family Research in Diverse Contexts, PSU Donna Panasiti, Social Science Research Institute NIH is an agency of DHHS (one of 12) Other HHS Agencies include FDA, CDC, HRSA, SAMSA, IHS IC = Institute or Center NIH Fundamentals

I. The NIH Department of Health and Human Services 4/11/2002 I. The NIH Department of Health and Human Services National Institutes of Health 25 Awarding Institutes/Centers aka ICs (e.g., NICHD, NIMH, NIDA, NIA) Center for Scientific Review Office of the Director NIH is an agency of DHHS (one of 12) Other HHS Agencies include FDA, CDC, HRSA, SAMSA, IHS IC = Institute or Center NIH Fundamentals

The NIH Extramural Team 4/11/2002 The NIH Extramural Team A. Program B. Grants Management C. Review NIH Fundamentals

A. The Program/Institute Staff Program Administrator 4/11/2002 A. The Program/Institute Staff Program Administrator Maintains knowledge of scientific area Attends study section meetings Makes funding recommendations Monitors scientific progress Identifies scientific area of importance Reports to senior staff Development of programs and initiatives Different titles for same role. All are synonymous. They are the advocates at the IC level for grants in their portfolio Program Reports--may be used for Congress NIH Fundamentals

B. Grants Management Interprets Federal regulations and policies 4/11/2002 B. Grants Management Interprets Federal regulations and policies Assures compliance with Federal regulations and policies Monitors financial aspects of projects Interprets regulations and policy Interact mostly with PD Involved with grants “cradle to grave”. Advice when preparing/submitting an application through to Closeout. NIH Fundamentals

C. Review: Scientific Review Group (1st Level) 4/11/2002 C. Review: Scientific Review Group (1st Level) Center for Scientific Review (CSR) or NIH Institute & Center (IC) Scientific Review Group (SRG) Non-federal scientists with relevant expertise Led by a Scientific Review Officer (SRO) http://www.csr.nih.gov/Roster_proto/sectionI.asp NIH Fundamentals 8

C. Review: Advisory Council or Board (2nd Level) 4/11/2002 C. Review: Advisory Council or Board (2nd Level) The potential awarding IC performs the second level of review. Comprised of scientists from the extramural community and public representatives. NIH program staff examine applications for impact (formerly “priority”) scores, percentile rankings, & summary statements against the IC’s needs. Program staff provide grant funding plan to Advisory Council or Board. Advisory Council or Board advises the IC director. Director makes final decision. NIH Fundamentals 9

NIH Grant Application Cycle 4/11/2002 NIH Grant Application Cycle Investigator Institution NIH Initiates Research Peer Review Submits Application Council Review Allocates Funds Funding Decision Conducts Research NIH Fundamentals

Grant Application: It’s a process, not an event 4/11/2002 Grant Application: It’s a process, not an event Communicate with Program Officer Introducing ideas, getting feedback, pre-review 2. Get your proposal to the right review committee Review the rosters and talk to colleagues Effectively wording the abstract Make a written request 3. Seek feedback from colleagues and consultants on drafts of the grant (prepare ahead!) 4. Consider who is likely to review your grant (review the rosters) and make sure to know and cite their work when relevant 5. Recognize that funding on first submission is rare NIH Fundamentals

II. NIH Grant Mechanisms Ks: NIH Career Development Awards (K01, K02, K05, K07, K08, K22) P01: Research Program Project Grant P30: Center Core Grants R01: NIH Research Project Grant Program R03: NIH Small Grant Program R13: NIH Support for Conferences and Scientific Meetings (R13, U13) R15: NIH Academic Research Enhancement Award (AREA)

NIH Grant Mechanisms (continued) R21: NIH Exploratory/Developmental Research Grant Award R34: NIH Clinical Trial Planning Grant T series: NRSA Training Grants (T32, T34, T35, T90, etc.) U series: Research Project Cooperative Agreement Diversity Supplements: Research Supplements to Promote Diversity in Health-related Research Roadmap: NIH Roadmap Initiatives (Director’s Pioneer Award; Director’s New Innovative Program

Pre-doctoral and Post-doctoral Training Awards (F-series) 4/11/2002 Pre-doctoral and Post-doctoral Training Awards (F-series) Focuses on the training environment, research training plan, and the candidate’s strengths Research strategy is 6 pages F31 covers doctoral study tuition and living stipend (levels set by NIH). Up to 60 months of support. F32 covers living stipend and funds for research/professional development NIH Fundamentals

Career Awards (K-series) Nine different Career Development (K) Awards Body of application is 12 pages for most Support individuals after they have completed clinical training and have accepted a faculty position Provide for release time for research and monies for research/faculty development

R03 Small research grant Research strategy is 6 pages Can be used to collect pilot data, conduct feasibility studies Often analysis of an existing dataset May be used to develop new research technology $50K per year for 2 years

R21 Exploratory or Developmental Grant Often little or no pilot data Research strategy is 6 pages Often little or no pilot data Feasibility study 2 years of funding (Total of $275K) Not necessarily the first step for new investigators

R34 Supports the development of Phase III clinical trials Establishment of the research team Development data management tools Definition of recruitment strategies Finalization of the protocol Research strategy is 6 pages The planning grant is designed to lead to an application for support of a full-scale trial

R15 Area Award Used to stimulate research in educational institutions that have not been major recipients of NIH support Research strategy is 12 pages Opportunity to conduct research; strengthen the research environment of the institution; and benefit the student through exposure to and participation in research Limited to up to three years of funding for total of $150,000 Preliminary data generally not required

R01 (primary focus today) Used to support a discrete, specified, circumscribed research project NIH’s most commonly used grant program Research Strategy is 12 pages Advance permission required for $500,000 or more (direct costs) in any year Generally awarded for 3 to 5 years

Research Center Grants (P-series) 4/11/2002 Research Center Grants (P-series) Established by ICs to meet special needs May support research and/or core facilities Usually initiated by the IC Request for Applications (RFA) P30, P50, P60, U54 P01 Center grants are investigator-initiated, linked R01s with an additional core of funding NIH Fundamentals

Research Supplements to Promote Diversity in Health-Related Research Administrative supplement to R01 grants and many other types Need to have at least 2 years of funding left on the grant Contact project officer for permission to submit Can be used for students, doctoral students, junior investigators

New and Early Stage Investigators: A Competitive Edge New Investigator has not previously served as a PI for an R01; may have been an investigator or received other smaller, developmental or research training awards Early Stage Investigator (ESI) is within 10 years of completing his/her terminal research degree, or is within 10 years of completing medical residency

III. The Grant Writing Process Grant writing is: A skill like any other… But not the same skill as article writing Instead, more of a problem-based writing activity (theory and practice problem)

A few preliminary tips Start early, make a timeline and STICK TO IT Should allow time for serious pre-submission review & subsequent revision Develop a relationship with project officers It is not possible to overdo clarity Let your passion come through in your proposal Take advantage of early stage and new investigator opportunities

Getting ready to write Know what has been done Know what has been funded NIH website Decide on the problem Important enough to get funded but simple enough to explain as clean design in 12 pp (for the R01; less for other mechanisms) Mediators and moderators, mechanisms of change Assemble team--CAREFULLY Preliminary research (R01)

Getting ready to write Communicate with program officer Establish a relationship and trust (funding decision) Acquire information on mechanism and priorities Obtain input on aims/proposal

Main Sections of the NIH Application (see Francis et al. for example) Face Page Table of Contents Performance Sites Other information Project Summary/Abstract (Description) Public Health Relevance Statement Facilities & Resources

More Sections Key Personnel Budgets (for each study year) Biosketches --with personal statements Budgets (for each study year) Budget Justification Clinical Trial and Human Embryonic Stem Cell (HESC) List of Research Plan Attachments

Main Sections of the R01: Specific Research Plan [Introduction – revisions only] Specific Aims: The basis for the proposal’s organization Research Strategy Significance and Innovation Approach Preliminary studies Design Sample/recruitment/power analyses Procedures & measures Analyses

Page Limit Guide: Plan your proposal with these limits in mind Section of Application Page Limits Introduction (for resubmission application only) 1 Specific Aims Research Approach: R03, R13/U13, R21, R36, R41, R43, Fellowships (F), SC2, SC3 6 Research Approach: R01, single project U01, R10, R15, R18, U18, R33, R24, R34, U34, R42, R44, DP3, G08, G11, G13, UH2, UH3, SC1 12 Biographical Sketch 4 Page limits may vary for other funding mechanisms. Check Funding Opportunity Announcement: http://grants.nih.gov/grants/guide/search_results.htm?scope=pa&year=active

And More Sections Protection of Human Subjects References Cited Women and Minorities Planned Enrollment Table Children References Cited Letters of Support Resource Sharing Plan Checklist SO START EARLY!

Keys to R01 Success Impact and Significance Theory (model) Innovation Practice (2-3 sentences) Prevalence of problem in population Important social concern Theory (model) Building, testing, using Innovation New directions, value added Importance of Preliminary Research

Keys to Success - Methods Methods are very important Overall -- clarity and detail May include a table that traces aims to hypotheses to constructs to measures (table/s) Is the design feasible? Are there gaps in the methods (e.g., fidelity for interventions) Stats are very, very important (methodologist team member) Include a detailed timeline

More Methods - Issues Community-based participatory research -- very strict about how this is done. Lab versus field Focus groups, interviews, analyses

And More Methods: The Sample Preference for representative samples Students only if relevant to age/situation (e.g., college drinking) Generalizability from a single entity (university, clinic, state) Unit of assignment is unit of analysis

Budgets Direct Costs Senior Personnel (PI, co-Is, project director) (PSU fringe at 36.5%) Other Personnel (full time staff, RAs, part time wages) (PSU fringe 36.5% for full time staff, 13.2% for Grad Assts AY; 7.9% for part time wages and summer) Equipment Travel Participant/Trainee Support Costs

Budgets.2 Other Materials and supplies Publication costs Consultant services Subawards/Consortium/Contractual Fees Indirect Costs (> 45% at PSU, but does not include all expenses) Budget justification

Receiving the Summary Statements: The Hardest Part! Reviews critical, even harsh Reviewers usually find grant’s weaknesses, while recognizing strengths Summary statements spend much more time on critique than praise Many investigators experience a mixture of rage and depression when they read their summary statements and easily lose perspective Take a day or two (or more!) and then read again with a cooler head

Receiving the Summary Statements: Bouncing Back! Ask experienced colleagues to read reviews Don’t interpret criticism as hopeless Program Officer may be helpful in clarifying critique If “discussed” (rather than triaged), you have a chance of funding in next round The lower the initial score, the fewer problems and more likely to be successful after revision

Resubmission: Resilience and Flexibility! Persistence pays off in the grant process!! Second submission must respond to the critiques through revision or clearly defending reasoning Same reviewers may or may not review resubmission, but will see critique

Most Common Reasons for a Poor Score (in priority order) Lack of impact or significance Lack of new or original ideas – show your passion! Hypotheses ill-defined, superficial, lacking, unfocused, or unsupported by preliminary data Methods unsuitable, not feasible, not rigorous or not likely to yield results; methods don’t clearly link to aims Design not logical, inappropriate instrumentation, poor timing or conditions; doesn’t link well to aims Data management and analysis vague, not rigorous; analyses don’t clearly link to aims Inadequate expertise or knowledge of field for PI; too little time to devote to the work Poor resources or facilities; limited access to appropriate population

When to Revise Basic idea was significant and innovative or these can be bolstered Design/measurement/analysis problems can be clarified (more information) or fixed Need preliminary data Problem is poor writing

IV. The NIH Review Process A. The Review Meeting B. Review Discussion C. The Scoring Process D. A Penn State Example

A. The Review Meeting: The SRO’s Role Prior to Meeting Point of contact until review group meets (then project officer) Analyze submissions for completeness and conflicts Recruit ad hoc reviewers as needed Schedule 1-2 day meeting Assign applications to reviewers (at least 3) Primary, secondary, discussant Create review order based on preliminary impact scores from best to worst within categories

Reviewers’ Role Prior to Meeting Familiarize self with criteria, mechanisms, and scoring Review assigned applications Assign scores to each criteria and other areas Write bulleted strengths and weaknesses for each criteria Reviews are advice to institutes for funding decisions, not advice to PI Post scores and comments on NIH Commons Read other reviews of assigned applications Prepare presentation of reviews Skim/read non-assigned applications

Format of the Review Meeting SRO opening remarks Chair orientation New investigator R01 grants Other R01 grants Other grant types (R03, R15, R21, R34) Applications discussed in order of Impact Score; bottom 50% are not discussed

SRO Opening Remarks Confidentiality Review order Proposals below median within each category may not be discussed

Chair Orientation Start with reviewer impact scores May differ from posted scores Goal of discussion is to clarify not reach agreement If scores are similar, shorter discussion If scores are dissimilar, longer discussion Recommended time Primary – 5 minutes Secondary – 3 minutes Discussant – 2 minutes

B. Review Discussion Identify proposal Members in conflict leave Reviewers provide preliminary impact scores Reviews Impact, Significance, Investigators, Innovation, Approach, Environment Stress main points, do not repeat previous points Non-reviewers typically ask questions to clarify Human Subjects issues affecting scoring Open discussion to entire committee

Review Discussion (continued) Ask for reviewers impact scores again Identify the reviewers’ recommended range Ask if anyone wants to score outside the range Entire committee records impact score Discuss budget and other issues

4/11/2002 C. The Scoring Process 1. Overall Impact Score: likelihood project will “exert a sustained, powerful influence on the research field(s) involved (1-9 scale) 2. A separate 1-9 score for each of 5 core criteria (Significance, Investigators, Innovation, Approach, Environment) 3. Additional review criteria help determine scientific and technical merit BUT are not scored separately 4. Additional review considerations are addressed by reviewers, but are not scored & are discussed after group scores. NIH Fundamentals 52

4/11/2002 Score Criteria Overall Impact: will project exert a sustained, powerful influence on the research field(s) as indexed by 5 core review criteria 1. Significance: important problem addressed; how will this improve scientific knowledge, technical capability, and/or clinical practice NIH Fundamentals 53

What is the Difference Between Impact and Significance ? Impact Addresses: Probability of whether the research will exert a sustained, powerful influence on the research field Significance Addresses: Does the project address an important problem or a critical barrier to progress in the field? If the aims are achieved, how will scientific knowledge, technical capability, and/or clinical practice be improved?

Score Criteria (continued) 4/11/2002 Score Criteria (continued) 2. Investigators: PI & other researchers well suited to the project; appropriate experience & training; ongoing record of accomplishments; complementary & integrated experience; leadership approach, governance, and organizational structure appropriate for project 3. Innovation: the work challenges and seeks to shift current research or practice paradigms; utilizing novel theory, approaches or methods, instrumentation, or interventions; the work is novel NIH Fundamentals 55

Score Criteria (continued) 4/11/2002 Score Criteria (continued) 4. Approach: strategy, methodology, analyses are well-reasoned and appropriate; potential problems & alternative strategies thought through; benchmarks set; risk is managed 5. Environment: the environment will contribute to the project’s success; institutional support, equipment, & other resources sufficient; unique features of the environment, subject population, collaborative arrangements NIH Fundamentals 56

Additional Review Criteria (not scored) 4/11/2002 Additional Review Criteria (not scored) Human Subjects: Protection of human subjects Data safety monitoring plan (clinical trials only) Inclusion of women, minorities, children Vertebrate animals Biohazards NIH Fundamentals 57

Additional Review Considerations 4/11/2002 Additional Review Considerations Budget and period of support Select agent research (infectious agents) Applications from foreign organizations Resource sharing plans Additional comments to applicant NIH Fundamentals 58

CRITERIA SCORING SYSTEM HIGH Exceptional: Exceptionally strong with essentially no weaknesses Outstanding: Extremely strong with negligible weaknesses Excellent: Very strong with only some minor weaknesses MEDIUM Very Good: Strong but with numerous minor weaknesses Good: Strong but with at least one moderate weakness Satisfactory: Some strengths but also some moderate weaknesses Low Fair: Some strengths but with at least one major weakness Marginal: A few strengths and a few major weaknesses Poor: Very few strengths and numerous major weaknesses

CRITERIA SCORING SYSTEM (continued) Minor Weakness: An easily addressable weakness that does not substantially lessen impact Moderate Weakness: A weakness that lessens impact Major Weakness: A weakness that severely limits impact

CRITERIA SCORING SYSTEM (continued) Final Overall Impact Score: Mean of all reviewers’ final impact scores X 10 Range = 10 (high impact) -- 90 (low impact) NOTE: New scoring likely to produce more applications with identical scores (“tie” scores). Thus, other factors (e.g., mission relevance, portfolio balance) will be considered when all other things are essentially equal

Key Sections in R01 Proposal Format Research Plan Components Specific Aims Includes language about the impact of the research Research Strategy Includes Background & Significance; Preliminary Studies/Progress Report; Research Design & Methods Facilities and Equipment Reflects the Environment criterion For ESIs describes the institutional investment in the success of the investigator Biographical Sketch Requires Personal Statement; no more than 15 pubs based on recency, importance to field, and/or relevance to the application

Key sections (cont’d) Innovation Does application challenge/seek to shift current research or clinical practice paradigms by utilizing novel theoretical concepts, approaches or methodologies, instrumentation, or interventions? Concepts, approaches or methodologies, instrumentation, or interventions novel to one field of research or novel in a broad sense? Refinement, improvement, or new application of theoretical concepts, approaches or methodologies, instrumentation, or interventions proposed? Not all applications need to be innovative!

Key sections (cont’d) Approach Are the overall strategy, methodology, and analyses well-reasoned and appropriate to accomplish the Specific Aims of the project? Are potential problems, alternative strategies, and benchmarks for success presented? If the project is in the early stages of development, will the strategy establish feasibility and will particularly risky aspects be managed?

Key Sections (cont’d) Investigators Personal Statement: Why their experience and qualifications make them particularly well-suited for their roles in the project Publications: Recommended: no more than 15---up to five of the best; up to five of the most relevant to the proposed research; up to five of the most recent If Early Stage Investigators or New Investigators, do they have appropriate experience and training? If Established, have they demonstrated ongoing record of accomplishments that have advanced their field(s)? Okay. I changed the title

D. A Penn State Example Rhonda BeLue, Associate Professor of Health Policy and Administration, College of Health and Human Development Kristin Buss, Professor of Psychology, College of the Liberal Arts Michelle Frisco, Associate Professor of Sociology and Demography, College of the Liberal Arts

Links of Interest Enhancing Peer Review Criteria: http://grants.nih.gov/grants/guide/notice-files/not-od-09-025.html Page Limits: http://grants.nih.gov/grants/forms_page_limits.htm Human Subjects: http://grants.nih.gov/grants/policy/hs/index.htm SF424 guidelines for submission: http://grants.nih.gov/grants/funding/424/index.htm Glossary: http://grants.nih.gov/grants/glossary.htm

General Resources NIH RePorter (formerly CRISP) https://libraries.ucsd.edu/info/resources/nih-reporter-formerly-crisp Revised Applications http://grants.nih.gov/grants/policy/amendedapps.htm NIH Grant Writing Tip Sheets http://grants.nih.gov/grants/grant_tips.htm Getting an RO1 http://sciencecareers.sciencemag.org/career_development/previous_issues/articles/1190/getting_an_nih_r01

More general resources NSF Proposal Writing http://www.cs.cmu.edu/~sfinger/advice/advice.html Other Proposal Writing Guides http://www.learnerassociates.net/proposal/ Reasons Proposals Fail http://chronicle.com/article/How-to-Fail-in-Grant-Writing/125620/--let

New and Early Stage Investigator Resources New and Early Stage Investigators http://grants.nih.gov/grants/new_investigators/

NIH Websites http://public.csr.nih.gov/Pages/default.aspx http://grants.nih.gov/grants/oer.htm (forms, grant search, etc.)

SSRI Listserv New subscribers can join the SSRI listserv by sending mail to: mailto:L-SSRI-subscribe-request@lists.psu.edu