1. Writing implementation research grant proposals: 10 key ingredients
Writing implementation research grant proposals: 10 key ingredients....and a few reflections
Russell E. Glasgow
Materials adapted from
Enola Proctor and Washington Univ. colleagues
Univ. of Colorado School of Medicine ACCORDS D&I Program
NIH/Academy Health Technical Assistance, Dec. 2017

2. Session overview Challenges facing D&I Ten tips for D&I grant writing
Additional reflections and recommendations
Importance of working with program staff
We have three goals for today’s session:
Reviewing and discussing the challenges facing IR
Focusing on some tips for successful grant writing in this field
Underscoring the importance of working with program staff
Proctor et al. Writing implementation research grants. Implementation Sci :96
Brownson et al. Successful grant application writing. Clin Transl Sci. 2015 Dec;8(6):710-6.

3. Challenges universal to all grant writers
How to successfully communicate:
How your proposal is innovative and state of the science
Why you and your team should be funded at this time
That you can complete the study as proposed
All grants are assessed in terms of feasibility.
Especially for early stage investigators, it is critical to demonstrating that
the application reflects a well-thought out, detailed approach that can be successfully completed.

4. Additional challenges for implementation science researchers
Rising scientific bar:
Science of implementation demands moving beyond documentation of barriers and facilitators
Implementation research is complex, contextual & complicated
Literature scattered across disciplines
Implementation science is setting specific, but must advance generalizable science
Addressing both internal and external validity
But our focus today is implementation research, a still and rapidly developing field with particular challenges for grant writing.
Some of challenges are reflected on this slide…
I often characterize IS as demanding a “double major”
Implementation researchers must know the literature in their own area of health or health services
But also must keep track of the scattered literature that comprises implementation research.
Brownson, Proctor, & Colditz. Dissem & Impl Research in Health. Oxford: 2017 (2nd ed.) Glasgow et al. NIH approaches to D&I science. Am J Public Health. 2012;102:1274–1281.

5. The Question Aims and Approach Talk with Program Officer
Most important
The Question Aims and Approach Talk with Program Officer
Before we get to the tips per se,
Need to underscore that the most important factor in any grant proposal is the question or questions the proposal is designed to answer…
The question needs to be significant,
it needs to be feasible, it needs to be timely, and it needs to be one whose answer will advance the field.
The question being posed in a grant proposal must be innovative and significant to the field of implementation science….
Thus the demand to know the IS literature, in addition to the literature in your own particular area. Now to some specifics…

6. Presuming a Compelling Question
Ten Key Ingredients of a Competitive D&I Grant Application (no application will have all 10)

7. 1. Need for improvement….reducing gap between care that is and could be
Contribution: Project’s public health significance & impact Important to demonstrate: Poor health services or wide variation Potential to improve care through proposed work How? SUCCINCT Literature review Preliminary data
Reviewers will ask:
Does the proposal clearly demonstrate a need?
How great is the gap between usual services and EB services?
How?
practice variation research
Empirical literature (health services, QOC data)
Stakeholder letters

8. 2. Evidence-based intervention to be implemented
Demonstrate: Strong evidence* of efficacy and/or effectiveness Cochrane or other systematic review (ideally also pragamtism/PRECIS) Ready for implementation How your proposal is different or contributes to what already known How? SUCCINCT Literature review Preliminary data
Impt to demonstrate that an evidence-based treatment is ready for implementation research?
As disc in paper, thresholds vary but meeting some recognized threshold is important
For researchers who come to IR as treatment developers, effectiveness researchers, important for this to be part of own background work
For others, lit,
*Mehta et al. Tyranny of evidence. In A. Plough (ed.). Knowledge to action. Oxford, 2017

9. 3. Conceptual model/ theoretical framework
Selection and Contribution:
Why this model is applicable (and not others)
Innovation for implementation science
Scientific impact & generalizable knowledge gain
Justifies key variables to be tested
How?
Published papers using the model
Use the theory throughout proposal methods, measures, hypotheses, analyses!
A second key component of a grant application is a theoretical framework that is suited for and informs the proposed study.
Theory: not just name dropping
It is important to spell out how your potential findings will help confirm or raise questions about the conceptual model…..
Rarely a need to add to the proliferation of already published models
Refer to you Tabek et al., review and classify 109 different models , and how to select model
Tabak et al. Models for dissemination and implementation research. Am J Prev Med. 2012;43(3):337-50
Dissemination-implementation.org

10. 4. Stakeholder priorities, engagement in change & study
Contribution: Significance, impact, & feasibility of success.
- Meaningful involvement THROUGHOUT and with MULTIPLE stakeholders, at different levels (patients, staff, decision makers)
How?
Preliminary data (qualitative, quantitative)
Evidence of current and past partnerships (joint publications)
Method details (partnered research)
Letters- unique (not form letters) and from key decision makers
Successful I of EB programs &*services depends on their fit with preferences & priorities of those who shape, deliver, participate in health care
Who are the stakeholders?
service consumers
families
providers
administrators
funders
Legislators
What are their priorities?
for changing care, research, I process

11. 5. Setting’s readiness to adopt new intervention
Contribution:
Scientific generalizabilty & impact
Shows PI knowledge of study setting
Minor caveat- if large scale (e.g., national) dissemination, may want to take settings with a ‘range of readiness’
How to convey?
Preliminary data
Letters from specific organizations and national coordinating or professional organizations
Setting’s readiness to adopt new services/ TX/ interventions/ programs
Data:
Org culture, climate, provider attitudes
Prospective iD of barriers,
or prelim data testing EBI re: IO’s
BUT a study also may test/ establish answer to this Q

12. 6. Implementation strategy/process
The observed/introduced change STRATEGY
Contribution:
Public health significance
Impact
Feasibility- address likely challenges
How?
Detail in planned approach- differentiate impl. strategy vs. Interv.
Literature cited
Preliminary studies
Specify, provide manuals
Lit on implementation strategies may be among the least developed in the field
Few studies describe strategies in sufficient detail, conceptually or operationally, We have recommended reporting standards for strategies
There are few robust tests of strategies, especially comparative tests
Refer you to
Powell and colleagues, A compilation of strategies for implementing clinical innovations in health and mental health
In the 2012, or
Vol 69 of Medical Care Research and Review
Also IS 2013

13. 7. Team experience w/ setting, treatment, implementation
Contribution:
Feasibility
Capacity to complete study as proposed, including experience addressing expertise, recruitment, diversity, attrition
How?
Build team, cite work
Describe relevant experience in preliminary studies
Personalized Biosketches, adequate budget justifications
Personalized, unique letters of support
Implementation science is not a field I would recommend playing alone….challenges are too numerous and diverse.
A team can help ensure that the project can be successfully completed.
Important to demonstrate that the team has worked together, or can work together…thru
Preliminary studies
Team composition & Biosketches
Resources & Environment

14. 8. Feasibility of proposed research design & methods
Value: Conveys-
Feasibility of completing study as proposed
Investigator capacity (understanding of unique D&I issues and challenges)
Balance of internal and external validity
How?
Detailed “approach” section- save space for this!
Address choice junctures & contingencies
Preliminary recruitment & enrollment data
Letters (re: willingness to be randomized)
Grant writers urged to consider every possible question reviewers might raise about feasibility
What can go wrong?
How have I mitigated those problems?
Real world challenges:
agency/ provider reluctance to randomization
small “n” (changing even one agency is a challenge)
nesting: clients, within providers, within teams/ supervisors, within agency, within one policy context
Refer you to:
Landsverk and colleagues, chapter “Design and Analysis in Dissemination and Implementation Research, in book by Brownson, and colleagues, referenced on earlier slide

15. 9. Measurement and analysis detail
Contribution:
Approach
Feasibility of completing study as proposed
Relates to theory; advances measurement science
How?
Detailed measurement plan- PRAGMATIC measures*
Variation data and well-justified power analysis
Unit of analysis specified & consistent
Specified analyses that will exploit data and answer Q’s
Measurement plan is extremely important…
Measures should be tied to every construct in conceptual model, every variable to be measured…
Moreover, field needs harmonization in measurement….
Rabin paper,
And chapter on measurement in the Brownson et al book
Glasgow & Riley. Pragmatic measures. Am J Prev Med 2013;45(2):237–243
Kroenke, Monohan & Kean Pragmatic charac.of PROs. J Clin Epidemiol. 2015; 68(9): 1085–92. 

16. 10. Policy environment will leverage, support, sustain the change
Contribution:
Public health significance and impact
Sustainability
Feasibility- including costs, resources and burden- to both participants and settings
How?
Background literature on adaptations and sustainability models and research
‘Fit’ with changing context; feasible and sustainable in these settings
Letters with strong, specific commitment to continue
Resources & Environment
Finally, grant application should reflect PI’s understanding of the policy and funding context of implementation…..
Policy can be a lever
Also critical component of the context
Raghavan chapter in Brownson et al book
Chambers, Glasgow & Stange. Dynamic sustainability framework. Implementation Science 2013, 8:117
Wiltsey Stirman et al. Sustainability syst. review. Implementation Science 2012, 7:17

17. Can’t fit it all in!
Relative importance in eyes of experienced reviewers?
The Questions, the Aims, and Approach, Approach, Approach
Most essential:
Background of Evidence based Tx to be implemented
Methods and the team’s experience with setting/Tx/ implementation strategies and process
Research environment’s capacity to support IR
Page limits preclude exhaustive treatment on all 10 components
Asked small group of successful implementation researchers to rate 1-3
1= very important
2=helpful but not necessary
3 = not very important
They rated everything as important or helpful (nothing rated as not important) , and sometimes offered comments that “it’s all essential”

18. How to address these challenges?
Make strategic use of all grant application parts...especially Aims and summary
Draw on your experiences and unique strengths
pilot studies
work & research training experiences
Caveat: Not every application can/should include equal detail on each component
So, how to pull this off?
Suggest using checklist in paper as an “audit” of grant….
Checklist to help guide the grant writing,
Then as a review nearer the completion of the application
Brownson. Video on aims.

19. Russ’s Reflections and Biases
Work with your program officer- contact multiple ones if needed
Importance of Aims and Abstract- for both assignment and reviewers initial impression
Tables and Figures- clarify and worth a lot of points
Address tough decisions and weaknesses head-on
Consider using PRECIS-2 Figure
Consider using Standard Reporting for Research in Implementation (StaRI) reporting criteria
Loudon et al. (2015). The PRECIS-2 tool: designing trials that are fit for purpose. BMJ, 350, h2147.
Pinnock et al. StaRI statement. BMJ Open, doi: /bmjopen )

21. StaRI Reporting Criteria
27 criteria- includes almost all CONSORT items .... PLUS:
Implementation Strategy
Implementation Outcome(s)
Logic Pathway
Harms and Unintended Consequences
Context and Representativeness
Context Changes
Fidelity
Adaptations*
Costs and Resources Used Policy Implications
Pinnock et al. StaRI statement. BMJ Open, doi: /bmjopen )
*Chambers & Norton. The Adaptome. Am J Prev Med. 2016;51(4):S124– S131 

22. Acknowledgments and Key Resources
Enola Proctor, Ross Brownson, Ana Bauman and team at Washington University
Proctor et al. Writing IR grants. Implementation Sci :96
Brownson et al. Successful grant application writing. Clin Transl Sci. 2015
Dec;8(6):710-6.
NIH Program Officers
Univ. of Colorado School of Medicine ACCORDS D&I team:
Acknowledge my sources of support
Open for Q’s from webinar participants and of course from the project staff who are on the call…