1. D&I Grant Funding: Tips for Success and Possible Futures
Russell E. Glasgow, PhD
University of Colorado School of Medicine
Department of Family Medicine
Adult and Child Consortium for Health Outcomes,
Research and Delivery Science (ACCORDS)

2. Thank You to: Dr. Paul Estabrooks and planning committee
Drs. Ross Brownson and Enola Proctor
My CU School of Medicine D&I program colleagues
Faculty and coordinators of prior national and CU SOM D&I training and mentoring programs

3. Overview Rationale and Need for D&I Research
Grant Funding Tips – primarily for NIH, but many generalizable to the IDeA CTR Pilot & Scholar program applications
General
D&I specific
The future and final thoughts
Resources
Q&A
Is this what you meant by fonts the same level, Russ? And you want the space between The future… and Resources?

4. Bench to Bookshelf

5. Research to Pipeline
Green, LW et al. Diffusion theory and knowledge dissemination. Ann Rev Public Health 2009;30:151-74

6. Implementation Science
Implementation science is the study of methods to promote the integration of research findings and evidence into healthcare policy and practice.
Dissemination science is the study of methods of distribution of information and intervention materials to a specific public health or clinical practice audience.

7. When are you ready to study D&I?
Adapted from Figure 11.1 Implementation and dissemination of prevention programs. (Brownson et al. Dissemination and Implementation Research in Health, 2012)

8. Effectiveness/Implementation Hybrids
Curran, et al. Medical Care, 2012

9. Impact, Communication, Credibility
Specific aims…aims…aims…it’s all about aims
and delivering on aims
Have aims read by multiple experts in other fields and non-experts
Why will this make a difference? Why is this grant compelling?
Demonstrate succinctly your and your team’s capacity

10. Video on “Importance of Specific Aims”
Dr. Ross Brownson, Washington University

11. Aims…good, bad and ugly Poor We will improve depression by:
using a well known cognitive behavior therapy program X
testing it in a controlled study
analyzing which patients benefit the most
disseminating the results

12. Aims…good, bad and ugly (cont’d)
Better
Refine existing EBP X based on Theory Y guided by prior findings, stakeholder engagement, and pilot testing of implementation strategy Z
Reduce depression by at least 25% among diverse community health center patients compared with a minimal intervention control in a cluster randomized pragmatic trial
Study the following implementation outcomes based on the RE-AIM model for evaluation and a mixed methods approach: reach, effectiveness; and implementation
Advance D&I science by testing the generalizability of this EBP in diverse, low resource settings and patients, and studying contextual factors predicted to be related to implementation outcomes

13. Question
What sections / components of grants do you think are weighted most heavily in review?

15. Methods – Key Issues (most heavily weighted)
Congruence across sections - can map from theory to intervention to measures to analyses to impact to dissemination
Select and justify a theory or conceptual model
Consider relevance of theory and fit to this situation
Consider pros and cons of adapting theory or combining theories

16. www.dissemination-implementation.org www.re-aim.org

18. Measures and Analyses Linked and clear rationale for hypotheses
Are reliable and valid in this population
Address potential downsides, unanticipated consequences, attrition
Are pragmatic – feasible, brief, sensitive to change, not perfect
Grid Enable Measures?
Glasgow RE and Riley T (2013). Pragmatic Measures: What They Are and Why We Need Them.
American Journal of Preventive Medicine, August;45(2):
Grid Enabled Measures

19. Show your work & specify WHY it’s important
(Personal style) Do not hide tough decisions
(Briefly) discuss choices – show you are aware of alternatives
State succinctly how this is innovative and why this team in this setting at this point in time (write reviewers evaluation for them) 
How does this project:
advance science, and
impact public health (or some other societal or funding agency) goal?

20. Talk with project officers and “fit” your project to agency, key need or priority
Project Officer(s) plural – different agencies and possible funders

21. Team Science
Days of one investigator – unless descendent of Thomas Jefferson – are over
Demonstrate you have all the relevant expertise, e.g., economics, biostatistics, maybe demography, qualitative expertise, etc. on your team
Show how you have or can work together…if/how you are using team science for “real transdisciplinary collaboration”

22. Somewhat Specific to D&I Grants
Differentiate between intervention and implementation strategy
Example: intervention may be cognitive behavior therapy. IT should be evidence- based (separate and detailed discussion)
Implementation strategy may be facilitator training; or delivery via internet
Implementation strategy should have some evidence but can be innovative, if justified and “fits”
Sa?

23. Context is King
Demonstrate you are aware of both the general and relevant – usually multi-level contextual factors (social ecologic model)
Discuss how your program or policy or study addresses or fits well with local context
Ask yourself, is this program, policy or finding likely to be generalizable or fit different contexts
Feldstein A and Glasgow RE. Pragmatic Robust Implementation and Sustainability Framework. Joint Commission Journal on Quality and Safety, 34(4):

24. Context Reporting Recommendations*
Important to report conditions under which program was delivered.
To what extent is the program generalizable:
To similar settings?
To different settings?
Goal – to what intervention do you compare it (real world alternative)?
* Standards for Reporting Implementation Studies(StaRI) Statement. Hilary Pinnock, Melanie Barwick,Christopher R Carpenter, et al. BMJ 2017;356:i6795

25. Ultimate Use Question:
“What program/policy components are most effective for producing what outcomes for which populations/recipients when implemented by what type of persons under what conditions, with how many resources and how/why do these results come about?”

26. Stakeholder Engagement
In D&I, this is multi-level…at least 3 levels
System or even national organization
Local system or organization, e.g., school, YMCA, voluntary health agency, extension service
Supervisor or local decision-makers
Delivery staff or agents
Recipients, potential beneficiaries or citizens
Be SERIOUS about engagement – not lip service
Usually requires ongoing relationship, not ‘helicopter research’
Advisory boards, PBRNs, etc.
Should have already engaged stakeholders

27. Dissemination Plan
Do NOT say will ‘do publications and scientific presentations’ (at least as primary strategy)
Think creatively – how will you reach key target audience?
Go where they are – just because you build it, they will NOT come
Provide training, facilitation, resources and ongoing contact and FAQs – not one-time brief presentation

28. The Future???

29. Need for Rigor, Relevance and Transparency
New NIH guidelines on rigor and transparency – to address failure to replicate and data-sharing issues
Personal ‘wish’ and need: Combining issue of transparency, context reporting, and generalizability…
“Expanded-CONSORT”
References and link

30. Expanded CONSORT Figure
Total Number Potential Settings
Settings Eligible (n and %)
Settings and Agents Who Participate(n and %)
Settings and Agents Who Decline(n and %)
Other
(n and %)
Excluded by Investigator
(n, %, and reasons)
Total Potential Participants (n)
Individuals Eligible (n and %)
Individuals Enroll (n and %)
Individuals Decline (n, %, and reasons)
Not Contacted/Other (n and %)
Excluded by Investigator (n, %, and reasons)
ADOPTION
REACH
RE-AIM Issue
Extent Tx Delivered by Different Agents as in Protocol
Adaptations and Reasons
Component A = XX%
Component B = YY%
etc.
IMPLEMENTATION
Complete Tx (n and %) Amount of Change (by Condition)
Drop Out of Tx (n, %, and reasons) Amount of Change (by Condition)
EFFECTIVENESS
Present at Follow-up (n and %) and Amount of Change or Relapse (By Condition)
Lost to Follow-up (n, %, and reasons) Amount of Change or Relapse (By Condition)
Setting in which Program is Continued and/or Modified After Research is Over (n, %, and reasons)
Settings in which Program Adapted (n, %, and reasons)
MAINTENANCE
Settings in which Program Discontinued (n, %, and reasons)

31. Feasible and Pragmatic (Russ’s Dream)
How fits with local settings and resources, priorities
Estimated costs – time at minimum; replication costs; cost-effectiveness
How pragmatic is your project along different PRECIS dimensions for study design – Pragmatic Explanatory Continuum Indicator Summary (PRECIS-2)

32. https://www.precis-2.org/ We’ll have to fix these 2 slides.
RG- OK- do not look bad to me though. 
The font is still a bit hard to read…if you have a clearer image, we can use that.

33. RE-AIM Precision (Personalized) Medicine Questions
Determine
What percent and types of patients are Reached?
For whom is the intervention Effective in improving what outcomes, with what unanticipated consequences?
In what percent and types of settings and is this approached Adopted?
How consistently are different parts of it Implemented at what cost to different parties?
And how well are the intervention components and their effects Maintained?
Gaglio B, Glasgow RE. Evaluation approaches…In:Brownson R, Colditz G, Procter E, (Eds). Dissemination and implementation research in health: Translating science to practice. New York: Oxford University Press; 2012, pp NEW EDITION DUE OUT DECEMBER

34. Personal Advice to Mentees (most important)
Develop a thick skin
Persist – get back up and try again
It always takes longer than you think – budgets, especially contracts and multi-site studies, colleagues delayed in getting you their sections
Remember, it is not a fair world… “But if you try sometimes, you get what you need” ~Mick Jagger

36. First, Last and Always…
Check with your potential program officer(s)

37. Questions/Comments???
I’m all ears…

38. Key Resources- see handouts
U Col. SOM D&I website
10 Tips to Getting Funded. Proctor et al- see above
Ross Brownson –Funding in D&I Research-see above
Examples of successful D&I grants
UNC:
NCI: FundedContentAnalysis.pdf

39. Key Differences between Traditional Randomized Control Trials (RCTs) and Pragmatic Controlled Trials (PCTs)
A traditional RCT tests a hypothesis under ideal conditions
A PCT compares treatments under everyday clinical conditions
GOALS
To determine causes and effects of treatment
To improve practice and inform clinical & policy decisions
DESIGN
Tests the intervention against placebo using rigid study protocols & minimal variation
*Tests two or more real-world treatments using flexible protocols & local customization*
PARTICIPANTS
Highly defined & carefully selected
More representative because eligibility criteria are less strict
MEASURES
Require data collection outside routine clinical care
Brief and designed so data can be easily collected in clinical settings
RESULTS
Rarely relevant to everyday practice
Useful in everyday practice, especially clinical decision making
Do you want this slide to be in a table similar to #20 in the OSU set?

40. Scientific and Societal Trends
Behavioral or social science for its own sake decreasingly likely to be funded
Much larger collaborations; big data
Possibly more international collaborations
Even more transdisciplinary – and partner with at least one discipline and data source that ends in ‘omics’