RE-AIM Plus To Evaluate Effective Dissemination of AHRQ CER Products Michele Heisler, MD, MPA September, 2011

Web-Based Tool to Facilitate Diabetes Medications Decision-making (AHRQ R18, Heisler) iDecide Assessing the use of a web-based, interactive, tailored decision tool in improving diabetes health outcomes Decido Evaluando el uso de un herramienta de decisión en mejorar los resultados de la salud en pacientes con diabetes

Definition of Tailoring 1. assess an individual’s characteristics relevant to the behavior 2. Use assessment data to generate messages relevant to that individual’s specific needs 3. Deliver these messages in a clear, vivid—and interactive--format

iDecide Project description –Community health worker (CHW)-delivered intervention to improve knowledge of diabetes medications among low-literacy, low-income adults (Latinos and African Americans) with poor glycemic control/reported medication problems 2 arms (n=210) –Control group: reviews the static, paper AHRQ patient guide with a CHW –Intervention group: reviews the tailored, web- based program with a CHW

Goals of the study Aim 1: (Work together to) Build an interactive, tailored diabetes medication decision aid to help people: –Assess their treatment goals –Identify personal preferences and concerns –Understand options for improving their diabetes care (when appropriate)

Goals of the study Aim 2: Compare the tailored tool with the print AHRQ CER guides on participant: –Knowledge of medications –Satisfaction with information –Decisional conflict

Goals of the study Aim 3: Look at how the tool affects: –Participant changes to medication –Self-reported medication adherence –Participant beliefs about medication –A1c levels

Use open-ended techniques, e.g., interviews, observation Goal is understanding, rather than measurement Get into the “black box” Uncover the whys and hows behind quantitative measures Identify and understand implementation processes, how influenced by context and how affect implementation success

Development of RE-AIM Plus Systematically review each RE-AIM dimension and create open-ended questions Add information sources and data analyses to address DimensionQuantitative Measures Qualitative Inquiry Reach Effectiveness Adoption Implementation Maintenance

QUANTITATIVE MEASURES QUALITATIVE INQUIRY How many and what proportion of target population is being contacted and participating? Measures # eligibles contacted/# eligibles # eligibles participating/# eligibles Use measures to track patient contact and participation Look at variation across sites Data Source Clinic data, recruitment, and activity data sets What explains variation across sites? What are factors and processes underlying barriers to contacting patients and patient participation, and how do we address them? Data Sources: Semi-structured interviews with Key Informants Site visits and observations s between stakeholders and research staff RE-AIM Plus: Reach

QUANTITATIVE MEASURES QUALITATIVE INQUIRY Evaluate effects of intervention on outcomes Measure: Changes between baseline and follow-up in survey measures and A1cs Data Source Surveys and point-of-service A1cs Explain summative outcomes What are the conditions and mechanisms that lead to effectiveness? Why did the intervention work? Why not? What explains variation across sites? What are factors and processes underlying barriers to implementation, and how did we address them? Data Sources: All RE-AIM Plus: Effectiveness

QUANTITATIVE MEASURES QUALITATIVE INQUIRY I mplementation: Did CHWs use the decision aid with participants as anticipated? Measures % of enrolled patients with decision aid session % of enrolled patients who received follow-up call % of enrolled patients who discussed questions/concerns with MDs Data Source Activity reports/Surveys Pre-implementation: What are site-specific issues that might influence implementation? What modifications do we need to make and how do we make them? Implementation: What were the problems with key implementation processes? What are the barriers to conducting the sessions, follow-up calls, MD discussions Data Sources: all RE-AIM Plus: Implementation

QUANTITATIVE MEASURES QUALITATIVE INQUIRY Is the program maintained after the study period? Can the program be sustained after the study period? What is the feasibility of retaining program-specific staff to continue the intervention? What is the perception of program value among stakeholders? RE-AIM Plus: Maintenance

Important Methods Considerations Methods: Effective procedures and instruments to systematically collect data Multiple data sources Data management system to facilitate data retrieval –Including real time

Concluding Thoughts RE-AIM Plus: Useful in generating context-specific and generalizable information on implementation Allows understanding of implementation mechanisms behind both process and outcome measures May improve ways to adapt implementation in real time and increase likelihood of success when interventions are disseminated