1. Economic Studies in D&I Research Debra P. Ritzwoller Institute for Health Research Kaiser Permanente Colorado Evaluation and Measurement CRISP D&I Training Workshop 2013 www.ucdenver.edu/implementation

2. Example: Effectiveness and cost of IVR influenza vaccination reminders for adults with asthma and chronic obstructive pulmonary disease Matthew F. Daley, PI Jo Ann Shoup MA, PhD Candidate, Co-I (responsible for the cost analysis) Institute for Health Research Kaiser Permanente Colorado www.ucdenver.edu/implementation2

3. Background –influenza poses a significant risk of morbidity and mortality for patients with asthma or COPD –IVR successfully used in other prevention initiatives Objectives –To evaluate effectiveness and costs of IVR for flu vaccine reminders in a chronic disease population Population setting and Context –Adult members of KPCO with asthma /COPD –Influenza vaccine outreach initiatives fall of 2012 Stakeholders involved –Population and Preventive Services –Chronic Disease managers –Primary Care departments –Institute for Health Research www.ucdenver.edu/implementation3 Project Overview

4. Selecting your Framework What were the outcomes of interest? –Effectiveness & costs What is the target population? –All members or those with asthma/COPD? Participation and completion rates –Contacted vs proportion vaccinated How were study participants identified and/or recruited. –Registries, membership files, other www.ucdenver.edu/implementation4

5. Choosing your Study Design Who delivered the intervention? –What training and supervision was required. –What components of the intervention were delivered by specific staff? How long did the intervention last? What did it cost? Whose Cost*? –Clinic or Health system perspective –Patient costs may be important as well How would the estimated intervention costs change in different replication scenarios (change in size or characteristics of the target population, different technology, etc.)? www.ucdenver.edu/implementation5

6. Choosing your Study Design Minimize the burden on subjects and research staff Need to differentiate research and development costs from recruitment and implementation costs Capture all resources and costs that would be required for future implementation or replication Identify resources and costs that would vary in different replication scenarios - sensitivity analyses www.ucdenver.edu/implementation6

7. Choosing your Measures Primary outcome –Incremental change in proportion vaccinated Total costs by intervention implementation arm –postcard vs IVR vs IVR +postcard Incremental cost per in incremental change in outcome of interest –Change in % vaccinated vs QALYs, or other Sensitivity analyses –Change in target population, personnel involved, etc www.ucdenver.edu/implementation7

8. Data Analysis and Interpretation Stakeholders approached the IHR regarding study design Stakeholder were involved with all steps of the intervention and the evaluation –Informed issues related to observed impact on staff –Informed potential sensitivity analyses Stakeholders and staff willingly participated with efforts to capture all resources and costs associated with alternative intervention strategies www.ucdenver.edu/implementation8

9. Findings 12,285 individuals were randomly assigned to the 3 arm study 30 day outcome: 29.5% of the postcard only group, 31.1% of the IVR only group, and 30.6% of the IVR plus postcard group had received influenza vaccine. Older age, outpatient visits, prior receipt of vaccine, and being an employee (or dependent) were positively associated with receiving an influenza vaccination. www.ucdenver.edu/implementation9

10. CategoryPostcard, $ IVR, $ Postcard plus IVR, $ Registered nurse42.81 Health educator31.12777.94809.06 Database administrator364.08 Program manager561.33270.29831.62 Informatics analyst, architect, developer 1636.47 Internist29.72 Voice talent75.00 Total$999.34$3153.50$3788.76 Personnel Costs

11. CategoryPostcard, $ IVR, $ Postcard plus IVR, $ Postcard printing and postage ($0.54 each)2211.30 IVR T1 line21.74 Server1875.00 Licensing2.83 Total$2211.30$1899.57$4110.87 Supplies, Hardware, and Software

12. CategoryPostcard, $ IVR, $ Postcard plus IVR, $ Total personnel costs 999.343153.503788.76 Total equipment and supplies costs 2211.301899.574110.87 Total costs 3210.645053.077899.63 Intervention number4095 Cost per member in group$0.78$1.23$1.93 Total Costs

13. CategoryPostcard, $ IVR, $ Postcard plus IVR, $ Total personnel costs 999.343153.50 3788.76 Total equipment and supplies costs 54000.002120.6556120.65 Total costs54999.345274.1559909.41 Intervention number100000 Cost per member in group$0.55$0.05$0.60 Going to Scale: What if PPS Sent Reminders to 100K Members?

14. Findings (2) IVRImplementation costs per member were $0.78, $1.23, and $1.93 per member for postcard only, IVR, and postcard plus IVR, respectively. IVR,Sensitivity analyses: extrapolate costs to annual target membership of 100,000: $0.55, $0.05, $0.60 per member for postcard only, IVR, and postcard plus IVR, respectively. www.ucdenver.edu/implementation14

15. Sharing Findings The stakeholder proposed sensitivity analyses became the most important issues used to inform implementation Capacity (non-capacity) issues informed potential cost efficiencies IVR only was implemented in 2013 to all target members (children, chronic disease, elderly, etc). www.ucdenver.edu/implementation15

16. Lessons Learned 1.Engage stakeholder (be available for engagement!) early and often. 2.A non-intervention arm (no reminder) would have improved effectiveness measures, but postcard intervention was considered “usual care.” 3.Identify all resources (and relative costs) needed for each arm prior to implementation. 4.Consider variable intensity strategies www.ucdenver.edu/implementation16

17. Neumann PJ, Weinstein MC. Legislating against use of cost-effectiveness information. N Engl J Med. 2010 Oct 14;363(16):1495-7. PMID:20942664 Ritzwoller DP, Sukhanova A, Gaglio B, Glasgow RE. Costing behavioral interventions: a practical guide to enhance translation. Ann Behav Med. 2009 Apr;37(2):218-27. Epub 2009 Mar 17.PMID: 19291342 Krist AH, Cifuentes M, Dodoo MS, Green LA. Measuring primary care expenses. J Am Board Fam Med. 2010 May-Jun;23(3):376-83. PMID: 20453184 Dodoo MS, Krist AH, Cifuentes M, Green LA. Start-up and incremental practice expenses for behavior change interventions in primary care. Am J Prev Med. 2008 Nov;35(5 Suppl):S423-30.PMID: 18929990 Prescription for Health, Guide for Collecting Expenditure Data in a Clinical Intervention in a Primary Care Practice http://www.prescriptionforhealth.org/results/P4H_exp_template/hndt3_dtacol gd.pdf http://www.prescriptionforhealth.org/results/P4H_exp_template/hndt3_dtacol gd.pdf Chronic Disease Self-Management calculator http://www.ncoa.org/chamodules/documents/LivingWellProgramGuidebook. pdf http://www.ncoa.org/chamodules/documents/LivingWellProgramGuidebook. pdf References

18. The Diabetes Prevention Program. Costs associated with primary prevention of type 2 diabetes mellitus in the Diabetes Prevention Program. Diabetes Care 2003 26:36 https://www.niddkrepository.org/niddkdocs/DPP/DPP_forms_data.doc‎ Ritzwoller DP, Sukhanova AS, Glasgow RE, Strycker LA, King DK, Gaglio B, Toobert DJ. Intervention costs and cost-effectiveness for a multiple-risk-factor diabetes self-management trial for Latinas: economic analysis of ¡Viva Bien! Transl Behav Med. 2011 Sep 1;1(3):427-435.PMID: 22081776 Mahadevia PJ, Fleisher LA, Frick KD, Eng J, Goodman SN, Powe NR. Lung cancer screening with helical computed tomography in older adult smokers: a decision and cost-effectiveness analysis. JAMA. 2003 Jan 15;289(3):313-22. Gold MR, Siegel JE, Russell LB, Weinstein MC. Cost-effectiveness in health and medicine. New York: Oxford University Press; 2003 Fishman PA, Cook AJ, Anderson M, Ralston JD, Catz SL, Carrell D, Carlson J, Green BB. Improving BP Control through Electronic Communications: An Economic Evaluation. Forthcoming in AJMC. References Con’t