1. Measuring QI Intervention Implementation: Helping the Blind Men See? QUITS Trial (Smoking Cessation) QUERI National Meeting Working Group December 12, 2008

2. QI Intervention (QII) Example QUITS (Quality Improvement Trial for Smoking cessation) – Evidence-based quality improvement approach to implementing smoking cessation guidelines – Actually Pre-QUERI then part of Substance Use Disorders (SUD) QUERI – Scott Sherman, MD & Becky Yano, PhD (Co-PIs)

3. QUITS Smoking Cessation Trial EBQI Education “toolkit” Local QI plan development Expert review/feedback Performance feedback Leadership support “local buy-in” “priority-setting” Evidence base: SC clinic referrals Tobacco quitlines PC-based intn’s

4. Context Matters: Design for It QUITS – Regional concentration in southwest (3 networks) – Matched on size/academic affiliation within network Included VAMCs and CBOCs Excluded COE’s VAMC – We chose participating sites and randomized within pairs within network

5. Multiple Data Sources: Measuring Implementation QUITS Examples Semi-structured interviews: leaders, providers, mgrs -- Participation, level of implementation Organizational site surveys: PC/MH leaders, SC mgrs Clinic structure, processes, change Administrative data Visits, Rxs, costs Patient surveys PHQ9, quits Provider surveys Knowledge, attitudes, behaviors Practice checklists (experts) QII components

6. QUITS Organizational Site Surveys Pre-Implementation StructuresIntervention practices Control Practices Pre-visit smoking assessment56%38% Smoking status reminders100%75% Local screening perf tracked33%13% PCPs prescribe cessation Rxs38%50% Provider feedback on SC perf38%62% Smokers referred to SC clinics33%38% Source: Yano, Rubenstein, Farmer, et al., HSR, 2008;43(5), Part 1:1637-1661.

7. QUITS Administrative Data EBQI Period INTERVENTION PRACTICES CONTROL PRACTICES SCPCAttend Rate SCPCAttend Rate Pre 1214 (4.2) 91,741 (3.0) 55.61,582 (3.5) 80,775 (2.6) 76.8 Base- line 1,137 (4.4) 85,274 (3.2) 56.01,979 (3.2) 100,830 (3.0) 77.0 Post 1,926 (3.9) 111,536 (3.4) 72.61,993 (3.5) 114,357 (2.9) 68.3 Source: Yano, Rubenstein, Farmer, et al., HSR, 2008;43(5), Part 1:1637-1661.

8. QUITS Patient Surveys Intervention Smokers Control Smokers MD talked about quitting68%66%  If yes, prescribed patches35%41%* MD referred to SC program37%45%*  Of referrals, recall attending  Of attendees, Rx’d NRT 35% 33% 50%*** 63%*** Smoking cessation (>30d)9% * Screened >36,000 primary care patients to identify, enroll >2,000 current smokers. Source: Yano, Rubenstein, Farmer, et al., HSR, 2008;43(5), Part 1:1637-1661.

9. QUITS Practice Checklist Smoking cessation expert review of: – Local QI plans (with feedback to practices) – Implementation activities Completed practice checklist of intervention components – Evidence-based vs. non-evidenced based – Changes from QI plan to implementation Source: Yano, Rubenstein, Farmer, et al., HSR, 2008;43(5), Part 1:1637-1661.

10. QUITS Post-Implementation Survey PlannedImplemented Provider education (eg CME)67%44% Patient education/activation56%44% Provider profiling/feedback56%11% PC-based org changes*33%22% Smoking cessation clinic chg22% Provider incentives11% * Brief counseling program, computerized referral in PC or counselor/nurse hired in SC clinic Source: Yano, Rubenstein, Farmer, et al., HSR, 2008;43(5), Part 1:1637-1661.

11. QUITS Post-Implementation Survey # planned EBPs # planned EBPs implem’d # new elements # new elements implem’d Site 14121 Site 28684 Site 35000 Site 47242 Site 53110 Site 64220 Site 74000 Source: Yano, Rubenstein, Farmer, et al., HSR, 2008;43(5), Part 1:1637-1661.