Study Populations Effectiveness of Public-Private Collaboration in the Delivery of Influenza Vaccine Allison Kempe, MD, MPH Pragmatic TRIALS WWW.Ucdenver.edu/implementation  

Background 2012 IOM report highlighted importance of public-private collaboration in delivery of preventive care Influenza vaccine delivery an important area for collaborative delivery Universally recommended Must be delivered in narrow time frame No previous trials assessing effectiveness of public-private collaborative delivery of influenza vaccine

Objectives To compare effectiveness of influenza vaccine delivery for children within practices randomized to: Intervention: active collaboration between private practices and local public health departments (PHDs) in influenza vaccine delivery 2. Control: usual influenza vaccine delivery

Study Design Cluster-randomized pragmatic comparative effectiveness trial with randomization at level of practice Practices with common PHD (3 counties) Mixed methods with qualitative methods used in: Refinement of intervention Assessment of intervention

Study Design Baseline (2009/2010 season): Recruitment and randomization of practices Collaborative design of intervention by intervention practices and PHD Year 1 (2010/2011 season): Initial implementation of collaborations Qualitative assessment

Study Design Year 2 (2011-2012 season): Refinement of intervention Re-implementation

Data Sources Influenza vaccination rates: Influenza rates assessed: Colorado Immunization Information System Administrative claims data from practices Influenza rates assessed: (Pre-season) (Post-season) Baseline: August 1 2009 - July 31, 2010 I Year 1: August 1, 2010 - July 31, 2011 I Year 2: August 1, 2011 - July 31, 2012

Study Outcomes Major outcome = ∆ in influenza vaccination rates (Yr 2 versus Baseline) in Intervention vs Control practices Secondary outcomes: Comparative changes in rates by specialty Comparative changes in rates for age subgroups including 6 mos-5 years, 6-12 years, 13-18 years Comparative changes in rates for children with a high-risk condition

Why this Study? It was difficult! (Pragmatic = Messy) It was a first attempt to study a new collaborative model It was a topic with important pragmatic programmatic implications It illustrates multiple choices along spectrum from explanatory to pragmatic

Choices about Setting Needed group of practices with a common PHD with which to collaborate Budgetary limitations re number of clusters

Choices about Setting Wanted to minimize variability in health care delivery related to: Urban/rural location Type of practice organization (ie HMO, multi-specialty practice, MCO vs typical primary care single-specialty practice) Chose 1 health department covering 3 urban counties with many private practices

Choices about Setting Public Health Department Practice Practice

How did setting choices limit generalizability? Significantly! However…. In this case, the example chosen was the most relevant to policy question Rural areas already collaborate a lot therefore question less relevant HMOs, large multispecialty groups, MCOs often have centralized flu delivery methods that would mimic the collaborative method proposed

Setting--PRECIS Equally pragmatic/explanatory

Eligibility: Who is selected to participate? Needed roughly balanced study arms given small # clusters with respect to: Practice specialty (Peds or FM) Size of practices (# providers) Practice populations (% children with high risk conditions, % publically insured) But wanted to avoid selecting atypical practices

Recruitment: How were practices recruited? Listed all private practices in the counties and their characteristics Chose 6 FM and 6 Peds that matched fairly well w/in each specialty Approached all to be involved in randomized trial—took the first 4 that agreed

Methods: Randomization of Practices 8 Practices with common PHD and similar characteristics: # Providers % Children with high risk conditions % Publically insured 4 Pediatric 2 Pediatric Intervention 2 Pediatric Control 4 Family Medicine 2 Family Medicine Intervention 2 Family Medicine Control

Choices about Eligibility How did choices affect generalizability? Selection bias potentially introduced by practices agreeing to participation and by pre-selected balancing criteria--somewhat mitigated by randomization Single large PHD not entirely generalizable Not generalizable to other types of providers (rural, HMO, MCO, etc…)

Choices about Eligibility--PRECIS Equally pragmatic/explanatory

Organization: What expertise and resources are needed? Required collaboration between multiple primary care practices and a county PHD Resources required for development and implementation of intervention Multiple meetings to design feasible collaborations Intervention carried out by practices and PHD but study did reminders for joint community or practice-based clinics Study team assisted with collaboration

Organization: What expertise and resources are needed? How did these requirements influence the generalizability of your findings? LOTS! Sustainability would require time from both PCPs and PHD for planning and coordination Reminders would need to come from practices or PHD with costs attached

Lessons Learned Don’t try to do too cluster-randomized pragmatic trial with insufficient funding Don’t embark on pragmatic trial without adequate buy-in from participants Build sustainability into the plan from the beginning

So What Happened? Results!

Study Populations at Baseline Control Practices (2 Peds, 2 FM) Intervention Practices Mean Age in yrs (95% CI) 9.1 (6.7, 11.4) 8.5 (6.3, 10.6) Mean % High Risk 9.7 (6.6, 12.8) 9.4 (3.2, 15.5) Median # Providers (min,max) 4.25 (3, 6) 3.25 (3, 18) Median % VFC 15 (0, 33) 11 (4, 35) Ally, these data are at Baseline

Results: Intervention Strategies Study Years Collaborative Efforts in Intervention Practices Intervention Year 1 2010/2011 Mean 4 (3-5) per practice Mean 3 Community Clinics/practice Mean 1.5 PHD-assisted clinics/practice Intervention Year 2 2011/2012 Mean 3.25 (2-4) per practice Mean 2 Community Clinics/practice Mean 1.25 PHD-assisted clinics/practice Mean # times nurses assisting at practice would make the most sense

Results: Major Outcome   I total (N=26,123) C total (N=15,372) % Vaccinated Baseline 37.7 43.4 % Absolute Change I-Yr 2 vs. Baseline 9.2 3.2 Risk Ratio (95% CI) 1.15 (1.09, 1.21) Formatting needed==get rid of all lines except top blue line and align text at bottom

Results: Outcomes by Specialty   Pediatric Practices Family Medicine I total (N=25,108) C total (N=14,511) (N=1,015 ) (N=861) % Vaccinated at Baseline 38.5 44.6 17.7 23.3 % Change I-Yr 2 vs. Baseline* 9.5 3.5 8.5 0.1 Risk Ratio (95% CI) 1.14 (1.08, 1.20) 1.53 (1.17, 1.99) *p-value <.01 Need to get rid of black and dotted lines please

Results: Outcomes by Age   6 mos - 5 yrs 6 yrs -12 yrs 13 yrs – 18 yrs I total (N=8,969) C total (N=4,563) (N=10,684) (N=6,569) (N=6,470) (N=4,240) % Vaccinated at Baseline 50.0 52.9 32.1 43.7 23.1 27.5 % Change I-Yr 2 vs. Baseline 7.3 4.1 11.9 2.8 9.8 5.4 Risk Ratio (95% CI) 0.98 (0.90, 1.07) 1.39 (1.27, 1.51) 1.21 (1.06, 1.37) Get rid of black lines and dotted lines

Results: Outcomes by Age   6 mos - 5 yrs 6 yrs -12 yrs 13 yrs - 18 yrs I total (N=8,969) C total (N=4,563) (N=10,684) (N=6,569) (N=6,470) (N=4,240) % Vaccinated Baseline 50.0 52.9 32.1 43.7 23.1 27.5 % Change I-Yr 2 vs. Baseline 7.3 4.1 11.9 2.8 9.8 5.4 Risk Ratio (95% CI) 0.98 (0.90, 1.07) 1.39 (1.27, 1.51) 1.21 (1.06, 1.37) Get rid of black lines and dotted lines

Results: Outcomes by High-Risk   High Risk Condition I total (N=3,321) 49.1 C total (N=1,492) 50.3 % Vaccinated at Baseline % Change I-Yr 2 vs. Baseline 7.6 5.2 Risk Ratio (95% CI) 1.02 (0.87, 1.19) Get rid of dotted lines and align last line

Qualitative Results: Barriers to Collaboration Vaccine supply issues Difficult to predict when will receive (often late) therefore hard to plan collaborations Don’t want to use up demand and be left with unused supply and financial loss Coordination across immunization processes Forms used by practices vs. forms used by PHD Concerns about uploading to CIIS Don’t understand third point

Limitations Relatively small trial—4 practices per arm Generalizability limited: Practices and PHD in one urban CO county Conducted in state without universal purchase for influenza vaccine Cannot quantify contribution of recall activities to collaborative intervention

Conclusions Collaborative influenza vaccine delivery involving practices and PHDs successful in increasing rates among school-aged and adolescent children No significant effect among children <5 years and those with a high risk condition Effect size larger in family medicine practices where baseline vaccination rates lower

Conclusions Effect size not very large and limited by: Practices’ reluctance to participate early in the season when they were concerned about using up supply Lower patient participation later in season when practices were more willing to participate Difficulty sharing vaccine supplies between practices and PHD

Implications Collaborative influenza delivery between practices and PHDs should include methods of sharing vaccine without potential financial loss to either partner Collaborations should target those children not seen frequently in the office, such as older children in community settings Collaboration could be easier in states with universal purchase for influenza vaccine

University of Colorado Denver Study Team University of Colorado Denver Principal Investigator – Allison Kempe, MD, MPH Sean O’Leary, MD, MPH Karen Albright, PhD Miriam Dickinson, PhD Deidre Kile, MS Doron Shmueli, MS Juliana Barnard, MA Steven Lockhart, BA Eva Dibert, MHA Christine Babbel, MPH Jennifer Barrow, MPH CDC Collaborator Maureen Kolasa, RN, MPH 36

Study supported by Centers for Disease Control and Prevention Funding “Strategies to Vaccinate all Children for Influenza in a Practice Setting” Study supported by Centers for Disease Control and Prevention (UO1IP000320) The content is solely the responsibility of the authors and does not necessarily represent the official views of the Centers for Disease Control and Prevention Grant name and number needed

Publication Kempe A, Albright K, O'Leary S, Kolasa M, Barnard J, Kile D, Lockhart S, Dickinson LM, Shmueli D, Babbel C, Barrow J. Effectiveness of primary care-public health collaborations in the delivery of influenza vaccine: a cluster-randomized pragmatic trial. Prev Med. 2014 Aug 22. pii: S0091-7435(14)00310-7. Grant name and number needed

Methods: Statistical Analysis Generalized Estimating Equations (GEE) used to model risk ratio for change in vaccination rates between I-Yr 2 and Baseline Model accounted for repeated measures within subject across study years Independent variables included: Intervention group Practice specialty Year of study Year x intervention interaction term used to assess for differential change between I and C practices Are point 2 and subpoint 3 repetitive? How did we account for clustering?