1. Study Populations
Effectiveness of Public-Private Collaboration in the Delivery of Influenza Vaccine
Allison Kempe, MD, MPH
Pragmatic TRIALS

2. Background
2012 IOM report highlighted importance of public-private collaboration in delivery of preventive care
Influenza vaccine delivery an important area for collaborative delivery
Universally recommended
Must be delivered in narrow time frame
No previous trials assessing effectiveness of public-private collaborative delivery of influenza vaccine

3. Objectives
To compare effectiveness of influenza vaccine delivery for children within practices randomized to:
Intervention: active collaboration between private practices and local public health departments (PHDs) in influenza vaccine delivery
2. Control: usual influenza vaccine delivery

4. Study Design
Cluster-randomized pragmatic comparative effectiveness trial with randomization at level of practice
Practices with common PHD (3 counties)
Mixed methods with qualitative methods used in:
Refinement of intervention
Assessment of intervention

5. Study Design Baseline (2009/2010 season):
Recruitment and randomization of practices
Collaborative design of intervention by intervention practices and PHD
Year 1 (2010/2011 season):
Initial implementation of collaborations
Qualitative assessment

6. Study Design Year 2 (2011-2012 season): Refinement of intervention
Re-implementation

7. Data Sources Influenza vaccination rates: Influenza rates assessed:
Colorado Immunization Information System
Administrative claims data from practices
Influenza rates assessed:
(Pre-season) (Post-season)
Baseline: August July 31, 2010
I Year 1: August 1, July 31, 2011
I Year 2: August 1, July 31, 2012

8. Study Outcomes
Major outcome = ∆ in influenza vaccination rates (Yr 2 versus Baseline) in Intervention vs Control practices
Secondary outcomes:
Comparative changes in rates by specialty
Comparative changes in rates for age subgroups including 6 mos-5 years, 6-12 years, years
Comparative changes in rates for children with a high-risk condition

9. Why this Study? It was difficult! (Pragmatic = Messy)
It was a first attempt to study a new collaborative model
It was a topic with important pragmatic programmatic implications
It illustrates multiple choices along spectrum from explanatory to pragmatic

10. Choices about Setting
Needed group of practices with a common PHD with which to collaborate
Budgetary limitations re number of clusters

11. Choices about Setting
Wanted to minimize variability in health care delivery related to:
Urban/rural location
Type of practice organization (ie HMO, multi-specialty practice, MCO vs typical primary care single-specialty practice)
Chose 1 health department covering 3 urban counties with many private practices

12. Choices about Setting Public Health Department Practice Practice

13. How did setting choices limit generalizability?
Significantly! However….
In this case, the example chosen was the most relevant to policy question
Rural areas already collaborate a lot therefore question less relevant
HMOs, large multispecialty groups, MCOs often have centralized flu delivery methods that would mimic the collaborative method proposed

14. Setting--PRECIS
Equally pragmatic/explanatory

15. Eligibility: Who is selected to participate?
Needed roughly balanced study arms given small # clusters with respect to:
Practice specialty (Peds or FM)
Size of practices (# providers)
Practice populations (% children with high risk conditions, % publically insured)
But wanted to avoid selecting atypical practices

16. Recruitment: How were practices recruited?
Listed all private practices in the counties and their characteristics
Chose 6 FM and 6 Peds that matched fairly well w/in each specialty
Approached all to be involved in randomized trial—took the first 4 that agreed

17. Methods: Randomization of Practices
8 Practices with common PHD and similar characteristics:
# Providers
% Children with high risk conditions
% Publically insured
4 Pediatric
2 Pediatric Intervention
2 Pediatric Control
4 Family Medicine
2 Family Medicine Intervention
2 Family Medicine
Control

18. Choices about Eligibility
How did choices affect generalizability?
Selection bias potentially introduced by practices agreeing to participation and by pre-selected balancing criteria--somewhat mitigated by randomization
Single large PHD not entirely generalizable
Not generalizable to other types of providers (rural, HMO, MCO, etc…)

19. Choices about Eligibility--PRECIS
Equally pragmatic/explanatory

20. Organization: What expertise and resources are needed?
Required collaboration between multiple primary care practices and a county PHD
Resources required for development and implementation of intervention
Multiple meetings to design feasible collaborations
Intervention carried out by practices and PHD but study did reminders for joint community or practice-based clinics
Study team assisted with collaboration

21. Organization: What expertise and resources are needed?
How did these requirements influence the generalizability of your findings?
LOTS!
Sustainability would require time from both PCPs and PHD for planning and coordination
Reminders would need to come from practices or PHD with costs attached

22. Lessons Learned
Don’t try to do too cluster-randomized pragmatic trial with insufficient funding
Don’t embark on pragmatic trial without adequate buy-in from participants
Build sustainability into the plan from the beginning

23. So What Happened? Results!

24. Study Populations at Baseline
Control Practices
(2 Peds, 2 FM)
Intervention Practices
Mean Age in yrs
(95% CI)
9.1 (6.7, 11.4)
8.5 (6.3, 10.6)
Mean % High Risk
9.7 (6.6, 12.8)
9.4 (3.2, 15.5)
Median # Providers
(min,max)
4.25 (3, 6)
3.25 (3, 18)
Median % VFC
15 (0, 33)
11 (4, 35)
Ally, these data are at Baseline

25. Results: Intervention Strategies
Study Years
Collaborative Efforts in Intervention Practices
Intervention Year /2011
Mean 4 (3-5) per practice
Mean 3 Community Clinics/practice
Mean 1.5 PHD-assisted clinics/practice
Intervention Year /2012
Mean 3.25 (2-4) per practice
Mean 2 Community Clinics/practice
Mean 1.25 PHD-assisted clinics/practice
Mean # times nurses assisting at practice would make the most sense

26. Results: Major Outcome
I total
(N=26,123)
C total
(N=15,372)
% Vaccinated Baseline
37.7
43.4
% Absolute Change
I-Yr 2 vs. Baseline
9.2
3.2
Risk Ratio (95% CI)
1.15 (1.09, 1.21)
Formatting needed==get rid of all lines except top blue line and align text at bottom

27. Results: Outcomes by Specialty
Pediatric Practices
Family Medicine
I total
(N=25,108)
C total
(N=14,511)
(N=1,015 )
(N=861)
% Vaccinated at Baseline
38.5
44.6
17.7
23.3
% Change
I-Yr 2 vs. Baseline*
9.5
3.5
8.5
0.1
Risk Ratio (95% CI)
1.14 (1.08, 1.20)
1.53 (1.17, 1.99)
*p-value <.01
Need to get rid of black and dotted lines please

28. Results: Outcomes by Age
6 mos - 5 yrs
6 yrs -12 yrs
13 yrs – 18 yrs
I total
(N=8,969)
C total
(N=4,563)
(N=10,684)
(N=6,569)
(N=6,470)
(N=4,240)
% Vaccinated at Baseline
50.0
52.9
32.1
43.7
23.1
27.5
% Change I-Yr 2 vs. Baseline
7.3
4.1
11.9
2.8
9.8
5.4
Risk Ratio (95% CI)
0.98
(0.90, 1.07)
1.39
(1.27, 1.51)
1.21
(1.06, 1.37)
Get rid of black lines and dotted lines

29. Results: Outcomes by Age
6 mos - 5 yrs
6 yrs -12 yrs
13 yrs - 18 yrs
I total
(N=8,969)
C total
(N=4,563)
(N=10,684)
(N=6,569)
(N=6,470)
(N=4,240)
% Vaccinated Baseline
50.0
52.9
32.1
43.7
23.1
27.5
% Change
I-Yr 2 vs. Baseline
7.3
4.1
11.9
2.8
9.8
5.4
Risk Ratio (95% CI)
0.98
(0.90, 1.07)
1.39
(1.27, 1.51)
1.21
(1.06, 1.37)
Get rid of black lines and dotted lines

30. Results: Outcomes by High-Risk
High Risk Condition
I total
(N=3,321)
49.1
C total
(N=1,492)
50.3
% Vaccinated at Baseline
% Change I-Yr 2 vs.
Baseline
7.6
5.2
Risk Ratio (95% CI)
1.02 (0.87, 1.19)
Get rid of dotted lines and align last line

31. Qualitative Results: Barriers to Collaboration
Vaccine supply issues
Difficult to predict when will receive (often late) therefore hard to plan collaborations
Don’t want to use up demand and be left with unused supply and financial loss
Coordination across immunization processes
Forms used by practices vs. forms used by PHD
Concerns about uploading to CIIS
Don’t understand third point

32. Limitations Relatively small trial—4 practices per arm
Generalizability limited:
Practices and PHD in one urban CO county
Conducted in state without universal purchase for influenza vaccine
Cannot quantify contribution of recall activities to collaborative intervention

33. Conclusions
Collaborative influenza vaccine delivery involving practices and PHDs successful in increasing rates among school-aged and adolescent children
No significant effect among children <5 years and those with a high risk condition
Effect size larger in family medicine practices where baseline vaccination rates lower

34. Conclusions Effect size not very large and limited by:
Practices’ reluctance to participate early in the season when they were concerned about using up supply
Lower patient participation later in season when practices were more willing to participate
Difficulty sharing vaccine supplies between practices and PHD

35. Implications
Collaborative influenza delivery between practices and PHDs should include methods of sharing vaccine without potential financial loss to either partner
Collaborations should target those children not seen frequently in the office, such as older children in community settings
Collaboration could be easier in states with universal purchase for influenza vaccine

36. University of Colorado Denver
Study Team
University of Colorado Denver
Principal Investigator – Allison Kempe, MD, MPH
Sean O’Leary, MD, MPH
Karen Albright, PhD
Miriam Dickinson, PhD
Deidre Kile, MS
Doron Shmueli, MS
Juliana Barnard, MA
Steven Lockhart, BA
Eva Dibert, MHA
Christine Babbel, MPH
Jennifer Barrow, MPH
CDC Collaborator
Maureen Kolasa, RN, MPH 36

37. Study supported by Centers for Disease Control and Prevention
Funding
“Strategies to Vaccinate all Children for Influenza in a Practice Setting”
Study supported by Centers for Disease Control and Prevention
(UO1IP000320)
The content is solely the responsibility of the authors and does not necessarily represent the official views of the Centers for Disease Control and Prevention
Grant name and number needed

38. Publication
Kempe A, Albright K, O'Leary S, Kolasa M, Barnard J, Kile D, Lockhart S, Dickinson LM, Shmueli D, Babbel C, Barrow J. Effectiveness of primary care-public health collaborations in the delivery of influenza vaccine: a cluster-randomized pragmatic trial. Prev Med Aug 22. pii: S (14)
Grant name and number needed

39. Methods: Statistical Analysis
Generalized Estimating Equations (GEE) used to model risk ratio for change in vaccination rates between I-Yr 2 and Baseline
Model accounted for repeated measures within subject across study years
Independent variables included:
Intervention group
Practice specialty
Year of study
Year x intervention interaction term used to assess for differential change between I and C practices
Are point 2 and subpoint 3 repetitive? How did we account for clustering?