Design Strategies CRISP D&I Training Workshop 2013 Collaborative Population-based vs Practice-based Reminder/Recall to Increase Immunization Rates in Young Children A Pragmatic Comparative Effectiveness Trial Allison Kempe, MD, MPH Director Children’s Outcomes Research (COR) and Center for Research in Implementation Science and Prevention (CRISP) www.ucdenver.edu/implementation

Immunizations Second Only to Clean Water! Disease Pre-Vaccine Era Estimated Annual Morbidity* Most Recent Estimates‡of U.S. Cases Percent decrease Diphtheria 21,053 0† 100% H. influenzae (invasive, <5 years of age) 20,000 243†§ 99% Hepatitis A 117,333 11,049‡ 91% Hepatitis B (acute) 66,232 11,269‡ 83% Measles 530,217 61† >99% Mumps 162,344 982† Pertussis 200,752 13,506† 93% Pneumococcal disease (invasive, <5 years of age) 16,069 4,167‡ 74% Polio (paralytic) 16,316 Rubella 47,745 4† Congenital Rubella Syndrome 152 1† Smallpox 29,005 Tetanus 580 14† 98% Varicella 4,085,120 449,363‡ 89% *CDC. JAMA, November 14, 2007; 298(18):2155–63 †CDC. MMWR, January 8, 2010; 58(51,52):1458–68 ‡2008 estimates, S. pneumoniae estimates from Active Bacterial Core Surveillance §25 type b and 218 unknown

Background – The Problem Goal: Increase the proportion of children aged 19 to 35 months who receive all routinely recommended vaccines Target = 80% Estimated vaccination coverage for vaccination series among children 19-35 months – National Immunization Survey, United States 2010 State/Area Vaccine series modified United States 19-35 month olds 73% (27% not UTD) Colorado 19-35 month olds 68% (32% not UTD) Can we move increase to left and increase font size of target 80%

WHY?! Barriers to optimal immunization delivery Financial Access to care issues Lack of awareness Infrastructure and regulatory issues Complexity and expansion of vaccination schedule # of vaccines more than doubled in past 25 years By18 months of age U.S. children recommended to receive vaccines against 14 different diseases, requiring up to 26 different vaccine doses Vaccine hesitancy Misinformation Safety concerns

What Has Been Shown to Work? Based on strong evidence of effectiveness Community Preventive Services Task Force recommends reminder/recall (R/R) Notification to inform patients they are due (reminder) or overdue (recall) for immunizations Can be delivered by mail, phone, autodialer, text, e-mail In the past usually conducted at the level of the practice using practice-based data systems 47 RCTs included in recent Cochrane review with most showing effectiveness 5-15%....HOWEVER…

What Has Been Shown to Work? 35/47 trials were done by research teams coming into practices and none measured sustainability! Our study team spent years studying and trying to improve practice-based R/R RCTs to measure effectiveness in different settings and populations Qualitative work to assess barriers to conducting R/R and sustainability of R/R

What Has Been Shown to Work? What did we learn? R/R difficult to implement even within highly motivated practices using an Immunization Information System (IIS)—lots of barriers to implementation and sustainability identified (AMONG OTHERS…) Competing demands of primary care Inadequate systems and inadequate training Cost considerations National data suggest that 6% of physicians are conducting practice-based R/R

Learning from Implementation Failure Time for a new idea that accomplishes what practices want for their patients but facilitates implementation and sustainability!

Learning from Providers and Patients What did providers want? Would prefer to do R/R for their patients but majority think its not feasible Vast majority (85%) were alright with Public Health Department (PHD) taking a major role What did parents want? Roughly half preferred to receive R/R from their child’s provider and half did not care if R/R came from provider or PHD or preferred PHD

Our Implementation Idea: Centralized and Collaborative Make use of CIIS to simplify and centralize R/R Algorithms to determine children not UTD at population level Can generate mailing addresses, lists for generated autodialer or text messages State PHD can conduct R/R centrally Reduce burden of conducting R/R by practices Reach children without usual source R/R messages could come from practice and PHD Responsive to provider and parent desires If practices actively collaborated with PHD to update addesses this could decrease R/R costs care

Methods for Developing the Collaborative Intervention Surveys of Parents and Providers Qualitative data collection Advisory Committees of parents and providers who reviewed qualitative information and made recommendations www.ucdenver.edu/implementation

Implementation Principles Behind Collaborative Centralized Approach Make use of CIIS to simplify and centralize R/R Algorithms to determine children not UTD at population level Can generate mailing addresses, lists for generated autodialer or text messages State PHD can conduct R/R centrally Reduce burden of conducting R/R by practices Reach children without usual source cDesigning interventions for sustainability (low burden, capitalizing on existing technologies) Designing for scalability (populations)

Implementation Principles Behind Collaborative Centralized Approach R/R messages could come from practice and PHD Responsive to provider and parent desires If practices actively collaborated with PHD to update addesses this could decrease R/R costs care Intervention guided by preferences of providers and patients Endorsement by trusted professional/knowledge transfer Collaboration between stakeholders to increase efficiency and decrease cost

Implementation Principles Behind Collaborative Centralized Approach Advisory Committees of parents and providers who reviewed qualitative information and made recommendations for design of intervention Qualitative data collection from both parents and providers (before, during and after intervention) Surveys of parents and providers Iterative testing with stakeholders

Agenda Setting The goals were highly pragmatic—how to increase immunization rates for entire populations of children with highest level of efficiency, lowest cost and without undermining concept of medical home Required highly collaborative approach www.ucdenver.edu/implementation

Developing Partnerships Public Health Departments (PHDs) State PHD was a primary partner in design and execution Association of Local Public Health Organizations (CALPHO) consultant on our grant Local PHDs—presented idea at statewide meeting Input at meeting Key informant interviews before and after intervention CO Children’s Immunization Coalition (CCIC) Local primary care provider organizations (AAP, AAFP) to select Provider Advisory Committee Parent Advisory Committees selected with help from providers www.ucdenver.edu/implementation

Selecting the Population/Setting for Implementation The population included all children in identified counties focus on heterogeneous populations consistent with pragmatic trial rather than classic clinical trial Settings included both urban and rural counties and all primary care providers for children focus on heterogeneous settings and providers consistent with pragmatic trial rather than classic clinical trial www.ucdenver.edu/implementation

The Evaluation Framework: Re-AIM Reach (proportion of the target population that participated in the intervention) Effectiveness (success rate if implemented as in guidelines) Adoption (proportion of settings, practices, and plans that will adopt this intervention) Implementation (extent to which intervention is implemented as intended in the real world) Maintenance (extent to which a program is sustained over time) www.ucdenver.edu/implementation

Primary Objective To compare the effectiveness and cost effectiveness of conducting R/R to increase immunization rates in young children using two methodologies: 1. Collaborative Population-based R/R: conducted centrally by the state health department using the Colorado Immunization Information System (CIIS) with collaboration from private practices 2. Practice-based R/R: R/R conducted at the level of the primary care practice using CIIS

Comparison of “Reach” of Intervention

Percent Receiving Any Vaccine within 6 months (of those needing vaccines at baseline) We need stats—brenda is doing

Percent Brought Up-to-Date within 6 months (of those needing vaccines at baseline) We need stats here—brenda is doing Would prefer 3-d bars for all slides

Subgroup Analysis w/in Practice-based Counties Percent Brought Up-to-Date R/R vs no R/R Need second slide with the bar on the nest slide in green in back of and slightly to the right of bar on left—needs to be clear that it is separate and needs to be labeled population-based…also there needs to be a p value for the difference between 24 and 19 n = 887 n = 17848

Subgroup Analysis w/in Practice-based Counties Percent Brought Up-to-Date R/R vs no R/R

Cost of R/R Per Child who Received ≥1 Vaccine

Cost of R/R Per Child Brought Up-to-Date

Lessons Learned Design implementation interventions with sustainability in mind from the get-go! Know when to rethink the plan! (know when to hold ‘em, know when to fold ‘em….) 3. Listen to stakeholders and recipients of intervention and continue to adjust iteratively www.ucdenver.edu/implementation

References Kempe A, Saville A, Dickinson M, Reynolds J, Herrero D, Beaty B, Eisert S,  Albright K, Dibert E, Koehler V, Calonge N. Population-based versus Practice-based Recall for Childhood Immunizations:  A Randomized Controlled Comparative Effectiveness Trial. Am J Public Health. 2013 Jun;103(6):1116-23. doi: 10.2105/AJPH.2012.301035. PMID: 23237154 Albright K, Gechter K, Kampe A. Importance of mixed methods in pragmatic trials and dissemination and implementation research. Acad Peds 2013 Sep-Oct;13(5):400-7. Suh CA, Saville A, Daley MF, Glazner JE, Barrow J, Stokley S, Dong F, Beaty B, Dickinson LM, Kempe A. Effectiveness and net cost of reminder/recall for adolescent immunizations. Pediatrics. 2012 Jun;129(6):e1437-45. Epub 2012 May 7. Kempe A, Daley MF, Barrow J, Allred N, Hester N, Beaty BL, Crane LA, Pearson K, Berman S. Implementation of Universal Influenza Immunization Recommendations for Healthy Young Children: Results of a Randomized, Controlled Trial with Registry-based Recall. Pediatrics 2005 Jan;115(1):146-154. www.ucdenver.edu/implementation