Immunize LA Kids Coalition Evaluation of Immunize LA Kids’ Enhanced AFIX Intervention in South Los Angeles: What Worked? National Immunization Conference March 6, 2006 Working together for healthy children

Tamekia Mosley, MPA, CHES Authors Presenting Author Julia Heinzerling, MPH Contributing Authors Steve Baranov, C.Phil Oliver Brooks, MD Tamekia Mosley, MPA, CHES Lizz Romo, RD Terry Silberman, DrPH Working together for healthy children

Immunize LA Kids is… A REACH 2010 Project, funded by the CDC Supported by additional funding from the California DHS Immunization Branch Administered by South Los Angeles Health Projects (SLAHP), a community-based unit of the Los Angeles Biomedical Research Institute at Harbor-UCLA Medical Center Working together for healthy children

IZ LA Kids Goal To reduce disparities in immunization rates for Latino & African American children in Central & South Los Angeles Working together for healthy children

Improving Physician Office Immunization Systems & Rates Immunize LA Kids implemented an enhanced AFIX Intervention to promote effective strategies for improving immunization rates in physician practices Working together for healthy children

IZ LA Kids’ Strategy Assessment Feedback Incentive Exchange Move MDs & staff to stages where they are: aware, concerned and knowledgeable about IZs motivated to change ready to try new behaviors/strategies capable of sustaining these new behaviors Working together for healthy children

Enhancements to the AFIX Model Additional Follow-up and Support Telephone consults 3 months following assessment Face-to-face visits and small-scale assessments 6 months following assessment Ongoing technical assistance and resources through a telephone Warmline Support provided continuously for up to 5 years Working together for healthy children

Breaking Down the Enhanced AFIX Model Pre-CASA Interview REPEAT steps 2 through 7, annually 1 Ongoing support services provided by the Immunization Nurse, Health Educator, and Project Assistants Assessment CASA MOCHA 8 2 Ongoing consultation and Warmline 7 3 Observation Checklist 6 4 Feedback Inservice 6-month follow-up office visits 5 3-month follow-up telephone contact

Pre-CASA Interview Assess immunization practices & strategies in participating practices at baseline Face-to-face interview designed to: Gain an understanding of the office system Identify current immunization processes, strengths, and areas for improvement Findings guide the development of training curriculum, improvement plan, and goals Working together for healthy children

Assessment of UTD Rates, Practices, & Missed Opportunities CASA: Clinic Assessment Software Application Chart assessment/medical record review Assess Up-to-date (UTD) rates for 4:3:1:3:3 series at 24 months of age Up to 100 charts reviewed for each practice Results shared during in-service MOCHA: Missed Opportunity CHart Assessment Assess the impact of and potential reasons for missed opportunities to vaccinate Working together for healthy children

Assessment of Office Practices Observation of office practices that may influence patient satisfaction & promotion of immunizations Office environment Use of educational materials Staff/patient interaction IZ record screening Findings incorporated into feedback and recommendations provided at the inservice Working together for healthy children

Feedback and Consultation Within 4 weeks of the assessment, conduct a face-to-face inservice Share immunization UTD rates Highlight strengths & areas for improvement Discuss reasons for low UTD rates & missed opportunities Recommend effective strategies; develop an improvement plan; & identify an UTD goal Provide incentives, educational materials and resources Recognize practices that met their goal Working together for healthy children

Ongoing Support Telephone follow-up 3 months following assessment Assess progress, barriers, and successes; respond to questions; and provide additional consultation Face-to-face follow-up 6 months following assessment Conduct small-scale chart review to observe whether recommended strategies are being implemented & assess current IZ practices Ongoing technical assistance and resources Newsletters and updates, Referrals to training programs, Warmline, and Provider Recognition Event Working together for healthy children

Immunize LA Kids’ Reach Since the project’s inception, IZ LA Kids has provided support to 77 physician practices On September 30, 2005, IZ LA Kids was providing services to 52 active practices 25 practices have been lost to follow-up Practice closures New ownership/management Limited time/resources at participating practices Perceived duplication with required IZ/QI audits Working together for healthy children

Immunize LA Kids Evaluation: Assessing Our Impact & Identifying Effective Strategies Assessment of Changes in UTD Rates Analysis of annual UTD (4:3:1:3:3) data for participating physician practices Annual comparison of: Range of UTD Rates Mean UTD Rate Mean Improvement in UTD Rates Working together for healthy children

Analysis of Impact of AFIX Intervention Total of 77 Participating Providers Analysis includes: 21 Providers with 1 – 2 yrs follow-up 27 Providers with 3 – 4 yrs follow-up Analysis excludes: 10 Providers with baseline data only 19 with < 30 records at baseline or follow-up Working together for healthy children

AFIX RESULTS UTD Rates for 24 month-olds * Baseline Follow-up Change # CASA N * UTD N UTD 2 - 3 92 29% 87 29 % --- 4 - 5 137 29% 85 52 % 23 percentage points * Up-to-date: 4 DTP/DTaP, 3 HEP B, 3 HIB, 1 MMR, 3 Polio. * N = Mean number of records assessed. Working together for healthy children

AFIX RESULTS (continued) Providers with 3 – 4 years follow-up demonstrated significant improvement in 24-month UTD rates (> 20 percentage points) Providers with 1 – 2 years follow-up did not have any change in 24-month UTD rates Previous analyses with subset of providers demonstrated strong association between implemention of R/R and increase in UTD rates Working together for healthy children

Immunize LA Kids Evaluation: Assessment of Provider Motivators, Practices, & Preferences Informal interview with active physician practices Conducted with primary contact, using a standardized interview guide Identifies primary motivators for participation Assesses overall satisfaction with the intervention Pinpoints the services that were most & least helpful Identifies future low-intensity activities   Working together for healthy children

Immunize LA Kids Interview Participants 48 physician practices participated in the interview Majority of interviewees were office managers or medical assistants Several physicians participated in the interviews On average, interview participants: had received an average of 3.6 assessments from IZ LA Kids had worked in the physician practice or clinic for 7.9 years Working together for healthy children

Effective Recruitment Strategies How did your office or clinic learn about the Immunize LA Kids project? Nearly 50% respondents (47%) were contacted by IZ LA Kids staff 25% respondents don’t recall how they were recruited 10% respondents were referred by the Vaccines for Children program Others noted being referred by CHDP, other physician offices, and through health fairs Working together for healthy children

The Provider Perspective: Motivators for Participation What are the top 2-3 reasons why your office or clinic decided to participate in the Immunize LA Kids project? The strongest motivator for participation was to learn the proportion of patients who are UTD with immunizations (24% responses) The second most frequent response was to improve IZ coverage levels for health plan, IPA, medical group, VFC, CHDP, or Pay for performance audits (17% responses) 13% responses indicated that the practices joined because of a general desire to improve IZ rates and/or quality of care Working together for healthy children

The Provider Perspective: Motivators for Participation (Continued) About 10% responses suggested the practices joined to: a) improve low IZ rates or b) receive educational materials or incentives Less than 10% responses suggested the practices joined because: a) they were required to or b) wanted to improve patient satisfaction “Other” responses included: to learn strategies to encourage parents to return for recommended immunizations and to learn how their IZ UTD rates compare to other practices in the area Working together for healthy children

The Provider Perspective: Useful Services For each of the following, please let me know whether you felt the resource or service was: not at all helpful, somewhat helpful, or very helpful. All respondents found all services to be somewhat or very helpful. The assessment and in-service were most helpful. “Somewhat helpful”: 4%-13% “Very helpful”: 73% - 94% “Don’t Know”: 2%- 15% , with the largest proportion for the 3 and 6 month follow-up services. Working together for healthy children

The Provider Perspective: Frequency of Visits Do you feel that we provided too few contacts, an appropriate number of contacts, or too many contacts? 96% respondents felt we provided an appropriate number of contacts. 2% respondents felt we provided too few contacts 2% respondents indicated they did not know Several respondents noted that periodic contacts were helpful because they kept the “staff on their toes” and reminded them to implement our recommendations. Working together for healthy children

The Provider Perspective: Change in Immunization Practices As a result of the training and resources we provided, what changes have you made to your immunization practices? 81% respondents indicated they: Improved documentation of IZ in the medical record More consistently assess the immunization record at all visits 71% indicated they more consistently use reminder/recall 42% indicated they improved immunization access by providing immunization-only appointments 35% indicated they more consistently follow true contraindications Working together for healthy children

The Provider Perspective: Change in Immunization Practices (Continued) 27% indicated they more consistently offer shots at acute care visits Less then 25% indicated that they are: a) providing more immunization training to staff, b) using an immunization registry or c) offering culturally-appropriate educational materials/posters Other reported changes include: focusing on missed opportunities, reviewing charts to assess staff compliance with protocols, purging records, and empowering staff to take responsibility for IZs Working together for healthy children

The Provider Perspective: Recommended Changes to our Services What, if anything, could we have done that would have been more helpful to you in improving your immunization practices? Most did not recommend any changes Suggestions included providing: Immunization assessments more frequently A more structured, detailed training for staff Updates on new vaccines More educational materials Better promotion of the Warmline Working together for healthy children

The Provider Perspective: Recommended Changes to our Services Based on your experiences with the services we've provided, would you recommend that other physician offices or clinics participate in this project? Why or why not? 98% respondents would recommend the project to peers It increases attention on immunizations The assessment of UTD rates is valuable It provides information on specific children who are missing shots It provides reasonable strategies for improving IZ coverage It can improve patient satisfaction It works! Working together for healthy children

The Provider Perspective: Recommended Future Services For each of the following, please let me know whether you think the resource or service would be: not at all helpful, somewhat helpful, or very helpful. Most services were expected to be “very helpful” Immunization materials (96%) Quarterly immunization updates (92%) A listing of IZ training sessions (92%) A demonstration of the immunization registry (92%) A CME training program (90%) An educational videotape or CD (81%) A teleconference on immunizations was not strongly recommended 17%: “not at all helpful” ; 38%: “somewhat helpful” Working together for healthy children

The Provider Perspective: Recommended Future Services What other services or resources might be helpful to your office in improving immunization practices and coverage levels? Immunization registry presentation (requested by most participants) Website Information on travel vaccines Trainings and support materials on new vaccines Continued Immunize LA Kids technical assistance Working together for healthy children

Immunize LA Kids Perspective: Recommendations, Lessons, and Conclusions Recruitment Use innovative recruitment strategies and collaborate with other stakeholders WIC Outreach Team CHDP, Health Plans, and VFC State and County Immunization programs Promote the value of the immunization assessment and the benefit of improving performance for other audits Working together for healthy children

Immunize LA Kids Perspective: Recommendations, Lessons, and Conclusions Implementation Use standardized documentation forms and protocols from the start Assess IZ practices in offices annually and compare changes from year to year Provide regular follow-up that is tailored to provider needs Provide demonstrations of immunization registries Working together for healthy children

Immunize LA Kids Perspective: Recommendations, Lessons, and Conclusions Implementation (Continued) Refer to other trainings, mail alerts, and offer LOTS of free educational materials Avoid teleconferences If offering ongoing telephone consultation, clearly promote the benefits of the services and remind practices frequently Use a structured approach with an informal feel Working together for healthy children

Immunize LA Kids Perspective: Recommendations, Lessons, and Conclusions Implementation (Continued) Provide lists of children missing shots; assess current IZ status of these children at a future contact Provide practices with charts showing their change in UTD rates from year to year Encourage practices to set UTD goals Recognize practices for achievements “Waiting room” rewards not popular Working together for healthy children

Immunize LA Kids Perspective: Recommendations, Lessons, and Conclusions Quality Improvement Conduct regular data entry quality assurance audits Develop a program implementation manual Offer refresher trainings to staff Listen to your providers: conduct interviews to identify their preferences and priorities at initiation and during the project Assess practice changes annually; address any deficits in your curricula Working together for healthy children

Chief of Community Health Services To Learn More… Immunize LA Kids Julia Heinzerling, MPH Chief of Community Health Services (323) 757-7244, extension 228 Jheinzerling@slahp.org Tamekia Mosley, MPA Project Coordinator (323) 757-7244, extension 81 tmosley@slahp.org Acknowledgements Immunize LA Kids is supported by the Centers for Disease Control and Prevention (CDC) under the REACH 2010 Initiative and the California Department of Health Services Immunization Branch. Immunize LA Kids receives institutional support from the Los Angeles Biomedical Research Institute at Harbor-UCLA Medical Center. Working together for healthy children