1. Immunize LA Kids Coalition
Evaluation of Immunize LA Kids’ Enhanced AFIX Intervention in South Los Angeles: What Worked?
National Immunization Conference
March 6, 2006
Working together for healthy children

2. Tamekia Mosley, MPA, CHES
Authors
Presenting Author
Julia Heinzerling, MPH
Contributing Authors
Steve Baranov, C.Phil
Oliver Brooks, MD
Tamekia Mosley, MPA, CHES
Lizz Romo, RD
Terry Silberman, DrPH
Working together for healthy children

3. Immunize LA Kids is… A REACH 2010 Project, funded by the CDC
Supported by additional funding from the California DHS Immunization Branch
Administered by South Los Angeles Health Projects (SLAHP), a community-based unit of the Los Angeles Biomedical Research Institute at Harbor-UCLA Medical Center
Working together for healthy children

4. IZ LA Kids Goal
To reduce disparities in immunization rates for Latino & African American children in Central & South Los Angeles
Working together for healthy children

5. Improving Physician Office Immunization Systems & Rates
Immunize LA Kids implemented
an enhanced AFIX Intervention to
promote effective strategies for
improving immunization rates in
physician practices
Working together for healthy children

6. IZ LA Kids’ Strategy Assessment Feedback Incentive Exchange
Move MDs & staff to stages where they are:
aware, concerned and knowledgeable about IZs
motivated to change
ready to try new behaviors/strategies
capable of sustaining these new behaviors
Working together for healthy children

7. Enhancements to the AFIX Model
Additional Follow-up and Support
Telephone consults 3 months following assessment
Face-to-face visits and small-scale assessments 6 months following assessment
Ongoing technical assistance and resources through a telephone Warmline
Support provided continuously for up to 5 years
Working together for healthy children

8. Breaking Down the Enhanced AFIX Model
Pre-CASA Interview
REPEAT steps 2 through 7, annually 1
Ongoing support services
provided by the
Immunization Nurse,
Health Educator,
and Project Assistants
Assessment
CASA
MOCHA 8 2
Ongoing consultation and Warmline 7 3
Observation Checklist 6 4
Feedback Inservice
6-month follow-up office visits 5
3-month follow-up telephone contact

9. Pre-CASA Interview
Assess immunization practices & strategies in participating practices at baseline
Face-to-face interview designed to:
Gain an understanding of the office system
Identify current immunization processes, strengths, and areas for improvement
Findings guide the development of training curriculum, improvement plan, and goals
Working together for healthy children

10. Assessment of UTD Rates, Practices, & Missed Opportunities
CASA: Clinic Assessment Software Application
Chart assessment/medical record review
Assess Up-to-date (UTD) rates for 4:3:1:3:3 series at 24 months of age
Up to 100 charts reviewed for each practice
Results shared during in-service
MOCHA: Missed Opportunity CHart Assessment
Assess the impact of and potential reasons for missed opportunities to vaccinate
Working together for healthy children

11. Assessment of Office Practices
Observation of office practices that may influence patient satisfaction & promotion of immunizations
Office environment
Use of educational materials
Staff/patient interaction
IZ record screening
Findings incorporated into feedback and recommendations provided at the inservice
Working together for healthy children

12. Feedback and Consultation
Within 4 weeks of the assessment, conduct a face-to-face inservice
Share immunization UTD rates
Highlight strengths & areas for improvement
Discuss reasons for low UTD rates & missed opportunities
Recommend effective strategies; develop an improvement plan; & identify an UTD goal
Provide incentives, educational materials and resources
Recognize practices that met their goal
Working together for healthy children

13. Ongoing Support Telephone follow-up 3 months following assessment
Assess progress, barriers, and successes; respond to questions; and provide additional consultation
Face-to-face follow-up 6 months following assessment
Conduct small-scale chart review to observe whether recommended strategies are being implemented & assess current IZ practices
Ongoing technical assistance and resources
Newsletters and updates, Referrals to training programs,
Warmline, and Provider Recognition Event
Working together for healthy children

14. Immunize LA Kids’ Reach
Since the project’s inception, IZ LA Kids has provided support to 77 physician practices
On September 30, 2005, IZ LA Kids was providing services to 52 active practices
25 practices have been lost to follow-up
Practice closures
New ownership/management
Limited time/resources at participating practices
Perceived duplication with required IZ/QI audits
Working together for healthy children

15. Immunize LA Kids Evaluation: Assessing Our Impact & Identifying Effective Strategies
Assessment of Changes in UTD Rates
Analysis of annual UTD (4:3:1:3:3) data for participating physician practices
Annual comparison of:
Range of UTD Rates
Mean UTD Rate
Mean Improvement in UTD Rates
Working together for healthy children

16. Analysis of Impact of AFIX Intervention
Total of 77 Participating Providers
Analysis includes:
21 Providers with 1 – 2 yrs follow-up
27 Providers with 3 – 4 yrs follow-up
Analysis excludes:
10 Providers with baseline data only
19 with < 30 records at baseline or follow-up
Working together for healthy children

17. AFIX RESULTS UTD Rates for 24 month-olds * Baseline Follow-up Change
# CASA N * UTD N UTD
% %
% % percentage points
* Up-to-date: 4 DTP/DTaP, 3 HEP B, 3 HIB, 1 MMR, 3 Polio.
* N = Mean number of records assessed.
Working together for healthy children

18. AFIX RESULTS (continued)
Providers with 3 – 4 years follow-up demonstrated significant improvement in 24-month UTD rates (> 20 percentage points)
Providers with 1 – 2 years follow-up did not have any change in 24-month UTD rates
Previous analyses with subset of providers demonstrated strong association between implemention of R/R and increase in UTD rates
Working together for healthy children

19. Immunize LA Kids Evaluation: Assessment of Provider Motivators, Practices, & Preferences
Informal interview with active physician practices
Conducted with primary contact, using a standardized interview guide
Identifies primary motivators for participation
Assesses overall satisfaction with the intervention
Pinpoints the services that were most & least helpful
Identifies future low-intensity activities
Working together for healthy children

20. Immunize LA Kids Interview Participants
48 physician practices participated in the interview
Majority of interviewees were office managers or medical assistants
Several physicians participated in the interviews
On average, interview participants:
had received an average of 3.6 assessments from IZ LA Kids
had worked in the physician practice or clinic for 7.9 years
Working together for healthy children

21. Effective Recruitment Strategies
How did your office or clinic learn about the Immunize
LA Kids project?
Nearly 50% respondents (47%) were contacted by IZ LA Kids staff
25% respondents don’t recall how they were recruited
10% respondents were referred by the Vaccines for Children program
Others noted being referred by CHDP, other physician offices, and through health fairs
Working together for healthy children

22. The Provider Perspective: Motivators for Participation
What are the top 2-3 reasons why your office or clinic
decided to participate in the Immunize LA Kids project?
The strongest motivator for participation was to learn the proportion of patients who are UTD with immunizations (24% responses)
The second most frequent response was to improve IZ coverage levels for health plan, IPA, medical group, VFC, CHDP, or Pay for performance audits (17% responses)
13% responses indicated that the practices joined because of a general desire to improve IZ rates and/or quality of care
Working together for healthy children

23. The Provider Perspective: Motivators for Participation (Continued)
About 10% responses suggested the practices joined to: a) improve low IZ rates or b) receive educational materials or incentives
Less than 10% responses suggested the practices joined because: a) they were required to or b) wanted to improve patient satisfaction
“Other” responses included: to learn strategies to encourage parents to return for recommended immunizations and to learn how their IZ UTD rates compare to other practices in the area
Working together for healthy children

24. The Provider Perspective: Useful Services
For each of the following, please let me know whether you felt
the resource or service was: not at all helpful, somewhat
helpful, or very helpful.
All respondents found all services to be somewhat or very helpful.
The assessment and in-service were most helpful.
“Somewhat helpful”: 4%-13%
“Very helpful”: 73% - 94%
“Don’t Know”: 2%- 15% , with the largest proportion for the 3 and 6 month follow-up services.
Working together for healthy children

25. The Provider Perspective: Frequency of Visits
Do you feel that we provided too few contacts, an appropriate
number of contacts, or too many contacts?
96% respondents felt we provided an appropriate number of contacts.
2% respondents felt we provided too few contacts
2% respondents indicated they did not know
Several respondents noted that periodic contacts were helpful because they kept the “staff on their toes” and reminded them to implement our recommendations.
Working together for healthy children

26. The Provider Perspective: Change in Immunization Practices
As a result of the training and resources we provided, what changes have you made to your immunization practices?
81% respondents indicated they:
Improved documentation of IZ in the medical record
More consistently assess the immunization record at all visits
71% indicated they more consistently use reminder/recall
42% indicated they improved immunization access by providing immunization-only appointments
35% indicated they more consistently follow true contraindications
Working together for healthy children

27. The Provider Perspective: Change in Immunization Practices (Continued)
27% indicated they more consistently offer shots at acute care visits
Less then 25% indicated that they are: a) providing more immunization training to staff, b) using an immunization registry or c) offering culturally-appropriate educational materials/posters
Other reported changes include: focusing on missed opportunities, reviewing charts to assess staff compliance with protocols, purging records, and empowering staff to take responsibility for IZs
Working together for healthy children

28. The Provider Perspective: Recommended Changes to our Services
What, if anything, could we have done that would have been more
helpful to you in improving your immunization practices?
Most did not recommend any changes
Suggestions included providing:
Immunization assessments more frequently
A more structured, detailed training for staff
Updates on new vaccines
More educational materials
Better promotion of the Warmline
Working together for healthy children

29. The Provider Perspective: Recommended Changes to our Services
Based on your experiences with the services we've provided, would
you recommend that other physician offices or clinics participate in
this project? Why or why not?
98% respondents would recommend the project to peers
It increases attention on immunizations
The assessment of UTD rates is valuable
It provides information on specific children who are missing shots
It provides reasonable strategies for improving IZ coverage
It can improve patient satisfaction
It works!
Working together for healthy children

30. The Provider Perspective: Recommended Future Services
For each of the following, please let me know whether you think the
resource or service would be: not at all helpful, somewhat helpful, or
very helpful.
Most services were expected to be “very helpful”
Immunization materials (96%)
Quarterly immunization updates (92%)
A listing of IZ training sessions (92%)
A demonstration of the immunization registry (92%)
A CME training program (90%)
An educational videotape or CD (81%)
A teleconference on immunizations was not strongly recommended
17%: “not at all helpful” ; 38%: “somewhat helpful”
Working together for healthy children

31. The Provider Perspective: Recommended Future Services
What other services or resources might be helpful to your office in improving immunization practices and coverage levels?
Immunization registry presentation (requested by most participants)
Website
Information on travel vaccines
Trainings and support materials on new vaccines
Continued Immunize LA Kids technical assistance
Working together for healthy children

32. Immunize LA Kids Perspective: Recommendations, Lessons, and Conclusions
Recruitment
Use innovative recruitment strategies and collaborate
with other stakeholders
WIC Outreach Team
CHDP, Health Plans, and VFC
State and County Immunization programs
Promote the value of the immunization assessment and the benefit of improving performance for other audits
Working together for healthy children

33. Immunize LA Kids Perspective: Recommendations, Lessons, and Conclusions
Implementation
Use standardized documentation forms and protocols from the start
Assess IZ practices in offices annually and compare changes from year to year
Provide regular follow-up that is tailored to provider needs
Provide demonstrations of immunization registries
Working together for healthy children

34. Immunize LA Kids Perspective: Recommendations, Lessons, and Conclusions
Implementation (Continued)
Refer to other trainings, mail alerts, and offer LOTS of free educational materials
Avoid teleconferences
If offering ongoing telephone consultation, clearly promote the benefits of the services and remind practices frequently
Use a structured approach with an informal feel
Working together for healthy children

35. Immunize LA Kids Perspective: Recommendations, Lessons, and Conclusions
Implementation (Continued)
Provide lists of children missing shots; assess current IZ status of these children at a future contact
Provide practices with charts showing their change in
UTD rates from year to year
Encourage practices to set UTD goals
Recognize practices for achievements
“Waiting room” rewards not popular
Working together for healthy children

36. Immunize LA Kids Perspective: Recommendations, Lessons, and Conclusions
Quality Improvement
Conduct regular data entry quality assurance audits
Develop a program implementation manual
Offer refresher trainings to staff
Listen to your providers: conduct interviews to identify their preferences and priorities at initiation and during the project
Assess practice changes annually; address any deficits in your curricula
Working together for healthy children

37. Chief of Community Health Services
To Learn More…
Immunize LA Kids
Julia Heinzerling, MPH
Chief of Community Health Services
(323) , extension 228
Tamekia Mosley, MPA
Project Coordinator
(323) , extension 81
Acknowledgements
Immunize LA Kids is supported by the Centers for Disease Control and Prevention (CDC) under the REACH 2010 Initiative and the California Department of Health Services Immunization Branch.
Immunize LA Kids receives institutional support from the Los Angeles Biomedical Research Institute at Harbor-UCLA Medical Center.
Working together for healthy children