1. Comparing algorithms for linking immunization registry records to community-based outreach data
S. Findley, M. Irigoyen, P. Sternfels, F. Chimkin, M. Sanchez
Northern Manhattan Start Right Coalition
Northern Manhattan Immunization Partnership
Columbia University, New York, NY

2. The Challenge
Immunization registries offer great potential to support the immunization outreach work of community groups
Potential difficulties with community-based registry
Identifier information not unique or definitive, due to spelling errors, name differences, date errors
Children identified in community not in registry
Efficient strategies for matching immunization records are critical to optimizing linkages with other databases.

3. Objective
To assess the effectiveness of alternative matching algorithms for linking children identified through community-based programs with records in a practice network immunization registry

4. Study Community Northern Manhattan, New York City
Low-income, minority community of 420,000, with immunization coverage rates below city and national average

5. Northern Manhattan Start Right Coalition
Community coalition to improve early childhood immunization coverage
23 community social service organizations integrate immunization outreach into their routine programs
Supported by CDC’s Reach 2010 Initiative, Office of Minority Health

6. Community organizations identify children not current in immunizations
Trained community social service staff seek children <5 from among families they serve
Start Right workers enroll parents
Children’s immunization status is recorded with identifiers:
Child’s first and last names, parents’ last name, date of birth, gender
Start Right workers receive regular immunization updates from EzVAC, a network immunization registry

7. Immunization Status checked on the EzVAC Immunization Registry
Private network registry for community in Northern Manhattan, launched March 1999
Web-based and real time
Regular upload to the NYC Citywide Immunization Registry
Currently at 30+ practices
120,000+ children in the registry
EzVAC supported by NIP, CDC
Types of clinics and settings

8. Start Right to Registry Match Variables
Automated match processes
Child’s first and last name
Date of birth of child
Manual matching
Parents’ first and last name
Child’s gender
Phone number
Automated matching involves using programming in a Microsoft Access database. Manual matching is completed by hand and eye.

9. Start Right to Registry Match Procedures
1) Check for exact match
Date of Birth (DOB)
Last and first name
2) Check for Possible Matches
First 2 letters of first and last name
3 day window for DOB
3) Manual verification of possible match:
Correct vs. implausible
Uses extra information (e.g. parents’ name)
Increasingly resource intensive as enrollment grows
#1 and #2 completed by automated means.
After finding possible matches using automated processes, manual verification was completed by looking at the discrepancies between variables used in the automated processes, as well as other variables (e.g., gender, parents’ name, etc) to verify whether the possible match was a real match. As the number of cases grows, the more possible matches there are, and therefore, the greater the number of manually verifications are necessary.
Other variables could have been used for automated matches (e.g., parent’s maiden name, an “aka” variable), but when tested these variables provided many more rows of possible matches, but little, if any actual matches – mainly due to lack of data.

10. Alternative Matching Criteria
Match Criteria
Last Name
First Name
DOB I:
First 2 letters OR
First 2 letters
DOB +/- 3 days
II:
AND
III:
----
Exact first
IV:
Exact last
----- V:
II, IV, III sequentially
II, IV, III sequentially
Alg. I: first 2 letters of last name OR first 2 letters of first name, and DOB +- 3 days (initial algorithm)
Alg. II: first 2 letters of last name AND first 2 letters of first name, and DOB +- 3 days
Alg. III: exact first name and DOB +-3 days
Alg. IV: exact last name and DOB +-3 days
Alg. V: hierarchical combination of II then IV then III – once a child is considered a match in one algorithm, the child is removed from the list to be matched in the next algorithm.

11. Results for Criteria I Last OR First Name (First 2 letters) & DOB window
Exact match means that dob, last name, and first name matched.
Corrected match means that there was a possible match, which was manually verified as a match.
Implausible means that there was a possible match, which was manually verified as either no match or to questionable to be considered a match.
No match means that there no possible or exact match was obtained for that child.
Total matches = Exact Match + Corrected Match
Possible Matches to be manually verified = the number of rows of data that meet the algorithm criteria. These then need to be manually verified.
Total Matches = 379
Possible Matches to be manually verified = 5967

12. Results for Criteria II Last AND First Name (First 2 letters) & DOB window
Same note as previous slide (slide 12)
Total Matches = 318
Possible Matches to be manually verified = 189

13. Results for Criteria III Exact first name & DOB window
Same note from slide 12.
Total Matches = 270
Possible Matches to be manually verified = 308

14. Results for Criteria IV Exact last name & DOB window
Same note as slide 12.
Total Matches = 301
Possible Matches to be manually verified = 560

15. Results for Criteria V Hierarchical Combination of Algorithms II, IV, and III
Same note as slide 12.
Note that in Algorithm V, Algorithms II, IV, and III are completed consecutively. Those cases that matched in a previous algorithm are not included in the next algorithms matching. This greatly reduces the total number of possible matches.
Total Matches = 379
Possible Matches to be manually verified = 839

16. Comparison of Match Criteria
This chart compares the five algorithms and shows the differences between the types of matches.
Highlights:
Algorithm I has the fewest percentage of no matches, but at the same time has, by far, the largest percentage of implausible matches (possible matches manually verified as no match).
Algorithm II – IV completed separately reduce the percentage of implausible matches, but at the same time greatly increase the percentage of no matches.
Algorithm V works out best because it has the same raw number of matches as Algorithm I, but much less number of implausible matches.

17. Results Only 18% of the children were exact matches
Less restrictive match criteria improve the match rate, but require extensive manual verification of possible matches
Hierarchical application of criteria improved match rate to 35%, while decreasing hand verification to 43%
Improved match rate to 35% means that in addition to the 18% exact match, an additional 17% matches were found using the combination of automated/manual match verification.
Note that Algorithm I also had the same match rate, but many more possible matches needed to be verified to arrive at that amount.

18. Continuing Applications
Hierarchical criteria performed for matching purposes for two groups:
155 SR records from one group:
97 matches (62.6%)
45 exact and 52 manually verified as a match
Out of 220 records, 160 matches by hierarchical method vs. 105 for least restrictive method
Based upon our success utilizing the hierarchical combination of algorithms (Algorithm V), we continue to use this Algorithms for matching procedures.
More than 6½ times as many rows would have needed to be manually verified had we used Algorithm I (least restrictive method).

19. Conclusions
Outreach workers can provide information needed to identify children in immunization registries, but it is likely that many of the children will not be located using exact match procedures
Alternative and hierarchical matching algorithms
Improve match rates
Are efficient
This efficiency becomes increasingly valuable as immunization promotion program grows