Comparing algorithms for linking immunization registry records to community-based outreach data S. Findley, M. Irigoyen, P. Sternfels, F. Chimkin, M. Sanchez Northern Manhattan Start Right Coalition Northern Manhattan Immunization Partnership Columbia University, New York, NY

The Challenge Immunization registries offer great potential to support the immunization outreach work of community groups Potential difficulties with community-based registry Identifier information not unique or definitive, due to spelling errors, name differences, date errors Children identified in community not in registry Efficient strategies for matching immunization records are critical to optimizing linkages with other databases.

Objective To assess the effectiveness of alternative matching algorithms for linking children identified through community-based programs with records in a practice network immunization registry

Study Community Northern Manhattan, New York City Low-income, minority community of 420,000, with immunization coverage rates below city and national average

Northern Manhattan Start Right Coalition Community coalition to improve early childhood immunization coverage 23 community social service organizations integrate immunization outreach into their routine programs Supported by CDC’s Reach 2010 Initiative, Office of Minority Health

Community organizations identify children not current in immunizations Trained community social service staff seek children <5 from among families they serve Start Right workers enroll parents Children’s immunization status is recorded with identifiers: Child’s first and last names, parents’ last name, date of birth, gender Start Right workers receive regular immunization updates from EzVAC, a network immunization registry

Immunization Status checked on the EzVAC Immunization Registry Private network registry for community in Northern Manhattan, launched March 1999 Web-based and real time Regular upload to the NYC Citywide Immunization Registry Currently at 30+ practices 120,000+ children in the registry EzVAC supported by NIP, CDC Types of clinics and settings

Start Right to Registry Match Variables Automated match processes Child’s first and last name Date of birth of child Manual matching Parents’ first and last name Child’s gender Phone number Automated matching involves using programming in a Microsoft Access database. Manual matching is completed by hand and eye.

Start Right to Registry Match Procedures 1) Check for exact match Date of Birth (DOB) Last and first name 2) Check for Possible Matches First 2 letters of first and last name 3 day window for DOB 3) Manual verification of possible match: Correct vs. implausible Uses extra information (e.g. parents’ name) Increasingly resource intensive as enrollment grows #1 and #2 completed by automated means. After finding possible matches using automated processes, manual verification was completed by looking at the discrepancies between variables used in the automated processes, as well as other variables (e.g., gender, parents’ name, etc) to verify whether the possible match was a real match. As the number of cases grows, the more possible matches there are, and therefore, the greater the number of manually verifications are necessary. Other variables could have been used for automated matches (e.g., parent’s maiden name, an “aka” variable), but when tested these variables provided many more rows of possible matches, but little, if any actual matches – mainly due to lack of data.

Alternative Matching Criteria Match Criteria Last Name First Name DOB I: First 2 letters OR First 2 letters DOB +/- 3 days II: AND III: ---- Exact first IV: Exact last ----- V: II, IV, III sequentially II, IV, III sequentially Alg. I: first 2 letters of last name OR first 2 letters of first name, and DOB +- 3 days (initial algorithm) Alg. II: first 2 letters of last name AND first 2 letters of first name, and DOB +- 3 days Alg. III: exact first name and DOB +-3 days Alg. IV: exact last name and DOB +-3 days Alg. V: hierarchical combination of II then IV then III – once a child is considered a match in one algorithm, the child is removed from the list to be matched in the next algorithm.

Results for Criteria I Last OR First Name (First 2 letters) & DOB window Exact match means that dob, last name, and first name matched. Corrected match means that there was a possible match, which was manually verified as a match. Implausible means that there was a possible match, which was manually verified as either no match or to questionable to be considered a match. No match means that there no possible or exact match was obtained for that child. Total matches = Exact Match + Corrected Match Possible Matches to be manually verified = the number of rows of data that meet the algorithm criteria. These then need to be manually verified. Total Matches = 379 Possible Matches to be manually verified = 5967

Results for Criteria II Last AND First Name (First 2 letters) & DOB window Same note as previous slide (slide 12) Total Matches = 318 Possible Matches to be manually verified = 189

Results for Criteria III Exact first name & DOB window Same note from slide 12. Total Matches = 270 Possible Matches to be manually verified = 308

Results for Criteria IV Exact last name & DOB window Same note as slide 12. Total Matches = 301 Possible Matches to be manually verified = 560

Results for Criteria V Hierarchical Combination of Algorithms II, IV, and III Same note as slide 12. Note that in Algorithm V, Algorithms II, IV, and III are completed consecutively. Those cases that matched in a previous algorithm are not included in the next algorithms matching. This greatly reduces the total number of possible matches. Total Matches = 379 Possible Matches to be manually verified = 839

Comparison of Match Criteria This chart compares the five algorithms and shows the differences between the types of matches. Highlights: Algorithm I has the fewest percentage of no matches, but at the same time has, by far, the largest percentage of implausible matches (possible matches manually verified as no match). Algorithm II – IV completed separately reduce the percentage of implausible matches, but at the same time greatly increase the percentage of no matches. Algorithm V works out best because it has the same raw number of matches as Algorithm I, but much less number of implausible matches.

Results Only 18% of the children were exact matches Less restrictive match criteria improve the match rate, but require extensive manual verification of possible matches Hierarchical application of criteria improved match rate to 35%, while decreasing hand verification to 43% Improved match rate to 35% means that in addition to the 18% exact match, an additional 17% matches were found using the combination of automated/manual match verification. Note that Algorithm I also had the same match rate, but many more possible matches needed to be verified to arrive at that amount.

Continuing Applications Hierarchical criteria performed for matching purposes for two groups: 155 SR records from one group: 97 matches (62.6%) 45 exact and 52 manually verified as a match Out of 220 records, 160 matches by hierarchical method vs. 105 for least restrictive method Based upon our success utilizing the hierarchical combination of algorithms (Algorithm V), we continue to use this Algorithms for matching procedures. More than 6½ times as many rows would have needed to be manually verified had we used Algorithm I (least restrictive method).

Conclusions Outreach workers can provide information needed to identify children in immunization registries, but it is likely that many of the children will not be located using exact match procedures Alternative and hierarchical matching algorithms Improve match rates Are efficient This efficiency becomes increasingly valuable as immunization promotion program grows