1. Using Best Practice Recommendations to Improve Data Quality Assurance in an IIS. Nichole D. Lambrecht, MSc; Mike McPherson; Sue Bowden, Kristin Shore, Mike Parsons, and Susan Dickman Immunization Program Kansas Department of Health and Environment To protect the health and environment of all Kansans by promoting responsible choices. 43 rd National Immunization Conference April 2, 2009 Slide 1

2. Objectives Slide 2  Describe the AIRA-MIROW “Data Quality Assurance in IIS: Incoming Data” best practice guide.  Demonstrate how this best practice guide can be used as GAP analysis for improving data quality.  Show new report KS developed to improve DQ To protect the health and environment of all Kansans by promoting responsible choices.

3. Data Quality Assurance in IIS Slide 3 http://www.immregistries.org/pubs/mirow.phtml To protect the health and environment of all Kansans by promoting responsible choices.

4. Definitions Slide 4  Principles - 13  A principle gives a high-level direction that reflects business guidelines, practices, and norms that help guide the development of the more specific business rules.  Business Rules - 32 with 43 parts  A business rule represent specific requirements of how the business must operate based on the laws, policies, regulations, and chosen business/operational style.  Precertification and Providers’ Profile – 12  Precertification is the process of evaluating the incoming data quality of new submitters prior to allowing them to regularly add data to IIS. To protect the health and environment of all Kansans by promoting responsible choices.

5. Principles Slide 5 To protect the health and environment of all Kansans by promoting responsible choices.

6. Business Rules Slide 6 To protect the health and environment of all Kansans by promoting responsible choices.

7. Precertification & Provider Profile Slide 7 To protect the health and environment of all Kansans by promoting responsible choices.

8. How did Kansas rate?  Principles  8 out of 11 are followed (73%)  Not met:   P01: Consistency principle: The conditions (criteria) for validating data items should be the same regardless of how these data items have been reported to an IIS.   P05: Accuracy principle: The data recorded in the IIS should match exactly what happens in a clinical encounter, whether or not it is clinically appropriate.   P06: Appropriate vaccination principle: The vaccinations reported by a provider should be appropriate for the population served at the clinic.  The main barrier is how we treat interface data vs. direct entry. Slide 8 To protect the health and environment of all Kansans by promoting responsible choices.

9. Kansas Evaluation  Business rules QuestionPriority Group A % of APriority Group B % of BPriority Group C % of COverall Total Overall % #1 Fully Met 880%1148%550%2456% #2 Partially Met 220%626%220%1023% #3 Currently Not Met 00%626%330%921% #4 No plan to meet 00%0 0 0 Total10100%23100%10100%43100% Slide 9 To protect the health and environment of all Kansans by promoting responsible choices.

10. Kansas Evaluation cont…  Business rules (cont.)  Partially met   BR108 - A: Vaccinations should appear in the registry within 2 business days of the Report Submission Date.   BR110 - B: VFC-eligible children should have the manufacturer and lot number reported with vaccination event.  Not met  BR119 – B: Route and Site should be consistent with the vaccine type.  BR122 – C: A patient’s VFC eligibility should be consistent with the funding source of the vaccine administered.  Main barriers: The data quality requirements are different (interface vs. direct entry). Slide 10 To protect the health and environment of all Kansans by promoting responsible choices.

11.  Precertification and Providers’ Profiles  Fully met = 58% (7 out of 12)  Partially met = 17% (2 out of 12)   During the pre-certification source data is compared with the medical charts to validate content.  Not met = 25% (3 out of 12)   IIS maintains profiles for each provider practice that reflects age and type of population served, and expected administration of a certain range of vaccine types, in specific proportions.   IIS uses profiles for providers’ practices to identify systemic problems such as: miscoding issues (crosswalk errors), missing vaccine codes, systematic data entry errors (e.g., entering a pneumococcal polysaccharide instead of a conjugate vaccine).  Barriers: More of just a necessity and priority at this point. Kansas Evaluation cont… Slide 11 To protect the health and environment of all Kansans by promoting responsible choices.

12. Outcome of Kansas Assessment  The new data quality report was developed to summarize some of the “not met” and/or “partially met” recommendations.  Completeness - All core data elements were used to determine percentage of those fields filled in.  Timeliness - Reflects the time difference from the date of vaccination occurred and the date it was submitted to IIS.  Accuracy – Measures that help indicate if things “look” accurate were used by applying ACIP recommendations and general guidelines.  HL7 status (if appropriate) – Measures how often a provider sends VXQs, VXUs, and how many errors were occurring. Slide 12 To protect the health and environment of all Kansans by promoting responsible choices.

13. Patient Completeness Slide 13 To protect the health and environment of all Kansans by promoting responsible choices.

14. Vaccination Completeness Slide 14 To protect the health and environment of all Kansans by promoting responsible choices.

15.  Private Provider Timeliness Slide 15  Local Health Department To protect the health and environment of all Kansans by promoting responsible choices.

16. Accuracy – Single Vacc Event Slide 16 7 years of age 11 years of age 7 years of age 65 years of age 10 years of age 19 years of age 1 year of age 5 years of age 1 year of age 10 years of age 2 years of age 9 years of age 27 years of age To protect the health and environment of all Kansans by promoting responsible choices.

17. Accuracy – Multi Vacc Events Slide 17 7 years of age 20 To protect the health and environment of all Kansans by promoting responsible choices.

18.  HL7 status – not part of the data quality guide  This piece of the profile can give us a quick glance at how many messages (per message type) we have received in the last month and when the last date of the message.  With this information we can quickly identify if a provider is updating regularly or if there was a connectivity issue. HL7 Status Slide 18  Business rules to set ideal HL7 traffic  VXQ = VXR + VXX + QCK  VXU ≤ VXQ indicates normal  VXU > VXQ generally indicates batch/legacy upload  VXU > ACK{AE|AR}  VXX:VXQ >.5 – possible duplicate issue To protect the health and environment of all Kansans by promoting responsible choices.

19. Summary  Using common consensus-based best practice approaches to data quality assurance can provide the basis for system and practice evaluation and improvement.  KS - A new report was developed for providers to identify errors in their data entry and/or vaccine administration practices.  IIS staff will review these regularly to make sure there are no high offenders of data quality mishaps.  VFC staff will review these prior to making VFC/AFIX visits to identify any education needs, etc.  This will provide overall quantifiable measures for program evaluation and improvement. Slide 19 To protect the health and environment of all Kansans by promoting responsible choices.

20. Contact Details Slide 20 Nichole Lambrecht, MSc Kansas IIS (KSWebIZ) Project Manager Kansas Department of Health & Environment Phone: 785-296-8119 Email: nlambrecht@kdheks.gov Website: http://www.kdheks.gov/immunize/webiz.html To protect the health and environment of all Kansans by promoting responsible choices.