1. Quality Assurance Challenges Associated with Decentralizing the Entry of Tuberculosis Surveillance Data TB PEN Conference September 2011 Sandra P. Matus, M.P.H., Epidemiologist, PEN Focal Point Andrea Palmer, Karen Fujii, Cathy Goldsborough, Maureen Donovan, Tori Miazad, Wendy Cronin Maryland Department of Health and Mental Hygiene Infectious Disease and Environmental Health Administration Center for Tuberculosis Control and Prevention

2. Infectious Disease & Environmental Health Administration September 2011 2 To improve the health of Marylanders by reducing the transmission of infectious diseases, helping impacted persons live longer, healthier lives, and protecting individuals and communities from environmental health hazards We work in partnership with local health departments, providers, community based organizations, and public and private sector agencies to provide public health leadership in the prevention, control, monitoring, and treatment of infectious diseases and environmental health hazards. MISSION

3. Infectious Disease & Environmental Health Administration September 2011 3 Background Pre-2009 – TIMS database – Used MD-specific form – Data entered at state level – Random surveys revealed data were 99- 100% accurate Post-2009 – NEDSS database – Use revised RVCT – TB data entry was decentralized to local health departments (LHDs)

4. Infectious Disease & Environmental Health Administration September 2011 4 Purpose Assess the accuracy and completeness of TB patient data entered into the new surveillance system by LHDs, including Baltimore City Identify training needs and develop materials to assist in the accurate completion of the RVCT

5. Infectious Disease & Environmental Health Administration September 2011 5 Methods For LHDs with <10 cases in 2009, all patient records for 2009 were reviewed; for LHDs with ≥10 cases in 2009, a sampling of records, proportionately distributed across case managers, were reviewed. RVCT forms were compared with patient records for completeness and accuracy, with variables scored. 1 - 3 cases 17 - 70 cases No TB 4 - 8 cases Maryland Case Numbers, 2009

6. Infectious Disease & Environmental Health Administration September 2011 6 Study Demographics Country of Origin U.S.-Born Foreign-Born Age Age GroupPercent <52% 6-145% 15-2413% 25-4438% 45-6423% 65+19% Sex Percent Female48% Male52%

7. Infectious Disease & Environmental Health Administration September 2011 7 Results Summary Over a period of 8 months, 100 patient record reviews required an average of 40 minutes per record (~67 total hours for data collection). Of 8060 applicable RVCT responses: 491 (6%) were incorrect (LHD median: 4%, range:0-8%) 625 (8%) were missing (LHD median: 3%, range:0-33%) 305 (4%) were not documented (LHD median: 1%, range=0-8%) The greatest proportions of RVCT errors were as follows: 41% (39/96) incorrect: “Date Reported” 42% (25/60) missing: “Evidence of Miliary TB” subcategory under “Initial Chest Radiograph” 28% (28/99) not documented: “Immigration Status at First Entry” “Therapy Start Date” was considered acceptable: Median days difference between RVCT and chart documentation: 1 (range: 1-4)

8. Infectious Disease & Environmental Health Administration September 2011 8 Results Treatment Variables Variable Name **RVCT Scores NIMND Date Therapy Started9810%1% 0% Culture Conversion Date5032%4% Chest X-Ray (CXR)986%2% 1% CXR: Evidence of a Cavity8410%11% 4% CXR: Evidence of Miliary846%21% 2% Date Therapy Stopped6015%28%8% **RVCT responses were marked as: I=Incorrect: Response differed from data in the patient record M=Missing: No response on RVCT ND=Not Documented: Response could not be substantiated by data in the patient record NA=Not Applicable: Field did not apply to the patient **RVCT responses were marked as: I=Incorrect: Response differed from data in the patient record M=Missing: No response on RVCT ND=Not Documented: Response could not be substantiated by data in the patient record NA=Not Applicable: Field did not apply to the patient

9. Infectious Disease & Environmental Health Administration September 2011 9 Results, cont’d. Risk Factor Variables Variable Name **RVCT Scores NIMND Foreign-Born980% 6% HIV Co-infection999%0%8% Injecting Drug Use994%1%21% Non-Injecting Drug Use994%1%21% Excess Alcohol Use995%1%21% Homelessness984%0%12% **RVCT responses were marked as: I=Incorrect: Response differed from data in the patient record M=Missing: No response on RVCT ND=Not Documented: Response could not be substantiated by data in the patient record NA=Not Applicable: Field did not apply to the patient **RVCT responses were marked as: I=Incorrect: Response differed from data in the patient record M=Missing: No response on RVCT ND=Not Documented: Response could not be substantiated by data in the patient record NA=Not Applicable: Field did not apply to the patient

10. Infectious Disease & Environmental Health Administration September 2011 10 Results, cont’d. Tissue/Fluids Culture: Date Reported (1-119) (1-20 ) (1-190) (1-24) (1-30) (1-74) Range (in days)

11. Infectious Disease & Environmental Health Administration September 2011 11 Results, cont’d. Table 4. Examples of Clinically Important Results Variable NameRVCT ResponseInformation in ChartFreq. Site of DiseaseSpinalSpinal and Pulmonary1 Sputum Smear NegativePositive2 Not DoneNegative2 Sputum Culture NegativePositive1 Not Done/Unk.Negative6 Addtl. Risk FactorsNoneDiabetes/contact of TB4(3/1) Cult. of Tissue/Fluids NegativePositive1 Not DonePositive1 Negative1 CXR: Cavitary TB Unk.No2 YesNo3

12. Infectious Disease & Environmental Health Administration September 2011 12 Examples of Errors Excess alcohol – RVCT marked “no”. Patient record later documented that patient was repeatedly coming to clinic under the influence. Homelessness – Adult patient alternating stays between family and friends (transient housing) was marked as “not homeless.” Primary Occupation – Two-year-old marked as “unemployed.” Child should have been marked as “not seeking employment.”

13. Infectious Disease & Environmental Health Administration September 2011 13 Conclusions Decentralizing data entry from state to local level resulted in a significant decline in accuracy despite education and training efforts. Data quality assessments of this magnitude are time-consuming and costly. Ongoing QA of reported data essential to ensure accurate information for local, state, and national use.

14. Infectious Disease & Environmental Health Administration September 2011 14 Recommendations 1. Focus on problem variables through – Development of RVCT quick reference guides – Revised RVCT trainings 2. Standardize quality assurance measures and assess periodically, both centrally and locally. 3. Duplicate study in other states and programs to identify possible trends and problem fields.

15. Infectious Disease & Environmental Health Administration September 2011 15 Acknowledgements DHMH TB: Andrea Palmer Karen Fujii Cathy Goldsborough Maureen Donovan Tori Miazad Wendy Cronin Program Evaluation Team: Yvonne Richards Bonnie Lewis Judy Thomas Nicketta Johnson Loretta Gossett And all other local TB program staff who helped with this project!