Triangulation of Qualitative Methods* We used qualitative methods to describe these steps and interfaces, how activities of patients, providers, and clinic staff influence completion of the CRC screening process, and whether the data stored in the EMR and other clinical information systems lend themselves to quantitative time-based and healthcare team member-driven analyses. *Sobo, Elisa J, Culture and Meaning in Health Services Research. Walnut Creek, CA: Left Coast Press. Qualitative methods improve meaningful use of EMR data for understanding CRC screening completion SC Lee 1, SJ Inrig 1,2, RT Higashi 1, K McCallister 1, WP Bishop 1, B Balasubramanian 3, N Santini 4, JA Tiro 1,3 1 University of Texas Southwestern Medical Center, Dallas, TX; 2 Mount St. Mary’s College, Los Angeles, CA 3 University of Texas School of Public Health-Dallas Regional Campus, Dallas, TX; 4 Parkland Health and Hospital System, Dallas, TX. Recommendation EMR data alone are not sufficient to accurately measure and interpret performance across the CRC screening continuum; key screening information is stored in other clinical information systems. Triangulation of qualitative methods facilitate meaningful collection and analysis of data from these multiple clinical information systems and identify targets to improve programmatic delivery of screening in real-world settings. Structured Observation Observation of physicians, nurses, lab staff, administrators during usual working activities, including communications with patients and co-workers, EMR data entry. 90+ hours at 10 sites Structured Interviews Lead Physicians, Clinical and Administrative Staff at primary care and gastroenterology clinics N=32; transcribed and analyzed Document Analysis Analysis of policies, procedures, training manuals, quality reports, lab worksheets, notes from observation, transcripts Over 1,000 pages, using Nvivo Introduction Delivery of colorectal cancer (CRC) screening involves multiple steps and interfaces (communication and transfer of responsibility) among primary care and gastroenterology provider teams. Data derived from clinical information systems, including the electronic medical record (EMR), may not comprehensively capture these interfaces; this, in turn, may restrict quantitative analyses of clinical information systems and hamper clinically meaningful interpretation. Research Questions 1.What is the process by which FIT kits are tracked and resulted? When are insufficient, invalid, and cancelled FIT results used and how are they documented? 2.What factors influence the length of time between reporting of abnormal FIT results to completion of diagnostic colonoscopy? FIT Results Reporting Structured observation of lab processes, structured interviews with lab supervisors and document analysis revealed loss of key data from returned FIT kits, specifically: Instead of recorded in the pathology database, the following data are documented on paper and filed manually—collection date of each card, result of each card, reason for each invalid card (e.g., card not turned in, sample not dated, sample expired, too much/little specimen provided) EMR (Epic) set up to only accept certain data from Cerner, including: (1) number of cards tested, (2) overall result, and (3) notes (optional free text about results). Data not transferred from Cerner to Epic include: date of last sample, date of order [for billing purposes], date sample received in the lab, date of processing, date of entry. Implications: Information necessary to assess the frequency of and communicate reasons for invalid samples are not systematically available to providers, constraining ability to improve instructions to patients and avoid false negatives. Future work: Access pathology data to quantify the source of delays in reporting FIT results (attributable to patient, lab staff, or provider behavior) and average time associated with each delay. Distribution of FIT Kits Structured observation at primary care clinics and structured interviews with clinicians, clinical staff, and administrative staff revealed that some clinics modified the distribution of FIT kits by: adding colored labels to remind patients to write the date of collection on each of the three cards (lab staff are required to discard samples if collection date is missing and cancel the order); removing mailing envelopes from the kits to reduce confusion about how to return kits (patients are instructed to return kits in-person); providing verbal and written instructions to emphasize the importance of CRC screening and returning the kit within 10 days of first sample collection (samples older than 14 days are discarded, per manufacture protocol, and labeled invalid). Implications: Differences in FIT distribution procedures between clinics may contribute to variation in: patient adherence to FIT instructions proportion of returned samples that are labeled as canceled or invalid Future work: Quantify the impact of current FIT distribution procedures on the rate of canceled and invalid results. Colonoscopy Referral Process Structured interviews with primary care clinicians revealed: Frustration with the colonoscopy referral process because many referrals were being delayed or denied “pending further action” Structured observation of GI staff revealed: a centralized process for triaging and scheduling colonoscopy referrals; GI staff may delay or deny a referral pending medical clearance GI waits for the ordering provider to resolve issues and consult other specialists if needed. If the ordering provider takes no action, GI staff close the referral. Document analysis of GI clinic procedures detailed the triage criteria, but not how GI staff communicate these delays and denials. Implications: Confusion about coordination of care between primary care and GI may introduce delays and drop-offs during the colonoscopy referral step. Future work: Quantify whether delays are longer for high-risk, co-morbid patients. Acknowledgements We thank Parkland administrators, physicians and clinical staff for their candor and assistance with our data collection efforts. Research made possible by a grant award to the Parkland - UT Southwestern PROSPR Center from the National Cancer Institute (5U54 CA163308).