Business Case for Quality: Best Practices Surrounding Information Technology Stephen S. Raab, M.D. Department of Pathology, University of Pittsburgh, Pittsburgh, PA April 29, 2005

Background n Medical error is the failure of a planned action to be completed as intended or the use of a wrong plan to achieve an aim n Medical errors permeate all levels of patient care and the IOM report has resulted in the greater recognition of error and an increased focus on reporting and reducing errors

Background n Scant data on anatomic pathology error frequencies n Institutional error frequencies may be low, limiting hypothesis testing n Limited data on how pathology errors affect patient care n Root cause analysis of anatomic pathology errors rarely performed and results of these analyses are not disseminated

Error prevention n Minimize psychological precursors; design buffers; build in redundancy n Design tasks: simplification; constraints; standardization n Systems designed to absorb error n Root cause analysis n Shift from training-focus to performance-focus n Data collection to define problem

Initiatives to decrease errors n Creation of audit systems n Use of benchmarking and error tracking systems n Implementation of immediate error reduction systems (e.g., Toyota Production Systems, Six Sigma)

Information technology and patient safety n Most systems are focused on diagnostic reporting n Difficulty in performing quality assurance n Few checks to prevent errors n Systems themselves are often a source of error n Few databases to study quality

Quality assurance n Cytologic-histologic (CH) correlation performed in all American labs (CLIA’ 88) n No national standards to perform CH correlation n Information systems not structured to perform cytologic correlation –Case pairs may be visualized prior or after sign-out –Most CH correlation performed with manual review

Cytologic-histologic correlation n What is the best way to perform correlation and how does one use the data? n Letter sent to 162 American labs requesting information on how they performed correlation (response frequency: 32.1%) n Separated material into forms, logs and tally sheets

CAP Checklist n Cytology case number n Sign-out cytology diagnosis n Sign-out cytologist n Original cytotechnologist diagnosis n Sign-out cytotechnologist n Review cytology diagnosis n Review cytologist

CAP Checklist n Surgical pathology case number n Sign-out surgical pathology diagnosis n Sign-out surgical pathologist n Review surgical pathology diagnosis n Review surgical pathologist n Significance of discrepancy n Action taken n Reason for correlation

Minimum expected and additional variables listed on forms only. Bold line represents minimum expected variables that should be present (n = 15).

UPMC technical quality assurance n Record laboratory error: –Accessioning –Gross groom –Histology –Cytology –Transcription

National benchmarking n CAP Q-Probes studies – error rates, turn around time, amended report rates n CAP Q-Tracks studies –Cytologic-histologic correlation –Frozen-permanent section review –Small surgical specimen turn around time

Second viewing of cases n CAP – Q-Probes study 2004 n 74 labs, 6186 specimens, 415 discrepancies (6.7% rate) n Breakdown of errors –48% with change in same category of diagnosis –21% change in category of diagnosis –18% typographical errors –9% patient or specimen information –4% change in margin status

Cytologic-histologic correlation n Year end report for participating labs (56) n In 2003, a total of 19,478 Paps were correlated with 11,336 true positive correlations and 2,433 false positive correlations n Predictive value for positive cytology: 82.3% n Percent positive ASC diagnoses: 64.3% (range 34.1%-89.9%); percent positive AGC diagnoses: 31.4% (range 0%-66.7%)

Cytologic-histologic correlation n Best performers and clustering performed n Best performers: –Quarterly reports, track and trend –Summary reports to clinicians –Document correlation in report 5/9 labs –Keep written log of findings 2/9 labs –Review biopsy if discrepancy 3/9 labs

AHRQ national errors database n 5 year project to monitor pathology errors by creation of a multi-institutional database n University of Pittsburgh, University of Iowa, Henry Ford Health System, Western Pennsylvania Hospital n Goal is to devise plans to reduce cytology and surgical pathology errors

Specific aims 1. Create voluntary, Web-based database and collect errors detected by correlation, secondary review, amended reports, frozen section review 2. Quantitatively analyze error data and generate performance improvement reports 3. Perform root cause analysis; plan and implement interventions to reduce errors 4. Assess success of interventions by quantitative measure; disseminate successful error reduction plans

Database challenges n Current absence of standardized and detailed laboratory workload and quality assurance data sets in widely used laboratory information systems n Current lack of efficient and comprehensive electronic de-identification of un-linked institutional laboratory information system and clinical data

Database construction n The database is Oracle Enterprise Edition implemented on a Sun Ultra E450 Server running Solaris 2.9. The mid-tier is implemented with Oracle’s Application Server (v9.0.3) on a Compaq DL360 Server running Windows The application uses the Oracle http server and mod_pl/sql extensions to generate dynamic web pages from the database to the users via a Microsoft Internet Explorer web browser version 6.0 or higher.

Logical system design n Schema layer contains the actual data and data relations that are stored entirely as numbers and keys n Meta-data layer, the data is defined in terms of data elements and “data objects” n Procedure layer contains a set of dynamic procedures/functions (in PL/SQL) that externalize the data elements n Presentation layer is a series of forms (for data entry, display, query, etc.) that are populated by data elements from meta-data

Future database plans n Database interface to laboratory information systems n Data de-identification n Revision of database architecture n Incorporation of histopathologic image data

Variable frequency of errors across sites InstitutionNumber of errors Total number of cases Error percent A19628, B27966, C796143, D17482,

Correlating case error frequency Error frequency SiteGyn correlating %Non-gyn correlating % A B C D

Variability in assessing sampling versus interpretive error Institution ReasonABCDTotal Interpretation21%11%65%4%33% Screening2%8%4%2%5% Sampling61%73%38%92%60% Unknown18%9%03%6%

Agreement between original and review reason for discrepancy (interpretation or sampling) Review reason SiteABCD Original reason A B ,737 C DUnd

Percentage of total error by specimen type Institution OrganABCDTotal Lung52%20%35%17%34% Thyroid0%5%8%2%6% Bladder22%14%16%33%18% Breast2% 5%17%5%

Error outcome taxonomy n Significant event: error that affects patient outcome; may be classified by severity (mild, moderate, and severe) n No harm event: error does not affect patient outcome n Near miss event: intervention occurred before harm could take place

Differences in grading of discrepancy assignments Institution AssignmentABCDTotal Significant event 0028%14%22% No Harm86%98%32%50%44% Near miss14%040%14%32% Unknown02%021%2%