Measuring diagnostic accuracy of using digital slides in routine histopathology and analyzing sources of diagnostic errors László FÓNYAD 1st. Dept. of Pathology and Experimental Cancer Research, Semmelweis University, Budapest, Hungary, 3DHISTECH Ltd., Budapest, Hungary

Measuring diagnostic accuracy digital slides in routine histopathology The role of digital slides in pathology Our questions RecommendationsBackgroundMaterials & methodsResults Designing a retrospective, comparative study Statistics can be made to prove anything – even the truth What should the FDA regulate?

Background Spreading of digital microscopes (DM) Worldwide is unquestionable. Reports have been published concerning the reliability of digital slides (DS) in routine diagnostics showing great advances in the technical parameters of the scanning process and in diagnostic confidence. The real revolution of DSs are still waited for. Today the most common fields of using DSs are only research, education and as a paradox: quality controll. We wanted to estimate the major causes of dislike and/or dissatisfaction that could explain the mistrust in DS therefore interfere the real breakthrough of this techique in the routine work. Measuring diagnostic accuracy digital slides in routine histopathology

Background Our questions were: Can we define a list of samples according to the origin and type of stain etc. where the DM is sufficient to use in routine practice and define those where it is not recommended? Is it possible to estimate the type of errors resulting in misdiagnosed cases? Does the pathologists’ interpretative skills and experience effect the diagnostic results using DM and how important these factors are comparing to the actual quality of the DS and working conditions? Measuring diagnostic accuracy digital slides in routine histopathology

Materials & methods Materials: Scanners and softwares were provided by the 3DHISTECH Ltd., Budapest 280 cases were enrolled to the study 1771 slides were scanned 994 H&E, 70 Giemsa, 174 other special stains (mainly PAS, Prussian-blue, picrosyrius, Masson's trichrome), 553 immunohistochemistry slides. No smears or cytology samples were scanned were evaluated by the pathologists for digital diagnose Table 1., Enrolled cases according to the localisation skin49 bone marrow10 breast26 upper GI- tract37 soft tissue21 liver26 lymphnode18 thyroid gland15 lung22 large bowel27 kidney29 Measuring diagnostic accuracy digital slides in routine histopathology

Materials & methods Method: 7 pathologists Pathologista A, B and D received cases, specific to their field. Pathologist C, E, F, G received non-field specific cases too. Initially, only those slides that were available for the first assessment (mostly HE) were uploaded to the digital database. A Clinical Researc Form was filled out. The diagnostic concordance and the reasons related DS to diagnostic uncertainity were analyzed. The incoherent cases were graded and 4 types of diagnostic errors were defined. A. Scan quality 1-Unacceptable 2-Poor 3-Adequate 4-Good 5-Excellent The reason of dissatisfaction Important areas of the slide are out of focus (y/n) Incomplete scan (y/n) The color fidelity is poor (y/n) Other (free text) B: Diagnostic confidence 1-Uncertain 2-Likely 3-Confident The reason of uncertainty is due to: Case complexity (y/n) Poor image quality (y/n) Types of diagnostic error Type I. non relevant incoherence – uncertainty recorded. Type II. non relevant incoherence – uncertainty not recorded. Type III. relevant incoherence – uncertainty recorded. Type IV. relevant incoherence – uncertainty not recorded. Measuring diagnostic accuracy digital slides in routine histopathology

Results Technical results: Scantime ratio: 1,215 min/cm2 Average quality of the 1530 slides was 4,45/5. At 34 slides the reason of dissatisfaction was that “important areas of the slide were out of focus”. Twice the scan was considered incomplete and 10 times the color fidelity was rated poor. Measuring diagnostic accuracy digital slides in routine histopathology

Results Diagnostic results diagnostic confidence (1-3) uncertainty due to poore image quality concord ant discordantreassesedincoherentnon-rel - recorded non-rel - missed rel - recorded relevant - missed 2,744,64%78,57%21,43%9,64%11,79%1,07%2,50%5,71%2,50% Measuring diagnostic accuracy digital slides in routine histopathology

Results Diagnostic results Can we define a list of samples according to the origin where the DM is sufficient to use in routine practice and define those where it is not recommended? Measuring diagnostic accuracy digital slides in routine histopathology

Results Diagnostic results Does the pathologists’ interpretative skills and experience effect the diagnostic results using DM? Results excluding non-field specific cases: number of cases concordantdiscordantreassesedincoherentnon rel - recorded non rel - missed rel - recorded rel - missed %%%% 28078,57%21,43%9,64%11,79%1,07%2,50%5,71%2,50% 17283,14%16,86%8,72%8,14%0,58%0,00%5,81%1,74% Measuring diagnostic accuracy digital slides in routine histopathology

Recommendations 1.Based on others and our results we think that the level of diagnostic confidence using digital slides are acceptable. 2.Reasons responsible for diagnostic errors are mostly personal and reflects the competency of the examiner. 3.Technical reasons, potentially responsible for errors (poor color fidelity, blurred image) are detectable by the examiner and correction of it could be initiated. (rescan, recut, restain etc.) 4.However strict regulations required for the scanning process inserted to the prediagnostic phase Safety of sample recognition: how do we prevent data loss because of incomplete scan. Proper glass slide handling: provide accurate slide ID recognition. Minimalize the chance of breaking glass slides during scanning process. Measuring diagnostic accuracy digital slides in routine histopathology

Measuring diagnostic accuracy of using digital slides in routine histopathology and analyzing sources of diagnostic errors László FÓNYAD 1st. Dept. of Pathology and Experimental Cancer Research, Semmelweis University, Budapest,, Hungary Thank You for Your attention!