The Results of Automated Image Analysis Workshop at the 10th European Congress on Telepathology and 4th International Congress on Virtual Microscopy Arvydas Laurinavičius Pathology Visions 2010 VILNIUS UNIVERSITYNATIONAL CENTRE OF PATHOLOGY

Background and Disclaimer Pathologist (Renal) Director, National Center of Pathology, LT Professor, Vilnius University EU COST Telepathology Network EU LLL EUROPALS MB, IHTSDO (SNOMED CT) International Member, CAP User of Aperio and TissueGnostics tools No competing interests

2010 Vilnius Lithuania 2012 Venice Italy

Telepathology - Program Screen clipping taken: ; 11:14

The Goal To provide an overview of automated image analysis tools in terms of their robustness and workflow efficiency in a structured and comparable fashion

Outline Why – A Pathologist’s Vision of the Digital How – Workshop Design and Results What – Ways to Go

>19 th 20 th 21 st century Evolution of Pathology DIGITAL MOLECULAR IMUNOHISTOCHEMISTRY MICROSCOPY MACROSCOPY Spectrum of Technologies

Pathology Lab … transforms biological information into medical Biospecimen Pathology Diagnosis Patient Clinical Decision Spectrum of Technologies

Adding Digital Path-Way Tissue collected Tissue sampled Tissue processed Slides produced Pathologist ”reads” slides Pathologist interprets images Computer scans slides Computer analyzes images Digital pathology Competition? Ignorance? Synergy? Macroscopy Microscopy Immunohistochemistry

Questions asked: Does this work? Why is Digital better than Conventional? Tool or Toy? Long way to go… More specifically: Shall I scan everything? Should scanners be certified for diagnostic use? Is it legal to make a diagnosis on virtual slides? Can I work faster on digital images? Are quantification results reliable?

Innovation versus Routine Psychology Technology Involvement needs awareness

Treat the Tools and Humans equally Pathologist #1 Pathologist #2 perfect Pathologist #2 moderate Tool #1 Tool #2 perfect Tool #2 perfect inter-observer intra-observer moderate ??? Are different tools in agreement? Are they better than we?

Partitioning the Observer Tissue collected Tissue sampled Tissue processed Slides produced Pathologist ”reads” slides Pathologist interprets images Computer scans slides Computer analyzes images 1 st European Scanner Contest Automated Image Analysis Workshop “2 in 1” “Software” “Hardware”

Outline Why – A Pathologist’s Vision of the Digital How – Workshop Design and Results What – Ways to Go Next

Workshop Design Keep simple, explore feasibility of a Contest Estrogen Receptor and HER2 IHC –Whole slide and TMAs from the Spanish QA Program HER2 FISH –Whole slides from the Ntl Ctr Pathol Available for scanning >1 month (at the 1 st ESC) Participants presented their workflow and results at the Workshop Presentations posted at

Participants CompanySpeakerIHCFISH AperioKate Lillard√ Leica/SlidePathSean Costello√ BioImageneVikram Mohan√√ MetaSystemsChristian Schunck√*√* 3DHistechCsaba Hankó√√ * Used for analysis the FISH slides provided

Workshop Results Concordance testing of the results was out of scope, however, some output data provided by the Participants were analyzed

Estrogen Receptor, % Pos Nuclei 3 outlier cases by A, variable ROI selection?

Estrogen Receptor, Total Nuclei Counted B and C, different size of ROI?

Participant B and C Correlation p<.0001 Estrogen Receptor, % Pos Nuclei Strong correlation; nonlinearity possible?

Nonlinear regression p<.0001 B Estrogen Receptor, % Pos Nuclei Nonlinearity: C outputs higher values (frequent 100%) than B

Estrogen Receptor, % Pos Nuclei B tends to output lower values than A and C Not significant

HER2 IHC Score Agreement between B and C C n/aTotal B n/a Total Simple Kappa0.61 Weighted Kappa0.69 Note: different cutoff used by B and C for 3+ (10 vs 30%)

Lessons learned Plan thoroughly, involve Participants Improve scanning logistics, especially FISH Provide gold standard slides, preferably TMAs Define sampling –whole slide, manual annotation, automated ROI detection Harmonize output formats

Outline Why – A Pathologist’s Vision of the Digital How – Workshop Design and Results What – Ways to Go

Ways to Go Do nothing Do the same Do inter-observer (inter-Tool) variability study Develop an ongoing QA program Disseminate the results

Disseminate D-PathLympics Digital Pathology League Scanner Contest Image Analysis Contest