Issues in Validation of Whole Slide Digital Images for Frozen Section Interpretation Lewis A. Hassell, MD Pathology Visions October 2010

Objectives Historical context of remote FS interpretation--current models in use and potential impact of WSI Validation issues for incorporating WSI into FS evaluation Prospects for future

Telepathology The challenge for the isolated rural surgeon- What should I do? (I’m already in trouble)

Telepathology The challenge to the pathologist- How can I help? (without getting into trouble) -Gross and Microscopic issues -Static image transmission -Streaming video- dual eyes, non-path controlled -Robotic stage-pathologist controlled

Further Historical Challenges System costs for low volume use Transmission times, bandwidth Technical image quality Control issues- diagnostic field selection, etc.

Today Feasibility in FS settings well established Whole slide scanning available High quality video or still cameras available at relatively low cost Low volume, lower cost, potentially transportable scanners being introduced Scanning speeds near 1 min/slide Transmission protocols and bandwidth improved Mobile high resolution wireless displays

Static Image vs. WSI for FS Cost: Static<<WSI Image quality: Static=WSI Viewer control of fields and magnification: Static-none vs. WSI-total Time to capture/forward: Static<WSI

Robotic stage vs. WSI for FS Cost: Robotic<WSI Image Quality: Robotic=WSI Viewer control of image/magnification: Robotic=WSI Time to capture/forward: Robotic>WSI

Local Validation Considerations Off-label use in medical practice requires judgment and skill, and in laboratory medicine, VALIDATION.

Validation Plan Define scope of endeavor Identify factors “Critical to Quality” Consider role of random variation, non- controlled variables Test individual factors in simulation setting Analyze results Modify plan/procedure per findings Adopt

Technical concerns Is the range of devices used suited to the purpose? What confounding variables, or “interfering substances,” are involved? We looked at “off the shelf” range of computer hardware in use in our department and other potential sites of use (home, mobile) We looked at network issues with the goal to identify limitations in usability

Process Considerations Is the procedure equivalent to (or better than) currently available service options in terms of quality and risk? Do added steps introduce delay or other changes sufficient to cause harm or failure to meet standards of practice? Is the process sufficiently robust to perform well under anticipated demands and predicted variability? What if…?

Personnel Considerations Do those performing the testing understand the limitations of the procedure and have demonstrated competence? Liability concerns Credentialing issues Service issues

Study goals Validate WSI for FS interpretation by our departmental pathologists who wish to use Provide self-assessment tool for other pathologists wishing to measure skill and evaluate the potential for WSI FS interpretation— Open access site: ult.htm

What we’ve done so far

Results to date Technical considerations— -In-office hardware suitable for FS diagnosis using WSI -Highest resolution monitors not essential -Campus network (including wireless) acceptable for use

Process results to date Additional time required to scan FS slides will generally not adversely impact ability to meet TAT expectations intra-operatively Interpretation times are also within acceptable limits for most observers Type of FS impacts evaluation time, e.g. margin for basal cell vs. sentinel lymph node

Overall Diagnostic Performance Kappa statistic for initial FS diagnosis and WSI FS diagnosis concordance 0.84 Diagnostic accuracy of 98% (no discrepancy resulting in management change)

Next steps underway Look more carefully at FMEA for various variables Evaluate home-based and mobile settings Evaluate iPad and other mobile device utility and limitations Expand our data of pathologist interpretation of WSI FS cases Empower senior residents to perform after-hour FS evaluation with WSI- competent faculty back-up

Acknowledgements Joel Ramey, BS Michael Talbert, MD Kar Ming Fung, MD, PhD Linda Freeman

Questions?