1. Maura Bidinost User Experience Designer Omnyx LLC Usability: A Critical Factor in the Successful Adoption of Digital Pathology for Routine Sign-out

2. Primary Uses of Digital Pathology Today 2 Turn Around Time Volume NON-CRITICAL CRITICAL LOW HIGH Research Frozen Sections 2 nd Opinions Routine Sign-out Education Digital Pathology Rarely used

3. FDA CE Mark Provincial Regulations CAP Regulatory 3 What Hinders Adoption for Routine Sign-out?

4. Image Quality Time to Image Scalability FDA CE Mark Provincial Regulations CAP Technology Regulatory 4 What Hinders Adoption for Routine Sign-out? Quality Team

5. Image Quality Time to Image Scalability “I don’t have time to learn a new system.” “Computers make things complicated.” “It doesn’t accommodate my workflow.” “I can’t afford to be slowed down.” “I like my microscope.” “I need to be able to trust the system.” FDA CE Mark Provincial Regulations CAP Technology System Interaction Regulatory 5 What Hinders Adoption for Routine Sign-out? Quality Team Engineering

6. User-Centered Design (UCD) 6 UCD is a philosophy in which the end user is placed at the center of the process and given extensive attention at each stage. Design Development Analysis & Research Evaluation

7. User-Centered Design (UCD) 7 UCD is a philosophy in which the end user is placed at the center of the process and given extensive attention at each stage. Design Development Analysis & Research Evaluation

8. Design Development Evaluation User-Centered Design (UCD) 8 UCD is a philosophy in which the end user is placed at the center of the process and given extensive attention at each stage. Analysis & Research Site Visits Contextual Inquiry & Design Competitive Analysis Heuristic Evaluation Affinity Diagramming Analysis & Research Site Visits Contextual Inquiry & Design Competitive Analysis Heuristic Evaluation Affinity Diagramming

9. User-Centered Design (UCD) 9 UCD is a philosophy in which the end user is placed at the center of the process and given extensive attention at each stage. Design Development Analysis & Research Evaluation

10. Development Analysis & Research Evaluation User-Centered Design (UCD) 10 UCD is a philosophy in which the end user is placed at the center of the process and given extensive attention at each stage. Design Persona’s Storyboarding Needs Validation Wireframes Participatory Design Design Persona’s Storyboarding Needs Validation Wireframes Participatory Design

11. User-Centered Design (UCD) 11 UCD is a philosophy in which the end user is placed at the center of the process and given extensive attention at each stage. Design Development Analysis & Research Evaluation

12. User-Centered Design (UCD) 12 UCD is a philosophy in which the end user is placed at the center of the process and given extensive attention at each stage. Design Development Analysis & Research Evaluation

13. Development Evaluation User-Centered Design (UCD) 13 UCD is a philosophy in which the end user is placed at the center of the process and given extensive attention at each stage. Design Analysis & Research Evaluation Heuristic Evaluation Performance Testing Think-Aloud Usability Testing Evaluation Heuristic Evaluation Performance Testing Think-Aloud Usability Testing

14. Participants are presented with a realistic set of tasks Participants are asked to say whatever they are seeing, thinking, doing, and feeling, as they go about their task. Participant’s actions on the screen and facial expressions are generally recorded as well as observed first hand by a usability specialists. Upon completion of each task, participants are generally presented with a brief questionnaire that provides more quantitative feedback. What is Think-Aloud Usability Testing? 14

15. Think-Aloud Testing Conducted (so far) 15 Study #1 2 Institutions * 10 Subspecialties 1 Fellows 2 Residents 8 Pathologists * Study #2 2 Institution * 10 Subspecialties 2 Fellows 6 Residents 10 Pathologists * * Same institutions and many of the same pathologists used in both studies.

16. Date Collected 16 Participant Demographics Task completion rates Time on Task User subjective difficulty ratings Verbal and written feedback Professional observations

17. How Did We Ensure Data Accuracy? 17 Every pathologist is unique from the way they navigate within a case and between slides to how they organize their day and the tools they use. Creating a realistic environment in order to obtain accurate data is much more difficult than imagined.  Everything was set up to be as realistic as possible  Datasets included participants real cases  Testing was conducted in participants normal working environment  Testing was customized for each participant to match their individual workflow – including other software and devices typically used.

18. Analyzing the Findings 18 Usability Severity Rating Scale 4 Critical usability problem. Could lead to patient care issues or seriously affect the adoption of our system. No workaround exists. 3 Major usability problem. Could severely affect Pathologist efficiency and workaround is cumbersome. 2 Minor usability problem. Could moderately affect Pathologist efficiency, but easy workaround exists. 1Cosmetic or training issue Frequency Difficulty to Overcome Persistence Market Impact

19. The Results 19 46 Enhancements 2 Studies 2 Institutions 10 Subspecialties 3 Fellows 8 Residents 10 Pathologists

20. “I don’t have time to learn a new system.” “Computers make things complicated.” “It doesn’t accommodate my workflow.” “I can’t afford to be slowed down.” “I like my microscope.” “I need to be able to trust the system.” Before UCD 20 What are they saying now? “I went through a series of prostate cores and found that using the this system was faster than with a microscope.” “Faster, workflow is smoother than with glass. Toggling back and forth between slides is much easier.” “Capturing images for the tumor board is much more efficient than using a camera connected to the microscope.” “If I was trying to do the same case digital with another system, it would have been frustrating.” “Thrilled with the way this works, it’s fantastic.” “I think it’s brilliant to be quite honest.” After UCD

21. Q&A 21