1. 1 “Practical usability testing of a molecular diagnostic system” “it’s fun” Romanian Testing Conference 2015! May 2015 Patrick Duisters 1.0

2. Improve Quality Services B.V., The Netherlands2 Patrick Duisters ●>15 years in Software Testing & Quality Assurance ●Administrative, Technical, Medical, Financial, Governmental, Automotive ●Test Consultant, Trainer, Auditor ●Certification & accreditation: Full CTEL ● patrick.duisters@improveqs.nl @PatrickDuisters www.improveqs.nlpatrick.duisters@improveqs.nl

3. 3 Experiences & Observation

4. 4 Agenda ●Context ●Usability & Standards ●Usability Program ●Deployment ●Results & Conclusion

5. 5 Context: Product… Molecular Diagnostics Platform: IVD: e.g. medical DNA analysis ●‘Sample in – Result out’ ●Reduced lead time ●Innovative & totally new ●EU & US ●Verification & Validation in a regulated environment

6. ●Medical, EU & US ●Regulations & Regulatory bodies: i.e. FDA ●Risk management mandatory ●Norms & standards: −ISO14971: MDs - Application of Risk Management to Medical Devices −ISO 62304: Medical device software - Software life cycle processes −ISO 62366: Medical devices – Application of usability engineering to medical devices 6 Context: Environment…

7. What is Usability? Usability: characteristic of the USER INTERFACE that establishes EFFECTIVENESS, EFFICIENCY, EASE of USER LEARINING and user SATISFACTION (ISO 62366:2007)

8. ●User Friendliness ●Learnability ●Error Prevention ●Iterative Development & Testing 8 Elements of ISO 62366

9. Usability Program ●Reviews & Testing ●Usability Testing & survey ●Observations & survey 9 Development Verification Validation

10. Deployment: Development 10 ●Heuristic Analysis ●Exploratory Testing

11. Development: Heuristic Analysis 11 ●Kick-off ●Logging template ●Analysis ●Start early / include! http://www.nngroup.com/articles/ten-usability-heuristics/

12. Development: Exploratory Testing 12 ●4 sessions ●Charters ●Notes ●Video recordings ●Promissing results

13. Deployment: Verification 13

14. Verification: Use Cases 14 ●User centered ●Product Life Cycle ●Primary operating functions ●Frequently used functions ●Use cases - Basic - Alternatives - Exceptions

15. Verification: Preparation 15 ●Test environment ●Test procedures: level of detail ●Questionnaire (SUMI based) ●User selection

16. Verification: Execution 16 ●Introduction ●“User training” ●Test procedures ●Questionnaire (SUMI based) ●Debrief ●Results ●Reporting

17. Deployment: Validation 17 1

18. ●Subjective vs. Objective ●Expected results & Findings: −Verification −User Friendliness −Learnability −Error Prevention ●Standards can be helpful, e.g. treat as heuristics ●Proof required? Practical solutions ●It’s FUN! 18 Results & Conclusion

19. 19 Questions pdu@improveqs.nl