1. Matthew Roberts, Vanessa Bacal, Mohammed Mahdi, Ethan D. Grober Mount Sinai & Women’s College Hospital, Division of Urology, Department of Surgery, University of Toronto Intra-Operative Assessment of Technical Skill on Live Patients Using Economy of Hand Motion: Establishing Learning Curves of Surgical Competence

2. Technical Skill Acquired Objectively, reliably assessed

3. OSATS: Objective Structured Assessment of Technical Skill  Discriminate novice & experienced surgeons  Improvement with training over time  Skills sets transfer to higher fidelity simulations

4. Intra-Operative Assessment of Technical Skill - Real Patients, Real Operating Rooms - Subjective, Unvalidated

5. ? Variable patients Variable operations Variable instruments & technology OR team changes Resource & time pressures Measurement instruments

6. Hand-Motion Analysis  Electro-magnetic measurement of hand position & orientation  # of hand movements  hand travel distance  hand speed

7. Hand-Motion Analysis  In the surgical skills lab: Objective, reliable and valid measure of surgical competence on open, laparoscopic, endoscopic and microsurgical procedures  Correlates well with traditional lab-based measures of surgical competence – global ratings and checklists scores by trained experts  Pilot work and feasibility studies have been performed in real operating rooms

8. Hand-Motion Analysis

10. Study Objectives 1.Establish the feasibility of performing live intra-operative hand-motion analysis while operating on real patients 2.Validate live, intra-operative assessments of economy of hand motion as an objective measure of technical skill 3.Use hand-motion analysis to establish competency-based surgical learning curves based on standards established by performance of experienced surgeons

11. Methods Novice Experienced 2 Surgeons 2 Standardized Surgical Tasks Video recorded Hand motion analysis Vasectomy Microsurgery-Vasectomy reversal

12. 2 blinded, experienced surgeons  Global rating scores  Checklists scores  Final product scores Methods – Video Analysis

15. Hand motion data and blinded expert ratings of video-based surgical performance were:   Graphically compared over time   Correlated using Pearson calculations Methods – Data Analysis NoviceExperienced

16. Total Hand Movements # Movements Case # Experienced Surgeon Novice Surgeon Experienced Surgeon Novice Surgeon

17. Hand Travel Distances Experienced Surgeon Novice Surgeon Case # Hand Travel Distance (m) Experienced Surgeon Novice Surgeon

18. Global Rating Scores Experienced Surgeon Novice Surgeon Global Rating Score Case # Experienced Surgeon Novice Surgeon

19. Checklist Scores Experienced Surgeon Novice Surgeon Checklist Score Case # Experienced Surgeon Novice Surgeon

20. Final Product Scores Case # Final Product Score Experienced Surgeon Novice Surgeon Experienced Surgeon Novice Surgeon

21. Correlating Hand Motion & Video Performance # Hand movements Hand Travel Distance Time Global Rating Score r = -0.83-0.85-0.86 Checklist Score -0.78-0.81-0.86 Final Product Score -0.82-0.87-0.83 Pearson correlation significant < 0.05 level (2-tailed) # Hand movements Hand Travel Distance Time Global Rating Score -0.94-0.95-0.92 Checklist Score -0.88-0.83-0.78 Final Product Score -0.90-0.85-0.70

22. BADBETTER

23. Set up < 3minsSet up < 3mins Data obtained in 23/25 casesData obtained in 23/25 cases Surgical team easily trainedSurgical team easily trained HMA software and hardware $3500HMA software and hardware $3500 Hand-Motion Analysis: Feasibility

24. Conclusions 1.Establish the feasibility of performing live intra-operative hand-motion analysis while operating on real patients 2.Validate live, intra-operative assessments of economy of hand motion as an objective measure of technical skill 3.Use hand-motion analysis to establish competency-based surgical learning curves based on standards established by performance of experienced surgeons

25. Utility of Hand Motion Analysis # Movements Case # Experienced Surgeon Novice Surgeon 1) Measure & track intra-operative surgical performance over time 2) Tool for immediate feedback to surgical trainees and training programs 3) To establish that different procedures competency targets. HMA can be used to identify the ideal or minimum number of exposures required for technical competence 15

26. Economy of Hand Motion - Future Directions - Further validate this assessment technology:  More surgeons - with various levels of skill & experience  Different types surgeries - basic/complex, open/laparoscopic/endoscopic

27. Intra-Operative Assessment of Technical Skill - Real Patients, Real Operating Rooms -

28. Matthew Roberts, Vanessa Bacal, Mohammen Mahdi, Ethan D. Grober Mount Sinai & Women’s College Hospital, Division of Urology, Department of Surgery, University of Toronto Intra-Operative Assessment of Technical Skill on Live Patients Using Economy of Hand Motion: Establishing Learning Curves of Surgical Competence