Matthew Roberts, Vanessa Bacal, Mohammed Mahdi, Ethan D. Grober Mount Sinai & Women’s College Hospital, Division of Urology, Department of Surgery, University of Toronto Intra-Operative Assessment of Technical Skill on Live Patients Using Economy of Hand Motion: Establishing Learning Curves of Surgical Competence

Technical Skill Acquired Objectively, reliably assessed

OSATS: Objective Structured Assessment of Technical Skill  Discriminate novice & experienced surgeons  Improvement with training over time  Skills sets transfer to higher fidelity simulations

Intra-Operative Assessment of Technical Skill - Real Patients, Real Operating Rooms - Subjective, Unvalidated

? Variable patients Variable operations Variable instruments & technology OR team changes Resource & time pressures Measurement instruments

Hand-Motion Analysis  Electro-magnetic measurement of hand position & orientation  # of hand movements  hand travel distance  hand speed

Hand-Motion Analysis  In the surgical skills lab: Objective, reliable and valid measure of surgical competence on open, laparoscopic, endoscopic and microsurgical procedures  Correlates well with traditional lab-based measures of surgical competence – global ratings and checklists scores by trained experts  Pilot work and feasibility studies have been performed in real operating rooms

Hand-Motion Analysis

Study Objectives 1.Establish the feasibility of performing live intra-operative hand-motion analysis while operating on real patients 2.Validate live, intra-operative assessments of economy of hand motion as an objective measure of technical skill 3.Use hand-motion analysis to establish competency-based surgical learning curves based on standards established by performance of experienced surgeons

Methods Novice Experienced 2 Surgeons 2 Standardized Surgical Tasks Video recorded Hand motion analysis Vasectomy Microsurgery-Vasectomy reversal

2 blinded, experienced surgeons  Global rating scores  Checklists scores  Final product scores Methods – Video Analysis

Hand motion data and blinded expert ratings of video-based surgical performance were:   Graphically compared over time   Correlated using Pearson calculations Methods – Data Analysis NoviceExperienced

Total Hand Movements # Movements Case # Experienced Surgeon Novice Surgeon Experienced Surgeon Novice Surgeon

Hand Travel Distances Experienced Surgeon Novice Surgeon Case # Hand Travel Distance (m) Experienced Surgeon Novice Surgeon

Global Rating Scores Experienced Surgeon Novice Surgeon Global Rating Score Case # Experienced Surgeon Novice Surgeon

Checklist Scores Experienced Surgeon Novice Surgeon Checklist Score Case # Experienced Surgeon Novice Surgeon

Final Product Scores Case # Final Product Score Experienced Surgeon Novice Surgeon Experienced Surgeon Novice Surgeon

Correlating Hand Motion & Video Performance # Hand movements Hand Travel Distance Time Global Rating Score r = Checklist Score Final Product Score Pearson correlation significant < 0.05 level (2-tailed) # Hand movements Hand Travel Distance Time Global Rating Score Checklist Score Final Product Score

BADBETTER

Set up < 3minsSet up < 3mins Data obtained in 23/25 casesData obtained in 23/25 cases Surgical team easily trainedSurgical team easily trained HMA software and hardware $3500HMA software and hardware $3500 Hand-Motion Analysis: Feasibility

Conclusions 1.Establish the feasibility of performing live intra-operative hand-motion analysis while operating on real patients 2.Validate live, intra-operative assessments of economy of hand motion as an objective measure of technical skill 3.Use hand-motion analysis to establish competency-based surgical learning curves based on standards established by performance of experienced surgeons

Utility of Hand Motion Analysis # Movements Case # Experienced Surgeon Novice Surgeon 1) Measure & track intra-operative surgical performance over time 2) Tool for immediate feedback to surgical trainees and training programs 3) To establish that different procedures competency targets. HMA can be used to identify the ideal or minimum number of exposures required for technical competence 15

Economy of Hand Motion - Future Directions - Further validate this assessment technology:  More surgeons - with various levels of skill & experience  Different types surgeries - basic/complex, open/laparoscopic/endoscopic

Intra-Operative Assessment of Technical Skill - Real Patients, Real Operating Rooms -

Matthew Roberts, Vanessa Bacal, Mohammen Mahdi, Ethan D. Grober Mount Sinai & Women’s College Hospital, Division of Urology, Department of Surgery, University of Toronto Intra-Operative Assessment of Technical Skill on Live Patients Using Economy of Hand Motion: Establishing Learning Curves of Surgical Competence