Standardizing cataract surgery rating between resident and attending ophthalmologists: An educational intervention Nolan, Michael 1 ; Pittner, Andrew 1 ; McGaghie, William 3 ; Hill, Geoffrey 2 ; Dwarakanathan, Surrendar 2 ; Feder, Robert 4 ; Farooq, Asim V. 5 ; Traish, Aisha 5 ; Bouchard, Charles S. 1 1.Department of Ophthalmology, Loyola University Medical Center, Maywood, IL. 2. Leischner Institute for Medical Education, Loyola University Stritch School of Medicine, Maywood, IL. 3. Division of Ophthalmology, John H. Stroger, Jr. Hospital of Cook County, Chicago, IL. 4. Department of Ophthalmology, Northwestern University, Chicago, IL. 5. Department of Ophthalmology and Visual Sciences, University of Illinois at Chicago, Chicago, IL. Purpose Methods Results Proficiency in cataract surgery is an important goal for ophthalmology residents. We performed a collaborative multi- institution study adapting the assessment rubric from the International Committee on Ophthalmology Surgical Competency. We selected 5 of the 14 steps of the rubric to focus the grading process in this initial phase of the study: 1) commencement of the capsulorrhexis flap; 2) formation of the capsulorrhexis and circular completion; 3) hydrodissection; 4) irrigation and aspiration; and 5) lens insertion. The goal of this study was to create a standardized database of short video clips demonstrating varying surgical skill levels and use them to train resident and attending ophthalmologists to become expert graders of surgical skill. The authors collected video clips from Loyola University Medical Center, University of Illinois at Chicago, Northwestern University, and John H. Stroger, Jr. Hospital of Cook County. Through consensus among the authors, clips were chosen for a reference database clearly demonstrating 4 different skill levels ranging from novice, beginner, advanced beginner, to competent. Nine PGY-3 and eight PGY-4 ophthalmology residents, as well as five attending ophthalmologists from the 4 participating residency programs volunteered to take part in the study. The volunteers attended training sessions where the standardized video clips were viewed and discussed. The volunteers then took a rater competency test to assess their ability to rate the surgical skill level in different surgical video clips using the rubric. All volunteers completed four pre- training and four post-training online video assessments.. Study volunteers from 4 Chicago ophthalmology residency programs completed training to become “expert raters” by applying the grading rubric to surgical video clips and reaching the same consensus as the authors. The volunteers' responses in rating cataract surgery video clips were highly dispersed before the training session and became more homogeneous after the educational intervention. This study suggests that learning to assess cataract surgery skill level through a structured program improves residents' ability to recognize safe and effective surgical technique. This educational intervention serves a foundation for improving resident surgical training. Accreditation Council for Graduate Medical Education. Educating Physicians for the 21st Century. Systems-Based Practice, Available at: Chung AKK, Gauba V, Saleh GM. Assessing resident competency [letter]. Ophthalmology 2007; 114:1587–1588 Henderson BA, Rasha A. Teaching and assessing competence in cataract surgery. Curr Opin Ophthalmol 2007; 18:27–31 Cremers SL, Ciolino JB, Ferrufino-Ponce ZK, Henderson BA. Objective Assessment of Skills in Intraocular Surgery (OASIS). Ophthalmology. 2005;112: Cremers SL, Lora AN, Ferrufino-Ponce ZK. Global Rating Assessment of Skills in Intraocular Surgery (GRASIS). Ophthalmology 2005; 112:1655– 1660 Saleh GM, et al. Objective Structured Assessment of Cataract Surgical Skill. Arch Ophthalmol 2007;125: Golnik KC, Beaver H, Gauba V, Lee AG, Mayorga E, Palis G, Saleh GM. Cataract surgical skill assessment [letter]. Ophthalmology 2011; 118:427– 427 Figure 1. Screen shot of Moodle. A password protected online quiz was created for study participants to rate individual steps of cataract surgery. Participants were able to watch video clips and select choices from the grading rubric on the same screen Capsulorrhexis: commencement Capsulorrhexis: completion HydrodissectionIrrigation and Aspiration Lens insertion and positioning All steps Pre- training Post- training Pre- training Post- training Pre- training Post- training Pre- training Post- training Pre- training Post- training Pre- training Post- training All participants (n=21) Attending (n=5) PGY-4 (n=8) PGY-3 (n=8) Table 1. Mean standard deviations for the online quiz responses Capsulorrhexis: creation of flap and follow through 1 - novice1) Instrument use and control: difficulty entering eye, tentative movements 2) Globe Position: not well centered, red reflex frequently lost 3) Time Efficiency: long duration 4) Quality-Outcome: inadequate flap size or location cortex disruption 2 - beginner1) Instrument use and control: enters eye easily, mostly purposeful movements 2) Globe Position: works to maintain centered red rellex 3) Time Efficiency: few inefficient movements, moderate duration 4) Quality-Outcome: adequate flap created minimal cortex disruption 3 - advanced beginner 1) Instrument use and control: few awkward or repositioning movements 2) Globe Position: red reflex centered most of the time 3) Time Efficiency: short duration 4) Quality-Outcome: appropriate flap size and location 4 – competent1) Instrument use and control: smooth fluid movements 2) Globe Position: well centered, minimal movement 3) Time Efficiency: quick duration 4) Quality-Outcome: ideal flap size and location – sets up easy rrhexis completion Capsulorrhexis: formation and circular completion 1 – novice 1) Instrument use and control: ineffective or inappropriate regrabbing 2) Globe Position: pushing globe and frequent loss of red reflex 3) Time Efficiency: long duration 4) Quality-Outcome: poor rrhexis size or shape, may have tear out 2 - beginner 1) Instrument use and control: 2) Globe Position: frequent loss of red reflex but works to maintain centered red rellex 3) Time Efficiency: slow but methodical completion 4) Quality-Outcome: size, shape and position are barely adequate. Any tears are minimal 3 - advanced beginner 1) Instrument use and control: appropriate regrasps with minimal difficulty 2) Globe Position: red reflex centered most of the time 3) Time Efficiency: completed smoothly in a moderate amount of time 4) Quality-Outcome: appropriate size and location 4 – competent1) Instrument use and control: appropriate regrasps with minimal difficulty 2) Globe Position: red reflex centered most of the time 3) Time Efficiency: completed smoothly and efficiently 4) Quality-Outcome: appropriate size and location Intraocular Lens insertion and placement 1 - novice1) Instrument use and control: poor control, repeated attempts, lot of movement 2) Globe Position: much movement, poor control 3) Time Efficiency: slow, 4) Quality-Outcome: fair, poor IOL placement with corneal damage 2 - beginner1) Instrument use and control: Excessive manipulation to get into the bag, 2) Globe Position: fair-good 3) Time Efficiency: fair 4) Quality-Outcome: adequate placement and IOL rotation into the bag but may have needed too much manipulation 3 - advanced beginner 1) Instrument use and control: good control and insertion, good IOL placement I the bag 2) Globe Position: fairly good centration 3) Time-Efficiency: fair-good 4) Quality-Outcome: good, may have contacted optic unnecessarily while repositioning 4 – competent1) Instrument use and control: good control, minimal corneal distruption 2) Globe Position: stable 3) Time-Efficiency: quick and easily inserted 4) Quality-Outcome: IOL safely in the bag, minimal wound disruption Irrigation and aspiration 1 - novice1) Instrument use and control: difficulty entering eye with handpiece, Fails to recognize importance of occluding tip, Grabbing capsule often 2) Globe position: poor with significant movement and poor visualization 3) Time-Efficiency: slow 4) Quality-Outcome: less than adequate, unable to peel cortical material, risky maneuvers or positioning 2 - beginner1) Instrument use and control: Only occluding sometimes, may engage capsule sometimes 2) Globe Position: A lot of globe movement 3) Time-Efficiency: Slow - inefficient 4) Quality-Outcome: cortex removed adequately, fairly safe, may require many attempts 3 - advanced beginner 1) Instrument use and control: Occluded port most times, centripetal pealing, safely introduces aspiration tip in irrigation mode 2) Globe Position: good most of the time 3) Time-Efficiency: good, 2-3 minutes, 4) Quality-Outcome: removes all cortex 4 – competent1) Instrument use and control: Port properly occluded, Centripetal Peeling, Engaging cortex under the capsule 2) Globe Position: stable throughout 3) Time-Efficiency: Time efficient (~90-120s) 4) Quality-Outcome: superior, clear capsule, safely done with gentle tissue manipulation Hydrodissection 1 - novice1) Instrument use and control: rough handling of tissues, ineffective positioning of cannula applying lots of posterior pressure 2) Globe Position: globe movement and difficulty positioning cannula 3) Time Efficiency: long duration, multiple attempts 4) Quality-Outcome: no fluid wave, minimal nucleus rotation 2 - beginner1) Instrument use and control:multiple attempts required, applying posterior pressure, 2) Globe Position: mild movement of globe during cannula postioning 3) Time Efficiency: moderate duration 4) Quality-Outcome: some nucleus rotation, may still need to force rotation 3 - advanced beginner 1) Instrument use and control: Good fluid wave, decompression 2) Globe Position: Good position 3) Time Efficiency: short duration 4) Quality-Outcome: able to rotate nucleus but with some resistance 4 – competent1) Instrument use and control: immediate and appropriate fluid wave Good decompression 2) Globe Position: centered 3) Time Efficiency: quick 4) Quality-Outcome: nucleus rotates freely 45 degrees or more with minimal resistance A0391 Conclusion References