Improving Procedural Teaching by Understanding Features Associated with Cognitive Load Justin L. Sewell MD, MPH Christy K. Boscardin, PhD John Q. Young, MD, MPP Olle ten Cate, PhD Patricia S. O’Sullivan, EdD

Procedural skills training is a balancing act! Learners Teachers, patients, system

Learners Autonomy Feedback Deliberate practice Case volume

Case volume Teachers, patients, system Safety Quality Efficiency Patient-provider relationship Efficiency Quality Safety

Learners Teachers, patients, system Patient-provider relationship Efficiency Quality Safety Autonomy Feedback Deliberate practice Case volume

Procedural skills training in HPE Complexity of cognitive and psychomotor demands  high risk of cognitive overload HOWEVER, empirical evidence for contributors to cognitive load during procedural skills training is limited Better understanding of such contributors could lead to: Curricular innovations  Improved learning  Better patient outcomes???

Capacity for sensory input Cognitive Load Theory Capacity for sensory input Long-term memory Working memory ∞ ∞ Sweller J. Cogn Sci 1988. Figure adapted from Young JQ. Med Teach 2014.

Types of cognitive load Intrinsic load Task complexity Learner knowledge and experience Optimize Extraneous load Learning environment Instructional design Minimize Learner effort and metacognitive skills Germane load Maximize

Cognitive load & working memory Working memory overloaded – no space for germane load Extraneous load Intrinsic load Space available for germane load Extraneous load Intrinsic load Germane load Adapted from Young JQ. Med Teach 2014.

Colonoscopy Complex and requires >300 cases to attain competence Preferred method for colorectal cancer screening “Middle of the road procedure”  implications for spectrum of procedural skills

What affects cognitive load during procedural skills training?

Methods Post-colonoscopy survey distributed to 1,061 fellows in 2014-15 academic year Outcome: three cognitive load types Predictors: procedural learning features Anticipated relationships predicted between each feature and each type of cognitive load Multivariable linear regression model developed for each type of cognitive load

Outcome: cognitive load types Cognitive Load Inventory for Colonoscopy (CLIC) Estimate of IL, EL, GL Germane items Intrinsic items Extraneous items Psychometric instrument Multiple items averaged to estimate degree of intrinsic, extraneous and germane load Validity evidence presented Self-report instrument administered post-colonoscopy 0-10 scale, strongly disagree  strongly agree Sewell JL et al. Med Educ 2016;50(6):682-92, AERA 2016

Also supervisor takeover Predictors Year in training Prior experience Sleep Fatigue Cognitive load Learner Patient/ task Setting Super-visor Prep quality Tolerance Anesthesia Gender No. of maneuvers Also supervisor takeover Junior versus senior Engagement Confidence Queue order No. people in room On call Paged

Results – response rate 477 (45.0%) of 1,061 invited fellows participated 154 (95.1%) of 162 programs represented

Number of fellows per year in training

Prior colonoscopy experience

Intrinsic load model Feature Category Coefficient (95% CI) P-value Year 2 fellow (vs year 1) Learner -0.60 (-1.09,-0.14) -0.17 0.01 Year 3 or 4 fellow (vs year 1) -0.82 (-1.38,-0.26) -0.23 0.004 Prior colonoscopy experience -0.20 (-0.31,-0.09) -0.24 <0.001 Fatigue 0.01 (0.008,0.02) 0.21 Tolerated procedure well Patient/task -0.67 (-1.01-0.33) -0.16 No. ancillary maneuvers 0.31 (0.18,0.44) 0.19 Supervisor took over Multiple 0.51 (0.24,0.77) 0.16 Hours of sleep 0.11 (-0.07,0.29) 0.05 0.22 Good bowel prep (vs excellent) 0.14 (-0.17,0.44) 0.04 0.38 Fair or poor bowel prep (vs excellent) 0.17 (-0.26,0.59) 0.44

Extraneous load model Feature Category Coefficient (95% CI) P-value Fatigue Learner 0.01 (0.005,0.02) 0.18 0.001 Queue order Setting 0.07 (0.01,0.13) 0.10 0.03 Somewhat or very disengaged (vs very engaged) Supervisor 0.33 (0.02,0.64) 0.11 0.04 Less than very confident 0.49 (0.13,0.86) 0.13 0.009 Supervisor took over Multiple 0.86 (0.49,1.23) 0.24 <0.001 Year 2 fellow (vs year 1) -0.36 (-0.84,0.12) -0.12 0.14 Year 3 or 4 fellow (vs year 1) -0.32 (-0.89,0.26) -0.10 0.28 Prior colonoscopy experience -0.04 (-0.16,0.07) -0.06 0.46 Hours of sleep 0.17 (-0.01,0.35) 0.07 No. people in room 0.04 (-0.09,0.18) 0.52 Paged 0.16 (-0.19,0.51) 0.37 On call -0.23 (-0.68,0.21) -0.05 0.30 Somewhat engaged (vs very engaged) 0.45 (-0.03, 0.94) Neither engaged nor disengaged (vs very engaged) 0.15 (-0.34,0.64) 0.55

Germane load model Feature Category Coefficient (95% CI) β-coefficient P-value Somewhat engaged (vs very engaged) Supervisor -0.91 (-1.83,0.001) -0.10 0.05 Neither engaged nor disengaged (vs very engaged) -1.33 (-2.24,-0.42) -0.15 0.004 Somewhat or very disengaged (vs very engaged) -0.85 (-1.44,-0.26) 0.005 Year 2 fellow (vs year 1) Learner -0.30 (-1.19,0.60) -0.05 0.52 Year 3 or 4 fellow (vs year 1) -0.62 (-1.70,0.46) -0.11 0.26 Prior colonoscopy experience -0.14 (-0.36,0.07) 0.19 Fatigue 0.01 (-0.005,0.02) 0.06 No. maneuvers performed Patient/task -0.06 (-0.32,0.19) -0.02 0.62 Paged Setting 0.28 (-0.36,0.98) 0.04 0.39 On call -0.25 (-1.08,0.58) -0.03 0.55 Queue order 0.03 (-0.09,0.14) 0.02 0.65 Less than very confident 0.58 (-0.10,1.27) 0.08 0.10 Supervisor took over Multiple -0.43 (-1.16,0.30) -0.06 0.25 Intrinsic load --- 0.28 (0.08,0.48) 0.17 0.006 Extraneous load 0.25 (0.60,0.45) 0.14 0.01

In other words… Intrinsic load Extraneous load Germane load : fatigue, number of maneuvers, supervisor took over : year in training, prior colonoscopy experience, good patient tolerance Extraneous load : fatigue, queue order, supervisor takeover : more engaged supervisor, more confident supervisor Germane load : more engaged supervisor, intrinsic load, extraneous load

Implications for instructional design To optimize IL: Select (partial) task for learner’s competence and prior experience Take over quickly when early learner is struggling To minimize EL: Monitor for fatigue Consider target number of procedures per session Remain engaged and communicate confidence To maximize GL: Remain engaged with learner and procedure Adjust support based on learner’s experience and procedural complexity

Limitations Some features specific to colonoscopy CLIC measures learner perceptions of cognitive load No measures of learning Associations, not causation

Summary Learners Teachers, patients, system Patient-provider relationship Efficiency Quality Safety Feedback Autonomy Deliberate practice Case volume Different sets of features are associated with the three types of cognitive load among colonoscopy learners Classes of features studied are generalizable and adaptable Findings can inform future education interventions to help balance the scales during procedural skills training

justin.sewell@ucsf.edu