1. Assessing the Teaching of Procedural Skills: Can Cognitive- Task-Analysis Add to our Traditional Teaching Methods? Maura Sullivan, PhD, Adrian Ortega, MD, Nir Wasserberg, MD, Howard Kaufman, MD, Julie Nyquist, PhD, Richard Clark, EdD University of Southern California

2. Cognitive-Task-Analysis (CTA) Extends traditional task analysis to capture information about both the overt observable behavior and the covert cognitive functions behind it to form an integrated whole. Schraagen, Chipman & Shalin, 2000

3. Acquiring Expertise Declarative vs. Procedural Knowledge Cognitive Associative Autonomous

4. Expertise  Expertise is highly automated Ericsson, et al. 2006; Gagne et al. 1993; Clark & Estes, 1996  Automaticity interferes with articulation Clark et al., 2007, Feldon, 2004, Crandall & Getchell-Reitter, 1993, Chao & Salvendy, 1994  Learners need to be taught decision making strategies Ericsson et al., 2006; Clark et al. 2007

5. Expertise cont.  Experts underestimate novice difficulty Hinds, 1999  Experts not fully aware of 70% of own decisions Clark & Elen, 2006; Feldon & Clark, 2006  Experts’ decisions can be put into training materials Merrill, 2002; Schaafstal et al. 2000

6. CTA Gains  Overt observable behavior Deconstruct automated knowledge into concrete steps  Covert cognitive functions Identification of decision points throughout procedure Options related to each decision point  Ability to gain consensus amongst experts Provide residents with an advanced organizer

7. Purpose  To determine if surgeons omit relevant steps and decision points when teaching a colonoscopy  To determine if CTA can augment our traditional teaching methods

8. Methods  Three experts videotaped teaching a colonoscopy followed by “free recall”  Participated in CTA  Procedural Checklist and Cognitive Demands Checklist created  Videotape transcriptions transposed and omitted steps identified

9. Results “What” to do 26-step Procedure 14 Decision Points Surgeon A61%43% Surgeon B46%25% Surgeon C50%38%

10. Results cont. 26-step Procedure 14 Decision Points Surgeon A61% (50%)43% Surgeon B46% (30%)25% Surgeon C50% (26%)38% “What” to do “How to” do it

11. Conclusions  Experts did not articulate all relevant steps while teaching a colonoscopy or during free recall  CTA provides a means to capture omitted steps and decision points during traditional teaching

12. Limitations  Small N # of experts # of cases observed  No established “Gold Standard”  No inter-coder reliability

13. Future Studies  Quantify educational effectiveness  Determine number of experts needed  Determine number of cases required  Establish reliability  Accelerate the learning curve

14. Implications  CTA can be used to develop more comprehensive curricula for teaching procedural skills  Assures exposure to all steps/decisions involved in a procedure  More complete training of residents

15. mesulliv@usc.edu (323) 442-2368