Cognitive Task Analysis Kenneth Yates, Ed.D. Center for Cognitive Technology Rossier School of Education University of Southern California NCOA Leadership Workshop The USC Institute for Creative Technologies January 2010

2 Training Goal The Mission of the NCO Academy is to conduct tough and demanding primary, basic and advanced level courses which develop adaptive, self- confident leaders who motivate teams to solve complex problems, while operating in a full spectrum and contemporary operating environment in an army transforming into the future force. NCO Academy Ft. Knox

3 Topics  What is Cognitive Task Analysis (CTA)?  A brief demonstration  CTA applications  Expertise and Training  An overview of the 7 steps of CTA and examples  Arguments about CTA

4 Cognitive - Task - Analysis ( CTA ) Extends traditional task analysis to capture information about both the overt observable behavior and the covert cognitive functions behind it to form an integrated whole. Schraagen, Chipman & Shalin, 2000

5 A brief exercise in CTA Repeated decisions under varied conditions become automated and unconscious ? ?

6 Some Historical Perspective  Early 1900s: Frank and Lillian Gilbreth –Time and motion studies – Created “therbligs” (Gilbreth spelled backwards) –Decision-making and planning kept to a minimum  Mid-1900s: Emerging technology and cognitive psychology –Computers and expert systems –Troubleshooting  Late 1900s: Complex Problem Solving –Capturing expertise at work –Human Performance Technology

7  Aviation  Air Traffic Control  Military Commanders  System Development  Organizational Design  Product Design  Decision Support Systems  Systems Engineering  Human Computer Interfaces  Human factors  Software Design  Training Teams  Market Research  Ergonomics  Cybernetics  Medicine CTA Applications

8 CTA in Military Applications  Troubleshooting weapons systems –Schaafstal, Schraagen, & van Berlo (2000)  Stability and Support Operations – Pfautz & Roth (2006)  Decision-making in weapons systems under high-stress conditions –Klein, Kaempt, Wolf, Thorsden (1997)

9 The Training Dilemma

10 Expertise  Expertise is highly automated  Automaticity interferes with articulation  Experts not fully aware of 70% of own decisions  Experts underestimate novice difficulty

11 Transferring Expertise in Training  How can we optimize task analysis?  Capture experts’ decisions and include in training materials  Reduce the amount of information that does not support performance and adaptability SSG Tommy RichardsonSPC Jordan Lackie 39 th Support Battalion

12 Overview - The Seven Steps of CTA 1. Select SMEs with recent experience –Recent (within 3 months) field performance of task –Established track record of success  Solved task problems that novices cannot solve –Broad experience performing task in many different settings –Has not served as a trainer for the task –Use 2 to 3 SMEs  Cross-checking with other SMEs increases accuracy and completeness by 50 % or more (from 30% with 1 SME to 70% or more with 3 SMEs)

13 Overview - The Seven Steps of CTA 2. Outline sequence of tasks “as performed in field” –Sequence as “before, during, after” - then compare with doctrine –For each task, describe clearly enough so that trainee can apply  Field conditions (Where)  Cues (What Starts the task)  Sequence of Actions and Decisions (How)

14 Task title: Prepare a Situation Map, Prepare an Operations Overlay Expert : SSGT Artis Alexander, Ft. Carson, CO Procedures 1. Create and update sectors (basic SITMAP) 2. Track past and incoming SIGACTs 3. Post, activate, and deactivate current missions 4. Perform shift change procedures

15 Major Procedures – COIN Ops 1.Conduct Mission Analysis. 2.Conduct deliberate reconnaissance. 3.Review and approve targeting-synchronization matrix 4.Assess progress toward achieving overarching outcomes

16 Detecting IED on Mounted Patrol

17  Explain actions –Things people do (start with action verbs)  Explain decisions –Describe as “IF” and “THEN” sentences  MOST IMPORTANT: Write steps clearly enough so that a trainee could do what you are describing. Overview - The Seven Steps of CTA 3. Capture Actions and Decisions for each task

18 Step 1 (Action) Log on to computer system. Load Map Man software application and toolbars (Drawing, Copy Machine, Edit mode, and Task Organization Chart) and Load the 2D map Step 2: (Action) Center map, zoom on section of interest, and set as a Preset Map View Step 3: (Decision) Receive printed map from Battle MAJ or CPT - Review the printed map and compare it with existing overlays in the CPOF IF you find a similar overlay, THEN drag it into Copy Machine IF you cannot find an overlay that is similar OR if you’ve dragged all similar overlays into the Copy Machine, THEN go to Step 4 Name of Procedure: Create and update sectors (Basic SITMAP) Cue: BSNCO instructed to create SITMAP by Battle MAJ or Battle CPT

19 ELECT - UrbanSim Determine Threat Courses of Action Define the Battlefield Environment Evaluate the Threat Describe the Environment’s Effects Direct Instruction Part-task ‘how to” Demo and Practice UrbanSim Whole Task Varied Problem Practice

20 MCIT (IED)

21 Overview – The Seven Steps of CTA 4. Collect task-related information about:  Supplies and equipment  Performance standards (speed, quality)  Reasons (Benefits and Risks)  Sensory information (smell, taste, touch needs?) 5. Identify conceptual knowledge related to procedure:  Concepts (new terms, locations, ideas)  Processes (how things work)  Principles (what causes things to happen)

22 Overview – The Seven Steps of CTA 6. Collect five field problems trainees will learn to solve  One for demonstration during training  One for practice and feedback  One for progress check  Two for competency test 7. Take CTA document from SME A and give to SME B to “correct”  Develop a “gold standard” CTA for training

23 CTA Gains  Overt observable behavior –Deconstruct automated knowledge into concrete steps  Covert cognitive functions –Identification of decision points throughout procedure –Options related to each decision point  Development of curricular content  Increase learning curve and accelerate the acquisition of expertise  Ability to gain consensus amongst experts

24 What problems will CTA not solve ? IF :  No experts available and/or  Completely new (novel) equipment or  New processes and doctrine or  If “experts” not succeeding at task THEN: We use traditional analysis techniques

25 CTA Counter Arguments 1. Procedural training will not result in flexible, adaptable knowledge and skills 2. CTA will increase development time when we are trying to reduce it.

26 Responses to Counter Arguments 1. How do we train for flexible, adaptable knowledge and skills?  Must first learn how before becoming flexible  Flexible adaptation results from “varied practice” –We collect problems of increasing difficulty and novelty for practice and tests  Novel application requires conceptual knowledge that is connected to skills –We teach all necessary conceptual knowledge but connect it to each procedure

27 CTA Counter Arguments 2. CTA will increase development time when we are trying to reduce it –CTA increases design time by about 20% –But it decreases training time by 20 – 50%  By lowering cognitive load on trainee through combining theory and practice –If IMI is being developed, CTA permits rapid development

28 Summary Training “best practices” 1. Conceptual + Procedural knowledge from CTA PLUS 2. How to demo PLUS 3. Practice & Feedback solving increasingly complex problems in immersive environments 4. Produces flexible and adaptive leaders.

29 Questions? Comments?

Cognitive Task Analysis Kenneth Yates, Ed.D. Center for Cognitive Technology Rossier School of Education University of Southern California Questions, Comments, and References Thank you!