1 Research on the Design of Technology-Based Training Richard E. Clark Rossier School of Education Center for Cognitive Technology University of Southern California ICT Immersive Training Workshop May 19, 2005 Marina del Rey California

2 Topics for Today’s Workshop You will be able to answer five questions: 1.How does our cognitive architecture influence learning from training? 2.Do media influence learning? What are the six principles for effective media-based training? 3.What new and promising training design models are out there right now? 4.What motivates us to learn and how can we influence it during training? 5.How can we decide whether training should be delivered in the classroom or by multi media?

3 Have you ever? Spaced out when you were studying a chapter in a book? Had difficulty memorizing lists Noticed that you stopped listening to someone who was giving you information and missed some of it? Had trouble remembering when you were under pressure?

4 I. How does our cognitive architecture influence learning from training? 1.We have a very limited thinking space, easily overloaded to protect us from scrambling our brain ( 3 +/- 1 information chunks/thoughts at one time) –When overloaded, our brain short circuits and interferes with the mental resources needed to learn –Novices most vulnerable – experts insulated –Overload is not noticed – may be enjoyable? To overcome this limit we have two different but interacting knowledge systems –Declarative (what, why, where, when) –Procedural (how)

5 I. Cognitive Architecture 2.Declarative system is information rich and helps us handle novelty and develop new procedures to achieve goals –All new knowledge is declarative – about 10% –Knowledge stays flexible while it is being used to achieve our goals –Once it is successful it gradually becomes fixed, procedural, automated and unconscious – 90% Types: concepts, processes and principles Formats: Sensory Images, symbols

6 I. Cognitive Architecture 3.Procedural system operates outside of consciousness and so avoids 3 chunk limit on thinking –Goal of nearly all training is to teach procedures –Consist of conditions (when, where) actions (what to do) and decisions (alternatives and criteria for deciding) –Expertise 90% automated & unconscious –Experts 50% inaccurate so “telling them how it is for an expert” is not enough

7 I. Cognitive Architecture 4.Once we accept a new goal, our “learning system” is structured to select and/or develop a procedure for achieving it. –General learning procedures include: Plans to achieve the goal Connections to relevant prior knowledge, Ways to select the important and ignore the rest Self monitoring for accuracy and results feedback Procedures for using feedback to tune the plan

8 I. Cognitive Architecture 5.Our “learning system” requires external help to learn when information is relatively new –Effective help procedures include: Plans to achieve the goal (and clear goals) Connections to relevant prior knowledge (examples) Ways to select the important and ignore the rest Self monitoring for accuracy and results feedback Procedures for using feedback to tune the plan

9 I. Cognitive Architecture 5.Successful training provides clear goals and supports the learning strategies that trainees can’t or won’t provide for themselves. –Provide a clear “3C” goal (concrete, current, challenging), motivate them to accept it and persist at achieving it –Show them how to reach it with a “worked example” –Immerse them in a variety of realistic examples –Support their learning strategies as they develop their own version of a procedure

10 II. Do media influence learning? 1.Media influence the efficiency (cost, speed) of learning but not whether someone learns. –Learning is due to instructional strategies and the relevance of the information we provide. –More than one medium will result in comparable learning of any task 2.Instructional strategies influence learning. They are external representations of internal cognitive strategies needed to learn –Examples: Goals/objectives, learning plans, connections to prior experience (examples, analogies), practice exercises, progress evaluation, feedback

11 II. Do media influence learning? 3.Multi-media presentations often cause cognitive overload due to media features that split attention and require integration –Caused by background music, irrelevant sounds, animated “help” characters, separated video and print explanations, narration over print etc. 4.Goal is to lower cognitive load and focus attention only on essential, goal-relevant information in multi media programs How? Six Research Based Principles Next

12 Six Cognitive Principles about Media 1Contiguity principle. –Integrate all visual and aural information - use both together at the same time and in the same instructional space 2Multiple modalities principle. –Visual and sound modalities better than either alone – use both where possible

13 Spatial Contiguity Principle People learn better when corresponding words and graphics are placed near rather than far from each other on the screen. As a possible modification to the annotated illustrations, you might be tempted to separate the text and graphics. Your rationale might be that the learner can concentrate on the text and then concentrate on the graphics, and thus gain two exposures to the material. This strategy, however, violates the spatial contiguity principle. When the driver steps on the car’s brake pedal, a piston moves forward inside the master cylinder. The piston forces brake fluid out of the master cylinder and through the tubes to the wheel cylinders. In the wheel cylinders, the increase in fluid pressure makes a smaller set of pistons move. These smaller pistons activate the brake shoes. When the brake shoes press against the drum, both the drum and the wheel stop or slow down. This is a screenful of printed text describing how brakes work. See graphics Return When you click on “See graphics” you see these graphics depicting how brakes work. Problem: When graphics are separated from the text, learners have difficulty in making connections between corresponding words and graphics.

14 Modality Principle People learn better from animation and narration than from animation and on-screen text. A solution to the split attention problem is to present spoken words rather than printed words, thus reducing the load on the visual system. “When the driver steps on the car’s brake pedal, a piston moves forward inside the master cylinder.” “In the wheel cylinders, the increase in fluid pressure makes a smaller set of pistons move.” “The piston forces brake fluid out of the master cylinder and through the tubes to the wheel cylinders.” Each sentence is spoken at a slow pace as the animation shows what happens when the driver steps on the brake petal. The words are spoken rather than printed on the screen.

15 Six Cognitive Principles about Media 3Coherence Principle –Eliminate all extraneous words, sounds (e.g. music) and pictures to reduce load –If information is not essential to achieve the goal, drop it or risk distracting people 4Narration principle –Narration is better than text in media displays – people listen more effectively than they read. Natural voice narration is better than machine voice.

16 Coherence Principle People learn better from multimedia messages when extra words, pictures, and sounds are excluded rather than included. It may be tempting to embellish a presentation by adding background sounds or music. However, the added materials can distract the learner and disrupt the learner’s building of a causal chain. “When you step on your car’s brake pedal, a piston moves forward inside your master cylinder.” “In your wheel cylinders, the increase in fluid pressure makes a smaller set of pistons move.” “The piston forces brake fluid out of your master cylinder and through the tubes to your wheel cylinders.” To add interest, you are tempted to add background music such as an instrumental loop. To add interest, you are tempted to add background sounds such as as swooshing sound of the piston To add interest, you add background sounds such as as sound of flowing liquid. To add interest, you add background sounds such as as mechanical sound of the piston moving. The music continues. The music continues.

17 “When you step on your car’s brake pedal, a piston moves forward inside your master cylinder.” “In your wheel cylinders, the increase in fluid pressure makes a smaller set of pistons move.” “The piston forces brake fluid out of your master cylinder and through the tubes to your wheel cylinders.” To create a sense of social interaction, the narration is spoken in a non- accented human voice that conveys friendliness, intelligence, and energy. The voice principle can be applied to conversational style as in this example, or to formal style. Narration Principle People learn better from narration than text and better yet when the narration has a human voice with a standard accent than a machine voice or an accented voice. Another possible way to create a sense of learning partnership is to use a standard accented human voice that the learners interpret as friendly, intelligent, and dynamic. The reason is that learners may try harder to understand when they accept the speaker as a social partner.

18 Six Cognitive Principles about Media 5Reverse expertise principle –Most load principles needed for novice and/or low spatial ability learners, not for experts (but lower load does not hurt experts) 6Worked-example principle –Discovery problem-based learning imposes high cognitive load with little benefit – use worked examples based on cognitive task analysis

19 Worked example: How to fix instructional materials that are classified as split attention risks. 1. Read the text material, and separate the text into segments where each segment describes or explains one area or element in the visual. 2.Identify the location in the visual that each segment of the text is discussing. 3. Decide wither the visual can be reformatted so that you can insert each segment of text immediately next to the part that it is discussing or whether you must draw arrows from the part being discussed to a text fragment that is placed as close as possible to the part? 4.Revise the visual so that text and visual are fully integrated.

20 Worked example: How to fix instructional materials that are split attention risks. Test : To test Insulation Resistance from conductors to earth. How conducted : i ) Disconnect appliances and busways during these tests. Make sure main switch is "on" and all fuses are "in". Remove main earth from neutral bar and set meter to read insulation. Connect one lead to earth wire at MEN bar and take first measure by connecting the other lead to the active. Take next measure by connecting the lead to the neutral.ii) If resistance is not high enough in either of the two tests in i) then measure each circuit separately. Results required :i) At least One Mega ohm ii) Same result as i) above

21 Corrected example:

22 IV. What new and promising training design models are available? Reminder: Subject matter experts design most training Expert’s knowledge is automated and unconscious When asked “how do you do that” expert explanations are approximately 50% wrong Less than 20% of what is learned in training transfers back to performance ISD models are suited to less complex knowledge

23 New complex knowledge training design models : Use Cognitive Task Analysis for accuracy Mix declarative and procedural knowledge Provide cognitive learning support Provide “Part” and “Whole” task practice Advanced assessment possibilities Examples: vanMerrieboer’s 4C/ID and Clark’s Guided Experiential Learning (GEL) IV. What new and promising training design models are available?

24 How Is CTA Conducted? Three or more experts interviewed separately: Sequence of activities (before, during, after) Conditions (when, cues, indications, contra indications) Actions (How to complete each step) Decisions (When, Alternatives, Criteria) Concepts, processes, principles operating in this task – case study examples Standards of performance Equipment, materials

25 New 10 Stage Design System: (vanMerrienboer' s 4C/ID system) 1 Decompose the skill 2 Sequence task classes 3 Design learning tasks 4 Analyze mental models - process 5 Analyze cognitive strategies 6 Design supportive information 7 List rules or procedures 8 Prerequisite knowledge 9 Design JIT information 10 Design part-task and whole task practice

26 A B C analyze non- recurrent aspects analyze recurrent aspects analyze rules analyze prerequisite knowledge analyze cognitive strategies analyze mental models decompose the complex skill sequence skill clusters and/or task classes A B C analyze non- recurrent aspects analyze recurrent aspects analyze rules analyze prerequisite knowledge analyze cognitive strategies analyze mental models decompose the complex skill sequence skill clusters and/or task classes

27 Overview of GEL Design Select Course & SME’s  Identify Many Job and Mission Problems  Cognitive Task Analysis  Identify How to act and decide New concepts, processes Equipment and materials Performance standards  Design Lessons Immersive demonstrations Checklists for practice Job aids for transfer Guided practice exercises Feedback for practice  Select delivery media  Design Four level evaluation Test of prior knowledge Transfer letters

28 Using Guided Experience to Design Courses Problem Selection and Examples Identify prior experience of trainees Identify five large, authentic field problems Easy to very complex Create “worked examples” of the problems Use cognitive task analysis What are conditions, input and output Solve for prior experience of trainees Select Course & SME’s   Identify Many Job and Mission Problems Cognitive Task Analysis

29 Conceptual Knowledge About Field Problems Collect information about New concepts (definition and example) Process (how it works – big picture) Procedure (How to do it, conditions and consequences) Identify How to act and decide New concepts, processes Equipment and materials Performance standards

30 Design Lessons Sequence groups of problems into lessons First performed in field are first taught If no fixed sequence, easy before difficult Develop goals for each lesson Remember definitions of concepts Remember description of processes Practice and perform procedure Remember conditions and consequences Design Lessons Immersive demonstrations Checklists for practice Job aids for transfer Guided practice exercises Feedback for practice

31 6 Instructional Methods for Each Lesson Lesson Sequence: 1. Goals – You will learn how to (REMEMBER, DO…) 2. Reasons Value of learning - consequences of not learning What you already know that you should use 3. What You Need to Know to Perform Teach new concepts and processes needed to learn procedure Design Lessons Immersive demonstrations Checklists for practice Job aids for transfer Guided practice exercises Feedback for practice

32 Lesson Sequence: 4. Demonstrate procedure Worked example – authentic setting Instructor or Model should be credible, similar 5. Practice with incomplete problems Trainees use partly worked example or demo Practice more complex/difficult problems that integrate previous learning Gradually fade support – training wheels come off! 6. Feedback focused on strategy improvement Design Lessons Immersive demonstrations Checklists for practice Job aids for transfer Guided practice exercises Feedback for practice

33 Select delivery media Design Four level evaluation Test of prior knowledge Transfer letters  Media Selection, Evaluation and Transfer Select media based on context, practice and cost Evaluate on four levels 1. Reactions (motivation - confidence, value) 2. Learning (use practice exercises to evaluate) 3. Transfer (check with supervisor/commander) 4. Impact (did it make a difference to bottom line?) Send letter to commander asking for transfer help

34 Assessment Possibilities Stages in the development of expertise –Strategy assembly, gradual automation –Links between sub goals and explanation –Increasing speed with more practice –Reliable, valid measurement of expertise –Clear targets for corrective feedback –Connections between expertise and measured performance outcomes

35 Evidence for Effectiveness? Trainees learn faster, make fewer mistakes: No differences on routine, directly observable actions Large differences on implicit, complex mental strategies –Novices learn 25% faster (range 20-80%) –Made 70% fewer mistakes on cognitive process –Mistakes of control groups are more serious

36 Evidence for Effectiveness? Strategy sequence learned more quickly Fewer and less important decision errors More accurate explanations of decisions Speed of automation increased Better assessment of developing expertise Coaching and feedback targets obvious Learning and transfer increased but not motivation

37 IV. What motivates us to learn and how can we influence it during training? Learning is 50% motivation Three motivation outcomes: –active choice, –persistence and –mental effort Active choice may be increased by multimedia Persistence may increase initially but it often decreases quickly Evidence that new media may decrease mental effort invested in learning

38 3. Design to support learner motivation Web persistence rates 50% less than classroom Problem with value for course content, social connections, poor planning and/or misperceived demands Apply motivation research (provide value, clear learning and performance goals, support confidence and planning)

39 V. Deciding whether training should be delivered live or by multi media? 1)Can application context be simulated on web? 2)Can timing, complexity of feedback be accommodated on web? 3)If “no” to 1) or 2), offer classroom instruction. 4)If 1) and 2) are “yes” Use ingredients method of cost/utility analysis – if web is cost beneficial, use it.

40 In Conclusion: Avoiding Misconceptions Start by analyzing problems to be solved with web-based courses Separate design AND development for learning AND motivation Use cognitive task analysis to capture expertise Use worked examples and scaffolding to support constant, incremental practice Reduce cognitive load to the minimum when training complex knowledge

41 Research on the Design of Technology-Based Training Richard E. Clark Rossier School of Education Center for Cognitive Technology University of Southern California ICT Immersive Training Workshop May Marina del Rey California