MURI: Training Knowledge and Skills for the Networked Battlefield ARO Award No. W9112NF Alice Healy and Lyle Bourne, Principal Investigators Benjamin Clegg, Bengt Fornberg, Cleotilde Gonzalez, Eric Heggestad, Ronald Laughery, Robert Proctor, Co-Investigators

Project Mission (As Defined by the BAA) Objectives “Develop and evaluate models that predict performance improvement or decrement for a range of militarily significant individual and collective tasks that can be linked to various types and amounts of training while considering the effects of aptitude and experience.”

Proposed Project (As Defined in Executive Summary) Goals Construct a theoretical & empirical framework for training Predict the outcomes of different training methods on particular tasks Point to ways to optimize training

Statement of Work The work to be performed falls into 3 interrelated categories: (1) Experiments (a) Development & testing of training principles (b) Acquisition & retention of basic skill components (c) Levels of automation, individual differences, & team performance (2) Taxonomic analysis (a) Training methods (b) Task types (c) Performance measures (d) Training principles (3) Predictive computational models (a) Formulated from experimental data (b) Applied to military tasks

Parts of Project (1) Experiments (a) Development & Testing of Training Principles (b) Acquisition & Retention of Basic Components of Skill (c) Levels of Automation, Individual Differences, & Team Performance (2) Taxonomy (3) Models (a) ACT-R (b) IMPRINT (c) Model Assessment

Three Major Parts of Present Meeting (I) Introduction (II) Plans For Project and Progress So Far (A) Experiments (B) Taxonomy (C) Models (III) Summary and Reactions

Introduction to MURI Personnel (1) University of Colorado (CU) Alice Healy, Principal Investigator Lyle Bourne, Co-Principal Investigator Bengt Fornberg, Co-Investigator Ron Laughery, Co-Investigator Bill Raymond, Research Associate (2) Carnegie Mellon University (CMU) Cleotilde Gonzalez, Co-Investigator (3) Colorado State University (CSU) Ben Clegg, Co-Investigator Eric Heggestad, Co-Investigator (4) Purdue University (Purdue) Robert Proctor, Co-Investigator

Roles in Project (1) Overview and Coordinate CU, Healy & Bourne (2) Experiments (a) Development & Testing of Training Principles CU, Healy & Bourne (b) Acquisition & Retention of Basic Components of Skill Purdue, Proctor (c) Levels of Automation, Individual Differences, & Team Performance, CSU, Clegg & Heggestad (3) Taxonomy CU, Raymond (4) Models (a) ACT-R CMU, Gonzalez (b) IMPRINT CU, Laughery (c) Model Assessment CU, Fornberg

Key Comments from Review (1) Tighter integration of the modeling effort with the learning research and experimentation is needed and should take place during the first few months of the project. (2) Data-tractability (how much data on the training and the subjects are needed to make reasonable evaluations) and computational tractability need to be addressed in greater depth. (3) Training in a complex networked environment could be addressed at greater depth, but, since even training for more elementary tasks is not yet understood, the proposed work is reasonable. (4) More emphasis on software and less emphasis on papers published in professional journals and books is needed in the deliverables. (5) There is a question about how the obligations of one senior MURI team member to a company and to the Advanced Decision Architectures Collaborative Technology Alliance will be coordinated with that member’s obligation to the MURI.

Outline of Plans for Project and Progress So Far (I)Preliminary Investigator’s Meeting, Boulder, May 25, 2005 (II) Preparation of Investigators’ WIKI and Public MURI website (III) Experiments (A) Development & Testing of Training Principles Healy & Bourne (B) Acquisition & Retention of Basic Components of Skill Proctor (C) Levels of Automation, Individual Differences, & Team Performance Clegg & Heggestad (IV) Taxonomy Raymond (V) Models (A) ACT-R Gonzalez (B) IMPRINT Laughery (C) Model Assessment Fornberg

Development and Testing of Training Principles Summary of 30 Training Principles: Prepared for NASA cooperative agreement Two Examples of Training Principles Strategic-Use-of-Knowledge Principle When a large amount of new factual information must be learned and retained, that information should be related to the learner’s existing knowledge in any way possible. Principle of Contextual Interference Introduce sources of interference into training material. Interference may weaken performance during training but should strengthen retention and transfer.

Development and Testing of Training Principles: Proposed and In-Progress Experiments (1)Tests of the generality across tasks of individual principles -- 1 in-progress on strategic use of knowledge (2) Tests of multiple principles in a single task -- 1 in- progress on serial position, list length, and chunking effects (3) Tests of principles in complex, dynamic environments -- 1 in-progress on contextual interference

CU Experiments: Communication with Modelers to Date (1) Data Entry: Fatigue Effects; Speed-Accuracy Tradeoffs Sent to Gonzalez & Laughery data from 2 previously published experiments (2) Hand-Eye Coordination: Specificity of Training; Retention and Transfer Effects Sent to Gonzalez & Laughery data from 1 unpublished experiment (3) Further Work on Data Entry: Multiple Principles in a Single Task Sent to Gonzalez & Laughery data from 8 previously published experiments examining (a) specificity of training, (b) procedural reinstatement, (c) depth of processing, (d) phonological coding Sent to Gonzalez & Laughery data from 2 newly completed experiments examining (a) cognitive and motoric fatigue, (b) feedback and cognitive load

Data Entry Experiments Task: Subjects see a 4-digit number, and they type it on a computer keypad Design: In each session half, subjects see and type 5 blocks of 64 numbers Measures: Both typing accuracy (proportion correct) and typing speed (total response time) are measured

CU Experiments: Communication with Modelers Planned for Next Year (1) Data Entry: Mental Rehearsal 2 experiments on repetition priming and motor imagery (2) Hand-Eye Coordination: Further Work on Specificity of Training 1 experiment assessing relative merits of specificity and variability of training 1 experiment on strategy instructions and gender effects 1 experiment on immediate testing and transfer (3) Duration Estimation: Functional Task Principle 1 experiment varying presence of secondary task 2 experiments varying features of secondary and primary tasks 2 experiments varying difficulty and modality of secondary task

CU Experiments: Expanded Work on Complex Tasks (1) RADAR Task from CMU Test of Training Difficulty Principle (2) First Responder Navigation Task with Emergencies from NSF SGER Grant Test of Memory Constriction Hypothesis Test of Look-Up Speed in Emergency Check-List

Summary of Response to Comments from Review (1) Tighter integration of modeling effort and experimentation Experimenter-taxonomist-modeler interactions are on-going, facilitated by meetings and WIKI (2) Data and computational tractability Assessments of data-computation compatibility are on-going, facilitated by meetings between Fornberg and research personnel (3) Training in a complex networked environment Experiments underway using complex and more naturalistic tasks, such as RADAR tracking, emergency response teams, flight simulation (4) More emphasis on software in the deliverables ACT-R and IMPRINT software products will be available at various points in the future (5) The multiple obligations of one senior MURI team member Periodic meetings between senior member and research associate, enabling use of IMPRINT by other team members to model existing data

Present and Future Activities (1) Activities (a) Experiments (b) Taxonomy (c) Modeling (2) How do we propose to get from the current state of knowledge to the final goal of predicting performance as a function of training