The final resting place for all this research… Ron Laughery, Ph.D. University of Colorado

Items to be covered… What is the problem this research is trying to solve from an operational perspective? What is the basic human performance modeling and simulation approach that this research will feed? What is the specific tool and architecture that we are working to advance? What are the issues in moving this research into practice?

What is the problem this research is trying to solve from an operational perspective? The £5,000,000,000 question… –In about 1995, Robin Miller, an operational analyst with the MoD asked us this question and made this statement at a meeting: “A question I get all the time can be summed up as this – should we invest £5B in new kit, or should we instead invest that £5B in training? If your models can’t help me answer that question, you’re not doing your job.” We are trying to ensure that we are doing our job in Mr. Miller’s eyes

What is the basic human performance modeling and simulation approach that this research will feed? In military and civilian systems, decisions are increasingly being made on the basis of model based analyses –System effectiveness depends upon…

Two basic approaches to modeling human/system performance Reductionist –Breaking human activity and interaction with the system into discrete activities

Advantages/disadvantages of reductionist modeling approach Advantages –Intuitive –Level of detail determined by need –Basic data are usually available or easily obtained –Consistent with many military systems and operational analysis models Disadvantages –Often requires extensive subject-matter expert input

Second approach to modeling human/system performance First principled/cognitive models –Based on theories of the underlying mechanisms that facilitate human behavior perception Working memory Long-term memory Iconic storage Central processing Response mechanisms

Example: ACT-R Representation and Equations RetrievalGoal ManualVisual Productions IntentionsMemory Motor Vision World Activation Learning Latency Utility Learning IF the goal is to categorize new stimulus and visual holds stimulus info S, F, T THEN start retrieval of chunk S, F, T and start manual mouse movement S201 Size Fuel Turb Dec L203Y Stimulus Chunk BiBi S SL S 13

Advantages/disadvantages of the first principle approach Advantages –Requires less data input from either experiments or subject matter experts –More first-principle based and, if component models are valid, easier to defend Disadvantages –Model construction can be quite cumbersome for simple tasks –We don’t have enough real first-principle models of human performance

A strategy that has worked- a hybrid approach The flexibility of reductionist models combined with the power of first principles of human behavior is the formula for success perception Working memory Long-term memory Iconic storage Central processing Response mechanisms

Reductionist modeling with Task Network Modeling Largely involves the extension of a task analysis into a network defining sequencing

Going from a task network to a running computer model Add timing information and task/system interdependencies

Add human decision making strategies Any defined branch point represents a need for a decision Logic and rule sets, goal seeking, naturalistic ?

Then, develop a scenario, equipment model and/or links to other simulations

Run the model to collect human/system performance data

Combining First Principles of human behavior with Task Network Models For the past 16 years, we have been embedding and linking first principle models of human performance into our tools including –Cognitive workload and human response –Micro models of human time and accuracy –Human error and system response to error –Performance shaping factor effects –Linkage to anthropometric, biomechanical models –Goal driven task scheduling –Naturalistic Decision Making –Situation awareness modeling –Integration of cognitive engineering models such as ACT/R Predicting training effects is still the weakest link!Predicting training effects is still the weakest link!

Improved Performance Research Integration Tool (IMPRINT): Capability and Application

What Does IMPRINT Do? It helps you... u Set realistic system requirements u Identify future manpower & personnel constraints u Evaluate operator & crew workload u Test alternate system-crew function allocations u Assess required maintenance manhours u Assess performance during extreme conditions u Examine performance as a function of personnel characteristics, training frequency & recency u Identify areas to focus test and evaluation resources

IMPRINT Architecture - Operations Modeling

Improved Performance Research Integration Tool (IMPRINT)

IMPRINT Architecture - Maintenance Modeling Send systems on missions as defined by scenario Simulate need for maintenance Systems Ready for Next Mission Repair systems Manpower Pool corrective & continue mission combat damage corrective & stop mission preventive Repair Parts

Who Has IMPRINT? u Army u Navy u Air Force u Other Government u Contractors u University and growing

Mental Workload

Task type (Taxon*) MOPPHeatColdNoiseSleepless Hours Visual T A T Numerical A TA Cognitive A TA Fine Motor Discrete T A T Fine Motor Continuous Gross Motor Light T T Gross Motor Heavy Commo. (Read & Write) A Commo. (Oral) T A A T = affects task time, A = affects task accuracy, TA= affects both Current IMPRINT Implementation: Stressors by Task Type * O’Brien, L. H., Simon R. and Swaminathan, H. (1992). Development of the Personnel-Based System Evaluation Aid (PER-SEVAL) Performance Shaping Functions. ARI Research Note 92-50

Approach to modeling human response to stressors The general effects... On a specific task…. under specific conditions... leads to this specific effect at this time….. task time = 112.3% of normal

Use task network models to study aggregate effects of PSFs

What view of training is in IMPRINT now?...

What we really need for a reasonably accurate representation of training… We need these functional relationships… –For different task types (the taxonomy) –For different “types” of training

Big questions… Purpose of models –Design of optimal training systems –Design of systems considering training Taxonomies –Training environment –Task type Scope/complexity of tasks studied –Do small tasks scale to large tasks? How do we treat Retention