Decision Models: Some Lessons From the Field G. Klein, R. Calderwood Presented by: Hilary Ince

Introduction Prevailing paradigms in decision research do not apply well to the real world Although diverse, decision theories center around the decision tree Decision maker presented as an analyst Generates alternatives, evaluates outcomes Decision event is the focus of concern

Introduction Main view is that humans are suboptimal, biased decision makers Training focuses on reducing bias These programs are of limited benefit Novice decision makers are not good decision analysts, and experts can make faster decisions Primary effort in naturalistic decision making is situation assessment, not the moment of choice

Introduction Classical decision theory is useful for prescribing how to make better decisions Not describing how people actually make them! Classical decision models aren’t all bad! Generic, can be widely applied Techniques ensure everyone speaks the same language, uses the same metrics Good for framing and guiding decision making When all underlying assumptions are met, the techniques ensure optimal selection

Introduction Assumptions can be problematic Goals can be isolated Utilities can be assessed independent of context Probabilities can be accurately estimated Choices, goals, and evidence are carefully defined Utilities of an outcome are independent of other outcomes All difficult to meet in the real world

Can Goals Be Isolated? Slow down an enemy by denying access to key roads Goal is linked to many others, like using the roads for a counterattack in a few days It can be dangerous to oversimplify the interactions of goals

Can Utilities Be Assessed Independent of Context? Chess is a relatively restricted environment Chess masters do not rate moves using a standard set of dimensions Moves are evaluated in context

Can Probabilities Be Accurately Estimated? Even without time pressure and stress, this is difficult

Will Choices, Goals, and Evidence Be Clearly Defined? The decision maker may not know the desired end state Fireman rushing to a fire Is goal to put out fire, or stop it from spreading? Can call for reinforcements, but cannot leave district vulnerable to other fires Actual goal is to do the best with what he has… not well defined

Are the Utilities of Outcomes Independent of Other Outcomes? Possible for static and limited tasks In complex operations, outcomes are interrelated

Recognitional Decisions Research has been done on urban fireground commanders (FGCs) Observations and retrospective account of emergency events Examples: whether to initiate search and rescue, whether to initiate an offensive attack or focus on defense, and where to allocate resources Difficult to fit their behavior into decision tree They resisted describing “making choices”, “considering alternatives”, or “assessing probabilities”

Recognitional Decisions Saw themselves as acting and reacting based on prior experience, planning, monitoring, and modifying plans to meet constraints No evidence of exhaustive option generation FCGs insisted that they did not deliberate about advantages and disadvantages of options They “watched” the choice play out mentally to see if anything went wrong, then did the same with another option if the first didn’t work NOT concurrent in the majority of decisions (only 18.6% concurrent)

Recognitional Decisions 26 interviews conducted, 32 incidents studied, 156 decisions probed in detail Decision point defined as the instances when the FGC could agree, after the fact, that alternative actions were possible 18.6% concurrent, 81.4% recongitional decisions These strategies can be described by the recognition primed decision (RPD) model

Recognitional Decisions RPD Model: A person understands a situation based on familiarity with previous experience Helps recognize feasible goals, relevant cues, expectancies to monitor, and plausible actions Decision maker is more likely to generate a viable option as the first one to consider Evaluation of the option is through mental simulation

Recognitional Decisions The model has distinct features: Assesses that experienced decision makers can generate plausible options rather than randomly generate a large option set Decision maker can use serial process rather the comparing options to each other Decision makers can use satisficing rather then taking time to pick the best possible option Evaluation can involve mental simulation Emphasizes situation understanding

Field Research: Tasks and Paradigms Many other studies have shown limitations of the decision tree model Command and control, business, jury deliberation Author suggests a continuum of RPDs and analytical decisions Different situations call for different decision making

Decision Support Systems and Training for Recognitional Decision Making Decision support systems based on recognitional decisions would improve speed and accuracy of situational understanding Analogue displays rather than alphanumeric data Databases of prior cases could be helpful Emphasis of training should be sensitivity to critical factors, effective use of expectancies, ability to anticipate important contingencies, and situational understanding

Naturalistic Decision Making G. Klein Presented by: Hilary Ince

Origins Major contribution of naturalistic decision making (NDM) research is to describe how people make decisions in the real world When NDM emerged, it was only clear how people DIDN’T make decisions Training methods and decision support tools developed with traditional decision making theory in mind weren’t useful

Origins NDN began with field research on decision making strategies Looked at how people made actual decisions under stress, not how people are suboptimal Army and Navy were initial financial backers Navy interested because of USS Vincennes incident

Origins At least nine NDM theories were developed in parallel before the first NDM conference in 1989 Hammond’s cognitive continuum theory states that decisions vary based on intuitive and analytical processes used Rasmussen’s model of cognitive control distinguished skill-based, rule-based, and knowledge-based behavior Klein’s recognition primed decision model

Origins Everyone reached the same conclusion: People don’t generate and compare options! Prior experience is important Making the decision means committing to an alternative even if other alternatives haven’t been examined

Recognition Primed Decision Model RPD describes how people use past experiences to make decisions (recognizable patterns) Patterns highlight most relevant cues, provide expectancies, identify plausible goals, and suggest typical reactions RPD model describes how good decisions are made without comparing options

Contributions of NDM Spurred development of cognitive field research and cognitive task analysis methods NDM has affected Army doctrine Army Field Manual on Command and Control has a section on intuitive decision making Swedish armed forces use the RPD model adapted for military planning guidance Decision training has been influenced Tactical Decision Making Under Stress (TADMUS) after USS Vincennes incident

Future of NDM Being used to study situational awareness, sensemaking, planning and replanning, and their interactions Future may focus on team decision making