1 Automaticity development and decision making in complex, dynamic tasks Dynamic Decision Making Laboratory Social and Decision Sciences Department Carnegie Mellon University Cleotilde Gonzalez Rickey Thomas Polina Vanyukov

2 Complex and dynamic tasks Executing a battle, driving, air traffic controlling, managing of a production plan, piloting, managing inventory in a production chain, etc. Demand real-time decisions (time constraints) Demand attentional control Require multi-tasking: they are composed of multiple and interrelated subtasks Demand the identification of ‘targets’ defined by multi- attributes Demand multiple and possibly changing responses

3 Automaticity in dynamic, complex tasks targets and distractors are often inconsistently mapped to stimuli and responses Often, we bring pre-learned categories and mappings to a task stimulus - category category - response L letter button click Are decision makers in dynamic situations operating in controlled processing continuously?

4 Proposed model of automaticity in DDM Goals (Relevancy) Task switching (resource allocation)

5 Experiments Automaticity develops with consistently mapped stimuli to targets, even when targets move and time is limited (Experiment 1) The consistency of target to response mapping also determines automaticity development (Experiment 2) Automaticity of a task component frees-up time and resources for high level decision-making (Experiment 3) Automaticity develops differently with different degrees of pre-learned categories (Experiment 4)

6 The Radar Task

7

8 General method Independent variables o stimulus mapping (CM or VM) CM = Search for Numbers in Letters VM = Search for Letters in Letters o cognitive load Memory set size (MSS): Number of possible targets to remember (1 or 4) frame size (FS): Number of blips present on the screen at a given time (1 or 4) o target present/absent (a target was present 75% of the trials) Dependent variables o Accuracy: proportion of correct detections or decision-making responses o Time: mean target detection or decision-making time in msec From 18 to 30 hours of practice, 3 hours per day 6 to 10 days

9 Experiment 1: Consistency of stimuli Replicate major findings from the dual-process theory (Schneider & Shiffrin, 1977) in a dynamic task Automaticity is acquired with practice in consistent mapping conditions, and automatic performance is unaffected by workload

10 Experiment 1: Method o CM vs. VM o Cognitive Load Variables Memory Set Size Frame Size o Only one possible response: pressing spacebar when target is detected

11 Experiment 1: Accuracy

12 Experiment 1: Detect Time

13 Experiment 1: Summary Radar’s manipulations of cognitive load interact with stimulus mapping in ways that parallel Schneider & Shiffrin’s results Automaticity develops with extended practice and consistently mapped stimuli even when targets move and time is limited Radar task can be used to study automaticity in dynamic stimulus environments

14 There is some evidence that response mapping is not critical for automaticity to develop (Fisk & Schneider, 1984; Kramer, Strayer, & Buckley, 1991) In complex tasks mapping of targets to responses can be inconsistent o Resulting in large processing costs, even when stimuli are consistently mapped to targets Experiment 2: Response Consistency

15 Experiment 2: Method o Only consistently mapped stimuli o Cognitive Load Variables Memory Set Size Frame Size o Response consistency varied in four levels

16 Mapped to Stimuli Fully Mapped to interface Partial Mapping to interface Random Mapping T T TT Response Mapping Conditions

17 Experiment 2: Accuracy

18 Experiment 2 : detect time

19 Experiment 2: Summary A consistent response reduces processing requirements Total task consistency (both, consistency of stimuli and consistency of responses) matters o There are processing costs if responses are not consistently mapped, even when stimuli are Implications o Interface design: interface influences processing of responses Response selection using track-up vs. north-up displays Make response selection intuitive Interface design, decision support tools, training o We can now systematically manipulate Radar to elucidate the effects of automaticity on high-level dynamic decision-making

20 Experiment 3: Automatic detection & high- level decision making How would automatic detection of a component help decision-making? Decision-making component required operators to analyze a sensor array of detected aircraft Sensor and weapon information changed dynamically

21 Experiment 3: Method Sensor Reading Task Determine if Target is Hostile o Scan Sensors o > 13 (Hostile) o < 13 (Non-Hostile) Press Ignore (5-Key) Select Response (Weapon Systems) o Guns vs. Missiles o > 10 Missiles (6-Key) o < 10 Guns (4-Key) Quiet Airspace Report o No targets detected o Click submit report with mouse key

22 Experiment 3: Detect Accuracy

23 Experiment 3: Decision-making Accuracy

24 Experiment 3: Detect Time

25 Experiment 3: Decision-making Time

26 Experiment 3: Summary Consistent mapping of targets improved he accuracy of the decision-making of the task Detect time, detect accuracy, and whole-task performance are sensitive to workload manipulations Implications o Consistent mapping actually improved whole-task performance by freeing up time for the controlled sensor- reading tasks to run to completion o Thus, processing speed-up associated with automatic detection can have a large impact on whole-task performance

27 But…? Is accuracy of decision-making improved simply because there is more time to process? Effect of detection on high-level decision-making in the presence of a dual-task

28 Experiment 3b: Method Secondary tone task: enter count of number of non- standard tones o Calibrated to standard tone at beginning of session for each participant o Non-standard tones higher/lower pitch than standard

29 Experiment 3b: results In fact the Radar task performance was the same with and without the tone task! Detect Time o No Effect of secondary task Detect Accuracy o No Effect of secondary task Decision-Making Time o No Effect of secondary task Decision-Making Accuracy o No Effect of secondary task

30 Experiment 3b: Implications No effect of dual task on RADAR performance Operators are allocating resources away from tone task to maintain RADAR performance Implications o Finding supports the hypothesis that consistent mapping improves decision-making performance by freeing up resources for other tasks o Thus, processing speed-up and low resource requirement associated with consistent mapping can have a large impact on performance in complex task

31 Experiment 4: Categorization Since consistent mapping is the search for numbers in letters, it is possible that load-free processing is due to categorization (Cheng, 1985) Purpose of this experiment is to establish the presence of load-free processing without categorization

32 Experiment 4: Method Incorporate memory ensembles where no possible categorization can take place either a priori or with learning CM vs. VM with tone o CM = {C, G, H, M, Q, X, Z, R, S} o VM = {B, D, F, J, K, N, W, P, L} Memory ensembles were equated o Angular {H,M,X,Z,F,K,N,W} vs. Round {C,B,D,G,Q,P,R,J} o Beginning {B,C,D,F,G,H,J,K} vs. End {M,N,P,Q,R,W,X,Z} Cognitive Load Variables o Memory Set Size (1 or 4) o Frame Size (1 or 4) Indicated detection of target by pressing spacebar o Detect Performance o Detect Response Time

33 Experiment 4: Detect accuracy

34 Experiment 4: Decision-making accuracy

35 Experiment 4: Detect time

36 Experiment 4: Decision-making time

37 Experiment 4: Implications Varied mapped performance is more sensitive to load than consistently mapped performance Individuals performed better in the high-level decision-making component of Radar when stimulus mapping was consistently mapped Implications o Categorization is NOT a necessary requirement for automaticity development o Consistent stimulus mapping is a necessary condition for the development of automatic detection

38 Summary of accomplishments Developed Radar, a dynamic simulation where it is possible to study (i.e., to measure) automaticity In Radar it is possible to elucidate the effects of automaticity on high-level dynamic decision-making Established the usefulness and applications of the dual- process theory of automaticity Deepen our understanding of the implications of automaticity development for practical real-world tasks Brought together two main theories of automaticity: instance- based theory and dual-process theory

39 Future research Consistency of mapping and responding is relative to the categories (i.e., similarity) that a user can form Thus, consistent mapping can lead to automatic responses for high-level decision-making after extended practice

40 Looking towards applications Test these hypotheses in airport luggage screening Decide whether to hand search the luggage There is no consistency but rather just similarity (relative to a ‘knife’ category)