1. Automation Usage Decisions: Controlling Intent and Appraisal Erros in a Target Detection Task Hall P. Beck Appalachian State University Mary T. Dzindolet Cameron University Linda G. Pierce Army Research Laboratory Perform the actions necessary to accomplish the objective via automated or manual control.

2. Automation Usage Decisions (AUDs) AUDs: Operator must choose between automated and/or a manual control or less technologically sophisticated means of control. Misuse: The overutilization of automation Disuse: The underutilzation of automation Purpose: To determine if feedback and scenario training mitigate misuse and/or disuse. Perform the actions necessary to accomplish the objective via automated or manual control.

3. Two Causes of Suboptimal AUDs Appraisal Errors: Operators are unable to determine if automation or a non-automated alternative maximizes the likelihood of task success. Intent Errors: Operators know the relative utilities of the options but disregard this information when deciding whether to use or not use automation. Perform the actions necessary to accomplish the objective via automated or manual control.

4. Design and Procedure A 2 (Feedback, No Feedback) x 2 (Scenario Training, No Scenario Training) x 2 (Superior, Inferior Machine) between-subjects in which an AUD is the dependent variable

5. Design and Procedure 1.Receive scenario training or control information 2.Perform 280 target detection trials 3.Feedback group is told how many errors that they and the machine made. No feedback group does not receive this information. 4.Operator either makes twice (superior machine condition) or half (inferior machine condition) as the machine. Perform the actions necessary to accomplish the objective via automated or manual control.

6. High Noon Ten trials will be replayed from the preceding 280. Extra credit depends on the number of these trials which are correct. Operator bases credit on their own or the machine‘s performance. Perform the actions necessary to accomplish the objective via automated or manual control.

7. Underlying Logic Suboptimal AUDs in the no feedback conditions could be due to appraisal and/or intent errors. Suboptimal AUDs in the feedback condition should only be due to intent errors. Feedback should reduce or eliminate appraisal errors. Scenario Training should decrease intent errors.

8. Scenario Training x Feedback

9. Conclusions Results suggest that operators‘ AUDs were determined by multiple contingencies. Operator training programs should include procedures should as scenario training to reduce intent errors. Advances in the reliability of decision aids and other automated devices will be a hollow achievement unless our knowlege of hardware and software is matched by an equally sophisticiated comprehension of the causes of misuse and disuse.

10. To Shoot Or Not To Shoot

11. Since 1900, 10% to 25% of US war fatalities in resulted from fratricide

12. Targeting Decisions: Possible Outcomes 1)Soldier and CID detect a friend. 2) Soldier and CID fail to detect a friend. 3) Soldier detects a friend and CID fails to detect a friend. 4) Soldier fails to detect a friend and CID detects a friend.

13. Automation Usage Decisions (AUDs) AUDs- Choices in which a human operator has the option of relying upon manual control or one or more levels of automation (LOAs) to perform a task. Optimal AUD-Soldier relies upon the form of control that is most likely to result in a correct decision.

14. Types of Suboptimal AUDs Misuse is over reliance, soldier employs automation when manual control or a relatively low LOA has a greater likelihood of success Disuse is the under utilization of automation, soldier manually performs a task that could best be done by a machine or a higher LOA. Perform the actions necessary to accomplish the objective via automated or manual control.

15. Beck, Dzindolet, & Pierce (2002) Appraisal Errors-Soldier misjudges the relative utilities of the automated (CID) and non- automated (e.g., view through gun site) options. Intent Errors-Soldier disregards the utilities of the alternatives when making AUDs.

16. Intent Errors: Two Images of an Operator An operator is a single-minded individual whose sole object is to maximize task performance An operator‘s decision to rely on automation is based on a number of contingencies only one of which is to achieve a successful performance. Perform the actions necessary to accomplish the objective via automated or manual control.

17. John Henry Effect John Henry Effect: Operators respond to automation as a challenger, competitor, or threat Increasing the operator’s personal involvement with the non-automated alternative augments the likelihood of a John Henry Effect.

18. John Henry Effect Variables that increase the strength of a John Henry Effect augment operators‘ preference for the non-automated over the automated alternative Heightened preference for the non-automated option should: 1) increase disuse and 2) decrease misuse

19. Design 2 (Operator: Self-reliant, Other-reliant) x 2 (Machine Performance: Inferior, Superior) x 14 (Trial Blocks) design Dependent Variable: Suboptimal AUDs (Superior Machine: Basing credit point on the operator’s performance; Inferior Machine: Basing credit on the machine’s performance)

20. Credit Choice Screen

21. Sample Helicopter Photograph

23. Operator Response Screen

24. CID Response Screen

25. Results Screen

26. Hypotheses Self-reliant operators will be less likely to base credit points on the CID than other-reliant operators Therefore –Disuse will be greater in the self-superior than in the other-superior condition –Misuse will be higher among other-inferior than self-inferior persons

27. Disuse Figure 1. Mean suboptimal automation usage decisions (AUDs) as a function of operator and trial block for persons working with the superior machine.

28. Misuse Figure 2. Mean suboptimal automation usage decisions (AUDs) as a function of operator and trial block for persons working with the inferior machine.

29. Conclusions 1) Self-reliant and other-reliant operators were yoked. Each had the same information. It seems reasonable to conclude that the difficulty in determining the optimal AUD was approximately equal in both conditions. Thus, the large differences in suboptimal AUDs were probably due to intent rather than appraisal errors. 2)Results support the hypotheses that factors which augment the degree of personal involvement or challenge from automated devices will increase the probability of disuse and decrease the likelihood of misuse

30. A Few Implications 1)Operator training programs should attempt to attenuate intent as well as appraisal errors. 2)At least on this task, intent errors were a significant source of suboptimal AUDs 3)Both appraisal and intent errors are sufficient to produce suboptimal AUDs although neither is necessary 4)It will be a hollow achievement if advances in our knowledge of hardware and software is matched by an equally sophisticated comprehension of the causes and control of misuse and disuse.