Psychological Aspects of Risk Management and Technology – G. Grote ETHZ, Fall09 Psychological Aspects of Risk Management and Technology – Overview

Psychological Aspects of Risk Management and Technology – G. Grote ETHZ, Fall09 "The correct functioning of the train control system and the automatic traffic control system is to be monitored by the signaller. If necessary, he/she has to intervene manually. During normal operation, no monitoring is necessary as long as the operational requirements are met. In the case of disturbances or incidents, the notification of the required services and the required alarm procedures must be guaranteed." (Excerpt from the rule book of a European railway company)

Psychological Aspects of Risk Management and Technology – G. Grote ETHZ, Fall09 Supervisory control (Sheridan, 1987)  Planning off-line what task to do and how to do it  Teaching the computer what was planned  Monitoring the automatic action on-line to make sure all is going as planned and to detect failures  Intervening when deviations occur or plans change  Learning from experience in order to improve performance

Psychological Aspects of Risk Management and Technology – G. Grote ETHZ, Fall09 Ironies of automation (Bainbridge, 1983)  "If decisions can be fully specified then a computer can make them more quickly than a human operator can. There is therefore no way in which the human operator can check in real-time that the computer is following its rules correctly. If the computer is being used to make the decisions because human judgement and intuitive reasoning are not adequate in this context, then which of the decisions is to be accepted? The human monitor has been given an impossible task."  Attempts at removing humans from automated systems increase their importance as system back-up, while at the same time reducing human capabilities and motivation for adequate judgment of technical functioning and adequate intervention.

Psychological Aspects of Risk Management and Technology – G. Grote ETHZ, Fall09 Design challenges in automated systems  Avoiding mix of qualitative overload and quantitative underload  Avoiding leftover activities in automation gaps  Securing implicit knowledge  Providing fit between accountability and control

Psychological Aspects of Risk Management and Technology – G. Grote ETHZ, Fall09 Strategies for distributing tasks between human and technology

Psychological Aspects of Risk Management and Technology – G. Grote ETHZ, Fall09  Coping with ill-defined problems  Handling complex, but well- defined tasks Relative capabilities of humans and machines

Psychological Aspects of Risk Management and Technology – G. Grote ETHZ, Fall09 Critique of comparison strategy  Humans and technology cannot be quantitatively compared  The same function is carried out in qualitatively different ways by humans and technology  Humans and technology do not substitute, but complement each other  Instead of either-or decisions, the interaction between humans and technology needs to be designed  Task requirements are determined by interactions between differen functions  Automating one function will influence handling of another function by the human

Psychological Aspects of Risk Management and Technology – G. Grote ETHZ, Fall09 Overcoming the ironies of automation through the design principle of complementarity  Designing the interaction between human and technical system based on complementary differences creating a new performance quality –technology not as competitor and not as imitation of the human operator, aiming at replacing him, but as –complementary support of human strengths (e.g. solving of ill-defined problems) and compensation of human shortcomings (e.g. unreliable repetition of monotonous operations)  Supporting human control over technical systems and the human's role as system manager

Psychological Aspects of Risk Management and Technology – G. Grote ETHZ, Fall09 Propositions for safe design of automation  Every automated system is a socio-technical system, irrespective of its degree of automation.  People are accountable for correct system functioning even in fully automated systems.  Accountability requires control over the technical system in terms of system transparency and predictability and proper means for influencing the system.  Instead of either-or decisions based on quantitative comparisons of capabilities and performance, human- technology interaction is to be designed based on complementary use of qualitatively different performance potentials.  System control is determined by the distribution of tasks between human and technology and between people.

Psychological Aspects of Risk Management and Technology – G. Grote ETHZ, Fall09 Objectives of the design method KOMPASS (Grote et al., 2000; Wäfler et al., 2003)  Embedding function allocation decisions in job and organizational design  Supporting task design that considers cognitive as well as motivational preconditions and outcomes  Supporting prospective design in interdisciplinary teams  Balancing of user participation and reliance on expert criteria  Implementing a complementary design philosophy

Psychological Aspects of Risk Management and Technology – G. Grote ETHZ, Fall09 KOMPASS: Global design objectives  Work system: Local regulation of system variances and disturbances  Individual work task: Competence development and intrinsic motivation  Human-technology interaction: Human control over technology

Psychological Aspects of Risk Management and Technology – G. Grote ETHZ, Fall09 KOMPASS: Design criteria

Psychological Aspects of Risk Management and Technology – G. Grote ETHZ, Fall09 Manual Manual, technically supported Manual, technically supported and constrained Automatic, manually comfirmed Automatic Manual, technically constrained Design objective: Fit between level of information and execution authority and between overall authority and responsibility Example of KOMPASS criteria: Information and execution authority

Psychological Aspects of Risk Management and Technology – G. Grote ETHZ, Fall09 KOMPASS: Design heuristic  Phase 1: Expert analysis of existing work systems.  Phase 2: Discussion of design philosophy. –Step 2.1. Definition of the primary task and the functions of the planned work system. –Step 2.2. Definition of a shared evaluation concept to differentiate between successful and unsuccessful work systems. –Step 2.3. Identification of the main potentials for improvement and design objectives –Step 2.4. Identification of the potential contributions to a successful work system by human operator, technical system and organizational conditions. –Step 2.5. Specification of the working conditions required for human operators to make their specific contributions. –Step 2.6. Decision on usefulness of the KOMPASS criteria for the analysis, evaluation and design of work systems.  Phase 3: Derivation of concrete design requirements. –Step 3.1. Derivation of requirements for system design. –Step 3.2. Definition of work packages.

Psychological Aspects of Risk Management and Technology – G. Grote ETHZ, Fall09 KOMPASS application  Analysis of different scenarios for task distribution and work process design regarding KOMPASS criteria  Applying KOMPASS design heuristic to gain understanding of (implicit) assumptions in scenarios regarding e.g. –competence for handling and power for transferring uncertainties by the different actors –role of technology vs. different human operators as back-ups

Psychological Aspects of Risk Management and Technology – G. Grote ETHZ, Fall09 But...  Are we approaching limits of human control over techology?  Increasing complexity and uncertainty in system networks and learning systems –example Air Traffic Management –example Pervasive computing

Psychological Aspects of Risk Management and Technology – G. Grote ETHZ, Fall09 Design for (partial) non-controllability  Giving human operators information about the limits of control  Shifting accountability to system designers and operating organizations for handling limitations of operator control  Gaining control by giving up control ?

Psychological Aspects of Risk Management and Technology – G. Grote ETHZ, Fall09 Designing driverless cars  Develop a general scenario for introducing driverless cars  Describe the distribution of tasks in the scenario  Describe the distribution of control and accountability in the scenario