A Case for Theory-Based Research on Level of Automation and Adaptive Automation David B. Kaber Department of Industrial Engineering North Carolina State.

Slides:

Advertisements

Similar presentations

Pat Langley Computational Learning Laboratory Center for the Study of Language and Information Stanford University, Stanford, California

Advertisements

Dynamic Decision Making Lab Social and Decision Sciences Department Carnegie Mellon University 1 MODELING AND MEASURING SITUATION AWARENESS.

1 A Methodological Approach to IS Research Presented by: Dr. Jay Nunamaker, Jr.

SECOND MIDTERM REVIEW CS 580 Human Computer Interaction.

Experiential Learning How People Learn: The Experiential Learning Model How do you learn?

Human Supervisory Control Human-Centered Systems Engineering Design Approaches Massachusetts Institute of Technology.

Relationships between Involvement and Use in the Context of Multi-site Evaluation American Evaluation Association Conference November 12, 2009.

The Art and Science of Teaching (2007)

1 Challenge the future The Dutch Automated Vehicle Initiative: Challenges for automated driving Dr. R.(Raymond) G. Hoogendoorn Assistant Professor Delft.

Automation and Ability Benjamin A. Clegg Department of Psychology Colorado State University Eric D. Heggestad Department of Psychology University of North.

Robert J. Mislevy & Min Liu University of Maryland Geneva Haertel SRI International Robert J. Mislevy & Min Liu University of Maryland Geneva Haertel SRI.

Effects of Automation in the Aircraft Cockpit Environment

To delegate or not to delegate: A human factors perspective of autonomous driving Dale Richards, Coventry University Presentation to: European Conference.

International Journal of Industrial Ergonomics 35 (2005) 939–953 Situation awareness and workload in driving while using adaptive cruise control and a.

Lecture 13 Revision IMS Systems Analysis and Design.

DEVELOPMENTAL RESEARCH  Mitch Kielb  Juanita Tilgner  Andy Sen.

Action Research Teacher as a Researcher Research & Action Research Manoj Kumar Dash DEP-SSA, IGNOU New Delhi

NASA Langley Core Competency Directors' Visit, September 4, 2003 Aspects of Adaptive Automation Support of Air Traffic Controllers David B. Kaber, Ph.D.

Chapter 4 Principles of Quantitative Research. Answering Questions  Quantitative Research attempts to answer questions by ascribing importance (significance)

The design process z Software engineering and the design process for interactive systems z Standards and guidelines as design rules z Usability engineering.

Usability Inspection of the MD-11 Aircraft Multifunctional Control Display Unit Kheng-wooi Tan and Jennifer M. Riley Mississippi State University.

Presentation for the Class of 2007

Margaret J. Cox King’s College London

ECD in the Scenario-Based GED ® Science Test Kevin McCarthy Dennis Fulkerson Science Content Specialists CCSSO NCSA June 29, 2012 Minneapolis This material.

Chapter 8 Architecture Analysis. 8 – Architecture Analysis 8.1 Analysis Techniques 8.2 Quantitative Analysis  Performance Views  Performance.

VTT-STUK assessment method for safety evaluation of safety-critical computer based systems - application in BE-SECBS project.

Inquiry Based Science Education The ESTABLISH project has received funding from the European Union’s Seventh Framework Programme [FP7/ ] under.

Service-Learning to Enhance Academic Achievement Shelley H. Billig Stephany Brown RMC Research Corporation.

LEARNING THEORIES ETEC 5300 By Jennifer Massey. What You Will Learn  Definitions to learning theories  Implications of theories in classroom  Discussion.

1 Drama in the Social Studies Dr. Tonja L. Root Early Childhood & Reading Education Valdosta State University Valdosta, GA

Chapter 4 Information, Management, and Decision Making.

MODULE 3 INVESTIGATING HUMAN AND SOCIL DEVELOPMENT IN THE CARIBBEAN.

HCI in Software Process Material from Authors of Human Computer Interaction Alan Dix, et al.

Implementation and process evaluation: developing our approach Ann Lendrum University of Manchester Neil Humphrey University of Manchester Gemma Moss Institute.

A Design Science (Multi-Methodological) Approach to IS Research Presented by: Dr. Jay F. Nunamaker, Jr. 1.

Human Supervisory Control Issues in Unmanned Vehicle Operations

URBDP 591 I Lecture 3: Research Process Objectives What are the major steps in the research process? What is an operational definition of variables? What.

MINISTRY OF EDUCATION AND SCIENCE OF UKRAINE NATIONAL AVIATION UNIVERSITY Air Navigation System Department.

1 IE 590D Applied Ergonomics Lecture 26 – Ergonomics in Manufacturing & Automation Vincent G. Duffy Associate Prof. School of IE and ABE Thursday April.

Situation Awareness & Levels of Automation David Kaber and Kheng-Wooi Tan Mississippi State University.

Ecological Interface Design in Aviation Domains Improving Pilot Trust in Automated Collision Detection and Avoidance Advanced Interface Design Laboratory.

Teaching to the Standard in Science Education By: Jennifer Grzelak & Bonnie Middleton.

Crysten Caviness Curriculum Management Specialist Birdville ISD.

EVALUATION OF HRD PROGRAMS Jayendra Rimal. The Purpose of HRD Evaluation HRD Evaluation – the systematic collection of descriptive and judgmental information.

Comparing Political Systems. Why Compare To develop perspective on the mix of constants and variability which characterize the world’s governments and.

1 The Theoretical Framework. A theoretical framework is similar to the frame of the house. Just as the foundation supports a house, a theoretical framework.

Nursing research Is a systematic inquiry into a subject that uses various approach quantitative and qualitative methods) to answer questions and solve.

Cognitive Engineering Perspective of ASAS Amy Pritchett Cognitive Engineering Center School of Aerospace Engineering Georgia Tech Atlanta, GA.

Theme 2: Data & Models One of the central processes of science is the interplay between models and data Data informs model generation and selection Models.

Developing the theoretical and conceptual framework From R.E.Khan ( J199 lecture)

Toulouse, September 2003 Page 1 JOURNEE ALTARICA Airbus ESACS  ISAAC.

1 ISE 412 ATTENTION!!! From page 147 of Wickens et al. ATTENTION RESOURCES.

U SER I NTERFACE L ABORATORY Situation Awareness a state of knowledge, from the processes used to achieve that state (situation assessment) not encompass.

Ch 10 Methodology.

Human-Centered Systems Background People play a critical role in the safety, reliability and performance of NASA systems. Their creativity, adaptability.

Aptitude, Levels of Automation, and the Effectiveness of Training Benjamin Clegg Department of Psychology Colorado State University Eric Heggestad Department.

Landscape Animation Summary of an experimental approach to inform the use of camera dynamics in visualization.

National PE Cycle of Analysis. Fitness Assessment + Gathering Data Why do we need to asses our fitness levels?? * Strengths + Weeknesses -> Develop Performance.

M I T I n t e r n a t i o n a l C e n t e r f o r A i r T r a n s p o r t a t i o n Impact of Operating Context on the Use of Structure in Air Traffic.

Michelle Hester Advanced Human Factors (PSYC 562)

Learning Analytics isn’t new Ways in which we might build on the long history of adaptive learning systems within contemporary online learning design Professor.

Investigate Plan Design Create Evaluate (Test it to objective evaluation at each stage of the design cycle) state – describe - explain the problem some.

Using Animated videos in graduate schools Role of Animated Videos On the Learning Process For Graduate Students Aaliah s Alhissan LRC: Teacher Research.

Copyright © Springer Publishing Company, LLC. All Rights Reserved. EVIDENCE-BASED TEACHING IN NURSING – Chapter 15 –

Principles of Quantitative Research

Management of Multiple Dynamic Human Supervisory Control Tasks

Usability Techniques Lecture 13.

Human Performance Contributions to Safety in Commercial Aviation

Motion Cueing Standards for Commercial Flight Simulation

16. Automation B737 Max 8.

Presentation transcript:

A Case for Theory-Based Research on Level of Automation and Adaptive Automation David B. Kaber Department of Industrial Engineering North Carolina State University This work was supported by a grant through the National Aeronautics and Space Administration Langley Research Center. The opinions expressed are those of the author and do not necessarily reflect the views of NASA.

Thoughts on the “what”, “how” and “when” of automation: “What?” “How?” “When?” Monitoring Generating Selecting Implementing Human Human/Machine Machine Stages of Information Processing Level of Automation (LOA) Adapted from Endsley (1997). TnTn T0T0 Adaptive Automation (AA)

Information Processing and Automation Design Theory: Qualitative models relating stages of information processing to automation design. – Parasuraman et al. (2000) – Model of types and levels of automation. A model of types and levels of automation (from Parasuraman et al. (2000)). Information Acquisition Information Analysis Decision- making Action Degree of Automation High Low High Low High Low High Low

Qualitative Models of Human- Automation Interaction (cont.): Endsley & Kaber (1999): – Four-stage model of information processing. – Used to define levels of automation or function allocation schemes. MonitoringGeneratingSelectingImplementing Human Human/Machine Machine Level of Automation Human Human/Machine Machine Human Human/Machine Machine Human Human/Machine Machine Adapted from Endsley and Kaber’s (1999) taxonomy of levels of automation.

Taxonomy of Levels of Automation: Endsley & Kaber (1999): – 10 discrete levels of automation (LOAs) in complex systems control.

Bases for Qualitative Models: Common models of HUMAN information processing (HIP). – Sanders & McCormick (1993) – Multi-stage model identifying functions of humans and machines. Adapted from Sanders and McCormick’s (1993) model of human and machine functions in human-machine systems. Sensing - Perception Information Processing Decision- making Action Functions Feedback Automaticity

Another Basis for Current Models: Wickens (1992) – Model of HIP influential in Parasuraman et al. (2000) model of types and LOAs. PerceptionDecision- making Response Execution Working Memory Long-term Memory Short-term Sensory Store Feedback Model of human information processing (adapted from Wickens (1992)).

Applying Information Processing Models to Automation Design: – Present ways of classifying functions of human-machine systems. Note: Entire process is similar to Wickens (1992) approach to systems design. Mission Analysis Function Analysis Task Analysis Basic System Concept Initial Function Allocation Dynamic Function Allocation Determine “what” to automate? Determine “how” to automate? Determine “when” to automate? Parasuraman et al. (2000) and Endsley & Kaber (1999) models:

Unique Aspects of Approach: Link LOAs to stages of information processing. Types of functions considered are general information processing functions. – Compare with Sheridan & Verplank’s (1978) hierarchy of LOAs – Functions primarily represent action states (“gets”, “starts”, etc.) or choice reactions (“selects”). Historical function allocation lists technology-centered (Fitts, 1956): – Advances in understanding of out-of-the (control) loop performance (Endsley & Kiris, 1995; Kaber et al., 1998) shifted focus of lists on human abilities. – Contemporary taxonomies of LOA (e.g., Endsley & Kaber (1999)) developed by considering performance consequences of automation Complacency, vigilance decrements, loss of SA, skill decay.

Importance of Research: Link general theories on information processing and automation to applications. – Parasuraman et al. (2000) - Application of model to air traffic control (ATC). Contemporary models of human-automation interaction (HAI) may serve as design rationale for human-centered automation. – Knowledge of performance with systems can be classified according to model and used to further develop general theory on HAI. Theory may serve to answer several questions: – What stages of information processing are conducive to automation from human perspective? – To what extent can stage be automated safely/effectively? – What stages are robust to automation reliability problems?

Theoretical Research Challenges: Validate processing stages of models in terms of representation of actual functions performed by human-machine systems. Models can be used for logical function allocation, but practical implications are difficult to predict. – Endsley & Kaber (1999): Intermediate LOAs moderate workload, maintain situation awareness (SA) and improve performance. Low level automation produced superior performance, but at cost of SA. “Good” SA observed under high LOAs. Hypothesis Results

Example Model Application: Applied Endsley & Kaber’s taxonomy of LOAs to advanced commercial aircraft (MD-11) (Tan et al., 2000): –Conducted complex systems analysis. –Developed high-fidelity simulation for evaluation of existing system automation. –Categorized actual modes of automation according to taxonomy.

Flight Control Panel (FCP) Out-of-Cockpit View Flight Mode Annunciator (FMA) Primary Flight Display (PFD) Navigation Display (ND) Multi-functional Control Display Unit (MCDU) Throttle/attitude control Flight Simulator Displays:

Defining Aircraft Automation in Terms of Taxonomy of LOA: Taxonomy and example LOA description:

2 Study AA by considering many points along continuum of automation. 1 Research to Enhance Basic Understanding of Automation: Adaptive automation (AA) (or “when” to automate) primarily studied using laboratory simulations of complex systems. Majority of AA research investigates concept from binary perspective. – Automation is continuous variable made discrete for research purposes. Automation Full Automation Manual Control “When?” TnTn T0T0 Decision Support Batch Processing Decision Support Supervisory Control Manual Control Action Support Binary Approach Continuous Approach Study AA in high- fidelity simulations, or in context.

3 4 5 Considering Models of HAI in Adaptive Systems Research: Focus of AA research has been on early stages of information processing: – Do not have understanding of effect of AA on decision-making and planning. – People may not adapt well to dynamic allocations of decision functions. Research has not systematically examined performance and SA effects of dynamic function allocations (DFAs) across stages of information processing. Study impact of AA on latter stages of information processing. Study SA and performance effects of AA across functions represented in models of HAI. Study impact of AA applied exclusively to each stage of information processing. Parasuraman et al. (1993)

7 Consider need to keep-track of LOAs in AA studies. 6 Compare voluntary, involuntary and shared DFA management Final Issues: – Computer authority shown to reduce excessive cyclings between control modes (Hilburn et al., 1993). – Automation directed DFAs improve human manual control – operator does not need to evaluate who should be doing what (Kaber & Riley, 1999). Approach AA research cautiously - complexity of system design may cause mode awareness problems (Sarter & Woods, 1995). Need to determine who (human or automation) should have authority over DFAs:

Conclusions: Can’t study AA without considering LOA – concepts “like peas and carrots” (F. Gump). Taxonomies of LOA provide means for systematically studying complexities of automation. Need to apply models of HAI to real systems to expand theory to design rationales. – Classify systems and understand underlying factors in automated system performance. Need to develop framework of AA research.