7 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 7: Hybrid Deliberative/Reactive Paradigm1 Part 1: Overview & Managerial Architectures Part 2: State Hierarchy Architectures

7 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 7: Hybrid Deliberative/Reactive Paradigm2 Objectives Describe the hybrid paradigm in terms of 1) SPA and 2) sensing organization Given a list of responsibilities, be able to say whether it belongs in the deliberative layer or in the reactive layer List the five basic components of a Hybrid architecture: sequencer agent, resource manager, cartographer, mission planner, performance monitoring and problem solving agent. Be able to describe the difference between managerial, state hierarchy, and model-oriented styles of Hybrid architectures. Be able to describe the use of state to define behaviors and deliberative responsibilities in state hierarchy styles of Hybrid architectures

7 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 7: Hybrid Deliberative/Reactive Paradigm3 Motivating Example for Deliberation: USAR Worker places robot at entrance to unstable building, loads in the floor plan, contextual knowledge and tells robot to look for survivors efficiently and map out safe routes for workers to pass through contextual knowledge includes probability of where people are more likely to be What can a reactive architecture do? What can’t it do? Path planning, handling detours due to blockage, map making, learn from past rescues

7 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 7: Hybrid Deliberative/Reactive Paradigm4 Organization: Plan, Sense-Act

7 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 7: Hybrid Deliberative/Reactive Paradigm5 Sensing Organization Deliberative functions *Can “eavesdrop” *Can have their own Sensors *Have output which Looks like a sensor Output to a behavior (virtual sensor)

7 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 7: Hybrid Deliberative/Reactive Paradigm6 Deliberation v. Reaction as a function of TIME Past, Present, Future Reactive –exists in the PRESENT (will a bit of duration) Deliberative –can reason about the PAST –can project into the FUTURE

7 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 7: Hybrid Deliberative/Reactive Paradigm7 Architectures:Key Questions How does the architecture distinguish between reaction and deliberation? How does it organize responsibilities in the deliberative portion? How does overall behavior emerge?

7 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 7: Hybrid Deliberative/Reactive Paradigm8 Architectures: Common Functionality Mission planner Cartographer Sequencer agent Behavioral manager Performance monitor/problem solving agent (fairly rare)

7 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 7: Hybrid Deliberative/Reactive Paradigm9 Architectures: 3 Styles Managerial (division of responsibilities looks like in business) –AuRA, SFX State Hierarchies (strictly by time scope) –3T Model-Oriented (models serve as virtual sensors) –Saphira, TCA

7 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 7: Hybrid Deliberative/Reactive Paradigm10 Mgr Architecture 1: AuRA (Autonomous Robot Arch.) Ron Arkin, Georgia Institute of Technology

7 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 7: Hybrid Deliberative/Reactive Paradigm11 AuRA Architectural Layout reactive deliberative

7 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 7: Hybrid Deliberative/Reactive Paradigm12 Architectures: Common Functionality Mission planner Cartographer Sequencer agent Behavioral manager Performance monitor/problem solving agent

7 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 7: Hybrid Deliberative/Reactive Paradigm13 AuRA Architectural Layout Cartographer Sequencer Mission Planner Behavioral manager (mgr+schemas) Performance Monitoring Emergent behavior

7 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 7: Hybrid Deliberative/Reactive Paradigm14 HOW WOULD THIS DO USAR TASK?

7 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 7: Hybrid Deliberative/Reactive Paradigm15 Example USAR (overlay) Cartographer accepts the map Navigator uses path planning algorithm to visit nodes in order of likelihood of survivors Pilot determines the list of behaviors, Motor Schema Manager instantiates them (MS & PS) and waits for termination Homeostatic might notice that robot is running out of power, so opportunistically picks up low probability room on way back to home

7 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 7: Hybrid Deliberative/Reactive Paradigm16 Mgr. Architecture 2: SFX (Sensor Fusion Effects) Focus on sensing Biomimetic organization deliberative layer consists of managerial agents reactive layer has tactical behaviors

7 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 7: Hybrid Deliberative/Reactive Paradigm17 SFX (Sensor Fusion Effects)

7 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 7: Hybrid Deliberative/Reactive Paradigm18 SFX (Sensor Fusion Effects) Behaviors (using direct perception, fusion) Sense Muscle Actuators Deliberative Layer Managers Sense Sensor Sense Receptive Field Choice of behaviors, resource allocation, motivation, context Focus of attention, recalibration Sensor Whiteboard Behavioral Whiteboard Deliberative Layer Reactive Layer Parameters to behaviors, sensor failures, task progress actions Superior Colliculus-like functions Cerebral Cortex-like functions Cartographer (model/map making) Recognition perception

7 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 7: Hybrid Deliberative/Reactive Paradigm19 Cartographer SFX Implementation Sensor Mgr Task Planner Interface Mgr Behaviors Sensors Acuators Effector Mgr Lisp C++ Behaviors Use, Fuse Information directs sensing HOW WOULD THIS DO USAR?

7 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 7: Hybrid Deliberative/Reactive Paradigm20 Example USAR (overlay) Cartographer accepts the map Task Planner agent asks for path, requests behaviors, passes to managerial layer Sensing and Effector Mgrs negotiate allocation Behaviors run until terminate or encounter exception (either preset condition by mgrs or through monitoring) Mgrs can see “below” but not above--cannot relax constraint of Planner/Boss

7 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 7: Hybrid Deliberative/Reactive Paradigm21 Tactical Behaviors

7 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 7: Hybrid Deliberative/Reactive Paradigm22 Summary: Managerial Architectures How does the architecture distinguish between reaction and deliberation? –Deliberation: global knowledge or world models, projection forward or backward in time –Reaction: behaviors which have some past/persistence of perception and external state How does it organize responsibilities in the deliberative portion? –hierarchy of managerial responsibility, managers may be peer software agents How does overall behavior emerge? –From interactions of a set of behaviors dynamically instantiated and modified by the deliberative layer –assemblages of behaviors

7 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 7: Hybrid Deliberative/Reactive Paradigm23 Chapter 7: Hybrid Deliberative/Reactive Paradigm Part 1: Overview & Managerial Architectures Part 2: State Hierarchy & Model-Based Architectures

7 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 7: Hybrid Deliberative/Reactive Paradigm24 State Hierarchy Architectures How does the architecture distinguish between reaction and deliberation? –Deliberation: requires PAST or FUTURE knowledge –Reaction: behaviors are purely reflexive and have only local, behavior specific; require only PRESENT How does it organize responsibilities in the deliberative portion? –By internal temporal state PRESENT (controller) PAST (sequencer) FUTURE (planner) –By speed of execution How does overall behavior emerge? –From generation and monitoring of a sequence of behaviors –assemblages of behaviors called skills –subsumption

7 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 7: Hybrid Deliberative/Reactive Paradigm25 3T Architecture Used extensively at NASA Merging of subsumption variation (Gat, Bonasso), RAPs (Firby), and vision (Kortenkamp) Has 3 layers –reactive –deliberative –in-between (reactive planning) Arranges by time Arranges by execution rate –ex. vision in deliberation Dave Kortenkamp, TRAC Labs (NASA JSC)

7 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 7: Hybrid Deliberative/Reactive Paradigm26

7 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 7: Hybrid Deliberative/Reactive Paradigm27 cartographer Mission planner sequencer Behavior mgr Performance monitor Emergent behavior

7 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 7: Hybrid Deliberative/Reactive Paradigm28 3T architecture Skills have associated events, to verify that an action had correct effect. Skills operate only in the Present, components of the sequencer layer operate on state information about the Past, as well as present,planner layer works state information about the Past and Present to plan the Future, slow algorithms are in the Planner, fast algorithms go into the Skill Manager Layer, so vision algorithms were placed in the Planner despite their low-level sensing functions.

7 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 7: Hybrid Deliberative/Reactive Paradigm29 Model-Oriented Architectures How does the architecture distinguish between reaction and deliberation? –Deliberation: anything relating a behavior to a goal or objective –Reaction: behaviors are “small control units” operating in present, but may use global knowledge as if it were a sensor (virtual sensor) How does it organize responsibilities in the deliberative portion? –Behavioral component –Model of the world and state of the robot –throwback to Hierarchical Paradigm with global world model but virtual sensors –Deliberative functions How does overall behavior emerge? –From generation and monitoring of a sequence of behaviors –voting or fuzzy logic for combination

7 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 7: Hybrid Deliberative/Reactive Paradigm30 Model-Oriented Architectures Top-down, symbolic flavor, symbolic manipulation around a global world model, world model supply perception with virtual sensors, different than the hierarchical model: - model restricted to labeling objects of interest like hallway, door, et, - perceptual processing is distributed and asynchronous - sensor errors and uncertaity can be filtered using sensor fusion over time to improve performance, - increase in processor speed and optimizing compilers solved the processing bottleneck

7 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 7: Hybrid Deliberative/Reactive Paradigm31 Saphira Architecture Developed at SRI by Konolige, Myers, Saffioti Comes with Pioneer robots Behaviors produce fuzzy outputs, fuzzy logic combines them Has a global rep called a Local Perceptual Structure to filter noise Instead of RAPs, uses PRS-Lite

7 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 7: Hybrid Deliberative/Reactive Paradigm32 Saphira and LPS

7 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 7: Hybrid Deliberative/Reactive Paradigm33 PRS - Procedural Reasoning System Reactivity - postponement of the elaboration of plans until it is necessary, plans are but determined continuously in reaction to the current situation, plans in progress can be interrupted and abandoned at any time, plans represent the robot’s desired behavior, a symbolic plan always drives the system,

7 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 7: Hybrid Deliberative/Reactive Paradigm34 Sequencer agent, Mission Planning, Performance mon. Cartographer Behavior mgr Emergent behavior

7 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 7: Hybrid Deliberative/Reactive Paradigm35 Saphira Three tenets of successful mobile robot architecture: - coordination of robot’s sensors, actuators and its goals over time ( not adressed by the reactive architecture), - coherence - ability of the robot to maintain global information about its world - essential for good behavioral performance and interaction with humans, - ability to community with other robots and humans deliberation activities distributed among software agents, some of them running on a local workstation, reactive components consists of behaviors, behaviors extract virtual sensor inputs from the Local Perceptual Space, behavioral output is fuzzy rules, which are fused together using fuzzy logic into a velocity and steer commands,

7 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 7: Hybrid Deliberative/Reactive Paradigm36 Task Control Architecture Developed by Reid Simmons Used extensively by CMU Field Robotics Projects –NASA’s Nomad, Ambler, Dante Closest to Hierarchical in philosophy, but strong reactive theme showing up in implementation

7 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 7: Hybrid Deliberative/Reactive Paradigm37 TCA closer to an operating system architecture, tasks instead of behaviors, uses dedicated sensing structures called evidence grids corresponding to a distributed global world model, Task Scheduling Layer (using Prodigy planner) determines the task flow, interacts with the user, determines the goals and order of execution, navigation handled by a Partially Observable Markov Decision Process (POMDP), Obstacle Avoidance Layer uses a curvature-velocity method to factor not only obstacles but how to respond with a smooth trajectory

7 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 7: Hybrid Deliberative/Reactive Paradigm38 TCA

7 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 7: Hybrid Deliberative/Reactive Paradigm39 Other Hybrid Architectures SSS - Servo Subsumption Symbolic - 3 layers, - world models are a convenience not a necessity, - symbolic (deliberative) layer switches behaviors on and off and provides parameters - symbolic layer - where to go next (strategic) - subsumption - where to go now (tactical) SOMASS Hybrid Assembly System - cognitive (deliberative)/subcognitive (reactive) - planner as ignorant as possible, hierarchical, - subcognitive components execute plan downloaded to the robot, - planning as configuration.

7 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 7: Hybrid Deliberative/Reactive Paradigm40 Other Hybrid Architectures Agent Architecture - plans are descriptions of intended behaviors, - two levels: physical (perception and action), cognitive (higher level reasoning), - functional boundary is blurred - reactive components can coexist within each level. Theo Agent - reacts when it can, plans when it must, - focusses on learning - how to become more reactive, more correct and more perceptive, - rule selection and execution, - when no rule available, new rule added by the planner

7 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 7: Hybrid Deliberative/Reactive Paradigm41 Evaluation of Hybrids Support of Modularity: high Niche targetability: high (ex. Lower levels of AuRA, SFX, 2 1/2 T is just reactive) Robustness: SFX and 3T explicitly monitor performance of the reactive behaviors and either replace or adapt the configuration as needed, better comply with the software engineering principles, global world models used only for sumbolic functions, frame problem reduced by the principle - “Think in closed world, act in open world” planners produce only partial plans asynchronously with the reactos

7 Introduction to AI Robotics (MIT Press), copyright Robin Murphy 2000 Chapter 7: Hybrid Deliberative/Reactive Paradigm42 Hybrid Summary P,S-A, deliberation uses global world models, reactive uses behavior-specific or virtual sensors Architectures generally have modules for mission planner, sequencer, behavioral mgr, cartographer, and performance monitoring Deliberative component is often divided into sub-layers (sequencer/mission planner or managers/mission planner) Reactive component tends to use assemblages of behaviors Roots in ethology and a framework for exploring cognitive science.