1. Intelligent Systems Branch Johnson Space Center Engineering Directorate Automation, Robotics and Simulation Division Hierarchical Architectures for Controlling Real-World Agents David Kortenkamp Pete Bonasso NASA Johnson Space Center Metrica Inc./TRACLabs kortenkamp@jsc.nasa.gov

2. Intelligent Systems Branch Johnson Space Center Engineering Directorate Automation, Robotics and Simulation Division Robot Architectures Sense  Plan  Act: Shakey 1969 Subsumption: Brooks 1986 Want power of deliberation and speed of reactivity Hybrid architectures [Erann Gat “Three Layer Architectures” in Artificial Intelligence and Mobile Robots 1998]

3. Intelligent Systems Branch Johnson Space Center Engineering Directorate Automation, Robotics and Simulation Division 3T World 1/10 sec. limited state limited projection memory interpreter task1 subtask task2 task3 1 sec. memory of immediate actions no projection 10s sec. persistent state and choices projection

4. Intelligent Systems Branch Johnson Space Center Engineering Directorate Automation, Robotics and Simulation Division Multi-agent path planner simulation IVHM scheduler Planner

5. Intelligent Systems Branch Johnson Space Center Engineering Directorate Automation, Robotics and Simulation Division 3T at JSC Particular instantiation of hybrid architecture –Skill manager –RAPS –Adversarial Planner Robotic and non- robotic applications User interaction

6. Intelligent Systems Branch Johnson Space Center Engineering Directorate Automation, Robotics and Simulation Division Skills Small pieces of C code that do something Data passed from one skill to another Actions and events Skill manager –Handles scheduling of skills –Handles data transfer between skills –Handles comms with second tier sensor actuator to RAPS

7. Intelligent Systems Branch Johnson Space Center Engineering Directorate Automation, Robotics and Simulation Division RAPS Jim Firby’s Reactive Action Packages [Firby 1989] Library of tasks Conceptual memory Run-time decomposition of tasks via context Tasks bottom-out in activation of skills Agenda task sub-task primitive wait-for RAP Memory RAP Library RAP Interpreter to/from skills

8. Intelligent Systems Branch Johnson Space Center Engineering Directorate Automation, Robotics and Simulation Division Example RAP (define-rap (bioreactor-startup ?configuration) (language-mappings (applicability (and (isa ?object i-bwp-system) (or (isa ?state i-stand-alone-drain-state) (isa ?state i-on) (isa ?state i-stand-alone-reject-state) (isa ?state i-integrated-state) (isa ?state i-innoc-no-feed-state)))) (method integrated-or-stand-alone (context (and (not (= ?configuration innoc-no-feed)) (manual-valve-position pbbwpskm bio_innoc ?value ?error) (= ?value open) (default-timeout ?to) (nominal-pump-speed pbbwpskm recycle ?nmrc))) (task-net (sequence (t0 (stop-watchdog pbbwpskm)) (t0-1 (kill-monitors monitor-bioreactor)) (t1 (configure-bwp-valves ?configuration)) (t2 (set-pump-speeds start) (update-bwp-text ?configuration)) (t2-2 (start-watchdog pbbwpskm)) (t3 (install-goal (quote (monitor-bioreactor ?configuration)) 3 => ?mgoal)))))

9. Intelligent Systems Branch Johnson Space Center Engineering Directorate Automation, Robotics and Simulation Division Language in RAPS

10. Intelligent Systems Branch Johnson Space Center Engineering Directorate Automation, Robotics and Simulation Division Adversarial Planner (AP) Non-linear, hierarchical task net (HTN) planner developed at MITRE Multi-agent Some scheduling Leaves of plan are matched to top-level RAPS in RAP library –RAPS provides abstraction from low-level for planner Execution monitoring

11. Intelligent Systems Branch Johnson Space Center Engineering Directorate Automation, Robotics and Simulation Division AP Example (Operator replace-crystals :purpose (state crystals replaced) :agents (?robot ?human) :constraints ((instance-of ?robot ‘robot) (instance-of ?human ‘human)) :preconditions ((state bay opened)) :plot (sequential (covers (monitor-crystals ?robot in-monitor) (display ?human monitored)) (crystals ?robot are-replaced)) :effects ((state crystals replaced)))

12. Intelligent Systems Branch Johnson Space Center Engineering Directorate Automation, Robotics and Simulation Division Integrated Application Four skill managers RAPS (100s of RAPS > 50,000 lines of Lisp code) Scheduler 24/7 autonomous operation 18 months of operation AI Magazine next issue

13. Intelligent Systems Branch Johnson Space Center Engineering Directorate Automation, Robotics and Simulation Division Skills ------------------------------------------------------- Skills -- for the RO agent ---------------------------------- Name RO Type device Params interval Outs none Function: A device skill that gets all the sensor values and provides them to the other skills. Also sends commands to the pumps and valves. Also every interval seconds, this skill broadcasts a data message with the values of all the channels listed above to the IPC server so that clients (e.g., a logging facility) can access them (see the IPC structure at the end of this document). Name valve_position Type query Params valve (process/pps_select) Outs value (for process:primary/secondary/purge/off/unknown; for pps_select:pps/tank/reject/off/unknown), and result (okay or Err) Function: Checks V02 or V03. One of lines V02_i1 through V02_i3 or V03_i1 through V03_i3 will be hi, and the rest will be low. If all are low, the result is off. Any other pattern is unknown. Name valve_at Type event Params valve (process/pps_select), value (for process:primary/secondary/purge/off; for pps_select:pps/tank/reject/off) Outs result (okay/ERR) Function: Waits for V02_i1 through V02_i3 or V03_i1 through V03_i3 to indicate value (see the valve_position skill). When the condition is achieved the event returns result. Name turn_valve Type block Params valve (process/pps_select), value (for process:primary/secondary/purge/off; for pps_select:pps/tank/reject/off) Outs none Function: Sets one of V02_o1 through V02_03 to hi the rest to low, except for off when all lines will be set lo.

14. Intelligent Systems Branch Johnson Space Center Engineering Directorate Automation, Robotics and Simulation Division RAPs (define-primitive-event (valve-at ?agent ?valve ?open-closed ?error) (event-definition (:valve_at (:valve. ?valve) (:value. ?open-closed))) (event-values :bound :bound :bound :unbound)) (define-rap (turn-valve-p ?agent ?valve ?open-closed ?timeout) (succeed (and (valve-position ?agent ?valve ?value ?error) (= ?value ?open-closed))) (timeout ?timeout) (method (primitive (enable (:turn_valve (:valve. ?valve) (:value. ?open-closed)) (wait-for (valve-at ?agent ?valve ?open-closed ?result) :succeed (?result)) (disable :above) )) ) (define-rap (processing-start ?stage ?adjust-time)... (method purge (context (and (= ?stage purge) (valve-position roskm pps_select ?old-pos ?error) (= ?old-pos pps) (nominal-pump-speed roskm feed ?wwsp) (default-timeout ?dto))) (task-net (sequence (t1 (syringe-pump-p roskm start feed ?wwsp 30)) (t2 (water-flowing-p roskm stop recirc 0 ?dto)) (t3 (turn-valve-p roskm pps_select reject ?dto)) (t4 (turn-valve-p roskm process purge ?dto)) (t5 (turn-valve-p roskm pps_select tank ?dto)))))

15. Intelligent Systems Branch Johnson Space Center Engineering Directorate Automation, Robotics and Simulation Division Scheduler

16. Intelligent Systems Branch Johnson Space Center Engineering Directorate Automation, Robotics and Simulation Division Learning in 3T Intra-tier learning Inter-tier learning –Planning  RAPS –RAPS  behaviors Automatic activation of behaviors –Behaviors  RAPS –RAPS  Planning

17. Intelligent Systems Branch Johnson Space Center Engineering Directorate Automation, Robotics and Simulation Division Motivation in 3T Goals and motivations come from outside or are pre-built for the specific application –Ideally motivations would come into the top layer and then “trickle down” to the other layers –However, often useful to manually place goals on the RAPS agenda via GUI or NL –Less useful to enable/disable skills except for testing Resource allocation mostly at top-level, some mechanisms in RAPS to handle contention for limited resources

18. Intelligent Systems Branch Johnson Space Center Engineering Directorate Automation, Robotics and Simulation Division Adjustable Autonomy Allows system to operate with dynamically varying levels of independence, intelligence and control Some levels of control –Completely in control –Shared control –Supervising manual control System still updates state, world model, etc. through its sensors

19. Intelligent Systems Branch Johnson Space Center Engineering Directorate Automation, Robotics and Simulation Division sensors perceptual memory local, short-term, dynamic robot-centric: where, identity, role object ontology global, long-term, static allo-centric: relationships, roles, recognition, etc. DAML/OIL task-level control how to do things, procedures, monitors, etc. control multi-agent, distributed, real-time closed-loop actuators Representation Reasoning about world Reasoning about task spatial reasoning learning task planning object recognition context objectspriming color, neural networks stereo vision, etc. World Generating task sequences, monitoring resources temporal reasoning contextual reasoning goals set points