1. Architecture for Autonomous Assembly 1 Reid Simmons Robotics Institute Carnegie Mellon University

2. Architecture for Autonomous Assembly 2 Prior Work: Multi-Robot Assembly Heterogeneous Coordinated Sliding Autonomy

3. Architecture for Autonomous Assembly 3 Overall Approach Utilize our previous work in robot autonomy –Multi-layered software architecture –Hierarchical, task-level description of assembly –Robust, low-level behaviors –Distributed visual servoing –Force sensing –Exception detection and recovery

4. Architecture for Autonomous Assembly 4 Architectural Framework Deals with goals and resource interactions Task decomposition; Task synchronization; Monitoring; Exception handling Deals with sensors and actuators Three-Tiered Architecture Reactive & Deliberative Modular Control loops at multiple levels of abstraction Executiv e Behavioral Control Planning

5. Architecture for Autonomous Assembly 5 Syndicate: Multi-Robot Architecture Synchronization / Coordination Granulatity Executiv e Behavioral Control Planning Executiv e Behavioral Control Executiv e Behavioral Control Planning

6. Architecture for Autonomous Assembly 6 Advantages of Tiered Approach Control loops at multiple levels of abstraction Integrates reactive and deliberative mechanisms Managed complexity Representations tuned to specific objectives “Plans as Advice”

7. Architecture for Autonomous Assembly 7 Why Multi-Robot Architecture? Modularity –Mobile manipulator agent –Perception agent –Coordinator agent Efficiency –Promotes concurrency –Can spread over multiple computers Reuse Existing Algorithms and Software

8. Architecture for Autonomous Assembly 8 Behavioral Layer “Skill Manager” –Skills are concurrent behaviors, including perceptual behaviors –Dynamic creation of real-time feedback loops Higher tier (executive) connects sensing and action modules and enables subsets of skills Skills indicate status by passing signals back to executive –Enables flexible (and distributed) feedback control loops (e.g., visual servoing) Virtual SensorsAction Behaviors Signals Configure Actions Arbiter

9. Architecture for Autonomous Assembly 9 Distributed Visual Servoing Mast Eye tracks fiducials –Uses ARTag software package to detect fiducials –Provides 6-DOF transform between fiducials Mobile Manipulator uses information to plan how to achieve goal –Use data base describing positions of fiducials on objects Behavioral layer enables dynamic, transparent inter-agent connections Mobile Manipulator VisualServo Relative positions ArmControl End effector delta Mast Eye Tracking Images (via cameras) The World Manipulate environment (via arm)

10. Architecture for Autonomous Assembly 10 Distributed Visual Servoing Fairly precise –millimeter resolution at one meter Relatively fast –3-4 Hz Basically unchanged from Trestle code Operates in relative frame –Poses of one object relative to another –Controller continually tries to reduce pose difference –Cameras do not need to be precisely calibrated with respect to base or arm

11. Architecture for Autonomous Assembly 11 Executive Layer Forms a Bridge Between Planning and Behaviors –Discrete vs. continuous control –Symbolic vs. numeric representations –Real-time considerations Basic Roles –Decompose task into subtasks and dispatch tasks –Monitor execution for contingencies and opportunities –Reschedule tasks (or schedule new tasks) upon failure

12. Architecture for Autonomous Assembly 12 TDL (Task Description Language) High-Level Language Tailored to Task-Level Control –Extension of C++ with explicit syntax for task-level control constructs –Syntax for task decomposition, sequencing, monitoring, error handling –Compiles into pure C++ with calls to task management library Requirements –Simple concepts should be expressible in simple terms –Do not preclude expression of complex control constructs –Natural integration with existing code

13. Architecture for Autonomous Assembly 13 Task Trees – Deliberative TDL Maintains and Coordinates Task Trees –Execution trace of hierarchical plans Created dynamically at run time Can be conditional and recursive –Temporal constraints (partially) order task execution –Planning and sensing treated as schedulable activities; Concurrent planning, sensing, and execution

14. Architecture for Autonomous Assembly 14 Task Trees – Reactive Task Trees Augmented with Reactive Elements –Task-specific execution monitors –Context-dependent, hierarchical exception handlers Replan by Analyzing and Manipulating Task Trees –Terminate subtrees –Add new nodes and/or temporal constraints

15. Architecture for Autonomous Assembly 15 Ace Task Decomposition Child link Serial Constraint

16. Architecture for Autonomous Assembly 16 Example TDL Code (somewhat simplified) Goal ClipInsertion ( ) { loadPlugArmPose : spawn ArmMove (loadPose); stowArmPose: spawn ArmMove (stowPose) WITH SERIAL loadPlugArmPose; roughBaseMove: spawn RoughBaseMove (roughBaseWaypoint) WITH SERIAL loadPlugArmPose; spawn RoughArmMove (roughArmWaypoint) WITH SERIAL roughBaseMove, SERIAL stowArmPose; … } A keyword that says this is not supposed to be a “leaf” in the task tree Reusing task with different parameters In the tree, this is the task name, but this is the actual function being executed Tells the system to execute this task after LoadPlugArmPose completes Wait until both tasks have completed before starting RoughArmMove

17. Architecture for Autonomous Assembly 17 Monitoring & Error Recovery Monitoring –Detect exceptions: significant differences between expectations and the state of the world External Events (contingencies, opportunities) Internal Events (sensors, actuators, computers) Task Status (preconditions, progress, postconditions) Diagnosis –Determine cause of exception Error Recovery –Reconcile expectations and world state Change world state (repair) Alter expectations (replan)

18. Architecture for Autonomous Assembly 18 Monitoring Detect Completion of Tasks –Success conditions Detect Failure Conditions and Lack of Progress Need to Consider All Possible Situations –Important to try to cover all failure scenarios –Can handle hierarchically Monitor for very general conditions as well as specific conditions

19. Architecture for Autonomous Assembly 19 Failure Recovery Executive ArmPlaceBeam GraspBeamDockBeam Spawn Serialize Behavioral VisualServo Activate LostBeam Activate Failure

20. Architecture for Autonomous Assembly 20 What About the Planning Layer? Planning layer creates task trees online –Based on knowledge of the system –Example: New task to be done. Planner knows which robots are available/capable to achieve it. Spawn the task on the desired robot. Our tasks are straightforward –Number and type of robots is known We hard-code our own task tree –One super-task spawns the main tree