1. Autonomy for General Assembly Reid Simmons Research Professor Robotics Institute Carnegie Mellon University

2. Autonomy for General AssemblyCarnegie Mellon2 The Challenge Autonomous manipulation of flexible objects for general assembly of vehicles –Dexterity –Precise perception –Speed –Reliability The Specific Task –Insert clip attached to cable into hole with millimeter tolerance –Year 2: Moving taskboard

3. Autonomy for General AssemblyCarnegie Mellon3 Overall Approach Utilize our previous work in robot autonomy –Multi-layered software architecture –Hierarchical, task-level description of assembly –Robust, low-level behaviors –Distributed visual servoing –Force sensing –Exception detection and recovery

4. Autonomy for General AssemblyCarnegie Mellon4 Architectural Framework Deals with goals and resource interactions Task decomposition; Task synchronization; Monitoring; Exception handling Deals with sensors and actuators Three-Tiered Architecture Reactive & Deliberative Modular Control loops at multiple levels of abstraction Executiv e Behavioral Control Planning

5. Autonomy for General AssemblyCarnegie Mellon5 Syndicate: Multi-Robot Architecture Synchronization / Coordination Granulatity Executiv e Behavioral Control Planning Executiv e Behavioral Control Executiv e Behavioral Control Planning

6. Autonomy for General AssemblyCarnegie Mellon6 Syndicate Layers: Behavioral Made up of “blocks” –Each block is a small thread/function/process –Represent hardware capabilities or repeatable behaviors –“Stateless”: relies on current data; no knowledge of past or future Communicate with sensors Send commands to robots and get feedback Communicate data to other blocks

7. Autonomy for General AssemblyCarnegie Mellon7 Ace Control Behaviors

8. Autonomy for General AssemblyCarnegie Mellon8 Distributed Visual Servoing Mast Eye tracks fiducials –Uses ARTag software package to detect fiducials –Provides 6-DOF transform between fiducials Mobile Manipulator uses information to plan how to achieve goal –Use data base describing positions of fiducials on objects Behavioral layer enables dynamic, transparent inter-agent connections Mobile Manipulator VisualServo Relative positions ArmControl End effector delta Mast Eye Tracking Images (via cameras) The World Manipulate environment (via arm)

9. Autonomy for General AssemblyCarnegie Mellon9 Distributed Visual Servoing Fairly precise –millimeter resolution at one meter Relatively fast –3-4 Hz Basically unchanged from Trestle code Operates in relative frame –Poses of one object relative to another –Controller continually tries to reduce pose difference –Cameras do not need to be precisely calibrated with respect to base or arm

10. Autonomy for General AssemblyCarnegie Mellon10 Distributed Visual Servoing Associating Fiducials with Objects –Programmer provides file listing the pose of each fiducial with respect to an object –Multiple fiducials can be associated with each object –Can measure directly, or use system to give us the poses Reducing Pose Differences –“Waypoint” is the pose of one object with respect to another Everything is relative! –Visual servo block multiplies pose difference by gain –Update moves when new information arrives

11. Autonomy for General AssemblyCarnegie Mellon11 Syndicate Layers: Executive Made up of “tasks” –Each task is concerned with achieving a single goal –Tasks can be arranged temporally Tasks can: –Spawn subtasks –Enable and connect blocks in the behavioral layer to achieve the task Enable  tell a block to start running Connect  tell blocks to send data to other blocks –Monitor blocks for failure –Provide failure recovery

12. Autonomy for General AssemblyCarnegie Mellon12 Ace Task Decomposition Child link Serial Constraint

13. Autonomy for General AssemblyCarnegie Mellon13 Example TDL Code (somewhat simplified) Goal ClipInsertion ( ) { loadPlugArmPose : spawn ArmMove (loadPose); stowArmPose: spawn ArmMove (stowPose) WITH SERIAL loadPlugArmPose; roughBaseMove: spawn RoughBaseMove (roughBaseWaypoint) WITH SERIAL loadPlugArmPose; spawn RoughArmMove (roughArmWaypoint) WITH SERIAL roughBaseMove, SERIAL stowArmPose; … } A keyword that says this is not supposed to be a “leaf” in the task tree Reusing task with different parameters In the tree, this is the task name, but this is the actual function being executed Tells the system to execute this task after LoadPlugArmPose completes Wait until both tasks have completed before starting RoughArmMove

14. Autonomy for General AssemblyCarnegie Mellon14 Initial Results (December 2007) Used Previous Hardware –RWI base –Metrica 5 DOF arm –Metal & plastic gripper Successfully Inserted Clip –60% success rate (15 trials) Mainly attributable to hardware problems –Fairly slow (~1 minute) –Scripted base move

15. Autonomy for General AssemblyCarnegie Mellon15 Insertion Video

16. Autonomy for General AssemblyCarnegie Mellon16 Current Status Moved to New Hardware –Powerbot base –WAM arm (Barrett) –All-metal gripper Still Successfully Inserting Clip –Much faster Better hardware “Rough” moves –Base motion is planned, not scripted –Uses force sensing to detect completion / problem –Have not yet characterized success rate

17. Autonomy for General AssemblyCarnegie Mellon17 Upcoming Work Near Term (1 month) –Complete hardware integration Laser, PTU, VizTracker –Characterize success rate of system Mid Term (2-6 months) –Convert to velocity control of WAM –Use force control for actual insertion –Increase reliability through execution monitoring and exception handling Farther Term (2 nd year of contract) –Insert clip into moving taskboard