Sliding Autonomy for Peer-To-Peer Human-Robot Teams M.B. Dias, B. Kannan, B. Browning, E. Jones, B. Argall, M.F. Dias, M.B. Zinck, M. Veloso, and A. Stentz IAS 2008 Research Sponsored by The Boeing Company

2 ResultsSliding AutonomySummary ApproachMotivation  Human-Robot pickup teams Un-known team composition Highly heterogeneous teams Dynamic operating environment Complex nature of tasks Need to adapt to changing situations

3 ResultsSliding AutonomySummary ApproachMotivation Sliding Autonomy  Existing Work Initial set of 3 characteristics proposed by Sellner et al. [1] Alternate set of 3 characteristics proposed by Bruemmer and Walton [2]  Our Work Define Sliding Autonomy for the Peer-to-Peer domain Describe a comprehensive set of characteristics for incorporating sliding autonomy in Peer-to-Peer teams Define a general approach for implementation Implement it on an example application Collect results for comparison Allowing team to adjust its level of autonomy as necessary

4 ResultsSliding AutonomySummary ApproachMotivation Sliding Autonomy in Peer-to-Peer teams Sub-team 1Sub-team 2 Human and multi-robot team members Sub-team 3 Robot team members Human and robot team members De-centralized situational awareness Varying prioritization among team member High priority Low priority High priority Low priority Varying levels of decision making Higher decision making Lower decision making Higher decision making Lower decision making Team members, humans or robots, are actively involved in deciding when to temporarily relinquish control to another member or to an entity outside the sub-team Team Dynamic Sub teams

5 ResultsSliding AutonomySummary ApproachMotivation Characteristics of Sliding Autonomy in Peer-to-Peer teams  Granularity of interaction  Maintaining coordination during interventions  Gaining and maintaining situation awareness  Prioritization of team members  Request help  Learning from interaction

6 ResultsSliding AutonomySummary ApproachMotivation System Components Distributed market-based planner Robots Human-Interface tools Tightly-coordinated multi-agent plan Plan selection Role assignments High-level task status Low-level status Errors Low-level commands Low-level status Errors, Maps, Location Low-level commands High-level task status High-level tasks Cost data capabilities - Overview- Sliding Autonomy Granularity of interactionCoordinationSituation AwarenessPrioritization of team members - currently fixed Requesting Help

ResultsSliding AutonomySummary ApproachMotivation 3 Experimental Domain - Treasure Hunt  Human-Robot teams coordinate to explore an unknown environment and locate items of interest Let’s form a sub-team search sector I Robot X retrieve Treasure A Treasure A Robot X

8 ResultsSliding AutonomySummary ApproachMotivation Example Walkthrough Distributed market-based planner Robots Human-Interface tools Tightly-coordinated multi-agent plan Plan selection Role assignments High-level task status Low-level status Errors Low-level commands Low-level status Errors, Maps, Location Low-level commands High-level task status High-level tasks Cost data capabilities

ResultsSliding AutonomySummary ApproachMotivation Implementation without Sliding Autonomy - Error Detection and Recovery Laser error: Autonomous detection Autonomous identification Autonomous/Human-assisted recovery No-Arcs error: Autonomous detection Autonomous identification Human-assisted recovery Pose error: Autonomous detection Autonomous identification No current easy recovery Where am I? ?

10 ResultsSliding AutonomySummary ApproachMotivation Implementation - Error Handling

ResultsSliding AutonomySummary ApproachMotivation Implementation with Sliding Autonomy - Error Detection and Recovery Laser error: Autonomous detection Autonomous identification Autonomous/Human-assisted recovery No-Arcs error: Autonomous detection Autonomous identification Human-assisted recovery Pose error: Autonomous detection Autonomous identification No current easy recovery Where am I? ? We expect the performance of system with sliding autonomy to better than the one without sliding autonomy

12 ResultsSliding AutonomySummary ApproachMotivation Experimental Setup  Indoor testing environment - Robotics Institute, CMU  3 robot team-members and 2 human-team members 3 robot team-members Pioneers - SICK LiDar and Fiber optic gyros  Planning and localization Segways - Camera  Following, relative localization and locating treasures ER1 - Camera  Tele-operation  3 different treasure configurations  Each run is for a fixed time period of 15 minutes  Total of 7 scattered “treasure”

13 ResultsSliding AutonomySummary ApproachMotivation Experimental Setup - Test Environment  dfdfdfdfd Highbay in the Robotics Institute, Carnegie Mellon University

14 ResultsSliding AutonomySummary ApproachMotivation Results - Expt 1: Team composition = 2 Humans, 1 Pioneer, 1 Segway RunTreasure seen (recovered) Error TypesError Source Error per robot T_12(2)Total: 1[L, 6.5 minutes]G(1)R1(1) T_11(1)Total: 2[P (2 min), L(5 min)] G(2)R1(1), R2(1) T_10(0)Total: 1[P(7.5 min)]G(1)R1(1) Sliding Autonomy disabled RunTreasure seen (recovered) Error TypesError Source Error per robot T_14(2)Total: 5[L(1), A(2), P(2)] N(5)R1(2), R2(3) T_14(2)Total: 5[L(1), A(2), P(2)] N(5)R1(2), R2(3) T_14(2)Total: 5[L(1), A(2), P(2)] N(5)R1(2), R2(3) Sliding Autonomy enabled # System is able to recover from multiple error instances

15 ResultsSliding AutonomySummary ApproachMotivation Results - Expt 2: Team composition = 3 Humans, 1 Pioneer, 1 tele-operated ER1 RunTreasure seen (recovered) Error TypesError Source Error per robot T_13(2) NTotal: 1[L1]G(1)R1(1) T_10(0) NTotal: 1[A(1)]N(1)R1(1) T_14(2) NTotal: 1[L(1)]G(1)R1(1) Sliding Autonomy disabled Skill level E - Expert, N - Novice RunTreasure seen (recovered) Error TypesError Source Error per robot T_14(4) NTotal: 2[L(1), P(1)]G(2)R1(1), R2(1) T_16(3) ETotal: 5[L(1), A(3), P(1)] G(2),N(3)R1(3), R2(2) T_14(2) ETotal: 3[L(1), A(1), P(1)] G(2),N(1)R1(2), R2(1) Sliding Autonomy enabled # Sliding Autonomy can improve team performance # Flexibility in accommodating different team configurations

16 ResultsSliding AutonomySummary ApproachMotivation Conclusion and Future Work  Conclusion Extend Sliding Autonomy to Peer-to-Peer human- robot teams Outline an approach for implementing SA Implement on an example human-robot team application Ability to dynamically adjust the level of autonomy can enhance system performance  Future Work Enhancing situational awareness via human interaction and state information Dynamic prioritization among team members Incorporate learning into the system

17 ResultsSliding AutonomySummary ApproachMotivation Questions  References [1] B. Sellner, F. W. Heger, L. M. Hiatt, R. Simmons, and S. Singh, “Coordinated Multiagent Teams and Sliding Autonomy for Large-Scale Assembly,” Proceedings of the IEEE, Vol. 94, No. 7, 2006 [2] D. J. Bruemmer and M. Walton, “Collaborative tools for mixed teams of humans and robots,” Proceedings of the Workshop on Multi-Robot Systems, 2003

18 ResultsSliding AutonomySummary ApproachMotivation Outline  Motivation  Sliding Autonomy for peer-to-peer teams  Approach  Results  Summary and future work

19 ResultsSliding AutonomySummary ApproachMotivation  Humans and robots working together to accomplish complex team tasks  Pickup teams - un-known team composition with members of varying capabilities, expertise, and knowledge  Robots - In-sufficient capabilities to handle complex situation  Human roles Predominantly - supervisory or end-user Alternate approach - peer-to-peer relationship  Effective use of the complimentary capabilities of humans and robots  Allow humans the flexibility to handle situations that the robots cant Key feature - Allowing team to adjust its level of autonomy as necessary

20 ResultsSliding AutonomySummary ApproachMotivation Fluid teams Team: time = t1 Team: time = t2 Team: time = t3

21 ResultsSliding AutonomySummary ApproachMotivation Trading system §Robots are organized as an economy §Autonomous task-allocation based on cost and capability §Team mission is to maximize production and minimize costs §Instantaneous allocation §Tiered-auctioning approach §Individual agents generate plans and auction them §Humans are yet not part of the auctions $ $ $ $ $

22 ResultsSliding AutonomySummary ApproachMotivation Play Manager §Play selection from playbook §Dynamic role assignment §Coordinates execution of action by sub-teams §Low-level commands to team members §Handles status messages to and from team-members Play 1 Play 2 Play 3 Role 1 Search Retrieve SelectionExecution Monitoring, Adaptation Robot 1 Tactic Robot 1 Tactic Robot N Tactic

23 ResultsSliding AutonomySummary ApproachMotivation System Components (cntd.)  Operator Tools High-level and low-level commands State information feedback Process status messages

24 ResultsSliding AutonomySummary ApproachMotivation System components (cntd.)  Play manager coordinates execution of action by sub-teams low-level commands to team members handles status messages to and from team-members Error Handling via help requests  GUI  Text-based  Help Intervention physical interaction direct low-granularity commands  Robot Software Abstract information - capabilities, actions and sensors Fluid participation

25 ResultsSliding AutonomySummary ApproachMotivation Our Approach  granularity level 1 - High-level task objective level 2 - Low-level robot commands  Coordination via simple communication protocol  Help requested for error-handling modes  Fixed prioritization technique - low-level commands over-rule human instructions  Situational awareness is handled via a customizable GUI reflecting state information