Bounded Optimal Coordination for Human-Robot Teams G. Ayorkor Mills-Tettey Thesis Proposal: 12 th December 2008 Committee: Anthony Stentz, M. Bernardine Dias, Michael Trick, Nicola Muscettola

INTRODUCTIONRELATED WORKPROBLEM STATEMENTTECHNICAL APPROACHWORK TO DATECONCLUSION 2 Motivation INTRODUCTION [IMAGE CREDITS]

INTRODUCTIONRELATED WORKPROBLEM STATEMENTTECHNICAL APPROACHWORK TO DATECONCLUSION 3 Example Problem INTRODUCTION Compute: Allocation of tasks to agents Schedule for each agent Route for each agent Maximizing:  (rewards for completed tasks) -  (travel costs for agents’ routes) -  (Cost for agents’ idle time)

INTRODUCTIONRELATED WORKPROBLEM STATEMENTTECHNICAL APPROACHWORK TO DATECONCLUSION 4 Example Problem INTRODUCTION Compute: Allocation of tasks to agents Schedule for each agent Route for each agent Maximizing:  (rewards for completed tasks) -  (travel costs for agents’ routes) -  (Cost for agents’ idle time)

INTRODUCTIONRELATED WORKPROBLEM STATEMENTTECHNICAL APPROACHWORK TO DATECONCLUSION 5 Problem Features Tasks Spatially distributed Multi-step Time constraints Location choice Precedence constraints Simultaneity constraints Proximity constraints Agents Heterogeneous Capacity constraints Locations Capacity constraints Mutual exclusion constraints INTRODUCTION

RELATED WORKPROBLEM STATEMENTTECHNICAL APPROACHWORK TO DATECONCLUSION 6 Thesis Goal: compute bounded optimal, anytime solution to complex coordination problem Allocation of tasks to agents Schedule for each agent Route for each agent Characteristics of application domains: Bounded optimality is important “Small” teams: 10s of agents, 10s-100s of tasks Planning time in minutes (or hours) acceptable Centralized planning acceptable INTRODUCTION

RELATED WORKPROBLEM STATEMENTTECHNICAL APPROACHWORK TO DATECONCLUSION 7 Related Work Problem FeaturesVehicle Routing Multi-Robot Task Allocation Spatial distribution Multi-step Time constraints Location choice Precedence constraints Simultaneity constraints Proximity constraints Heterogeneity Capacity constraints Mutual exclusion constraints Tasks Agents Locations RELATED WORK Optimal algorithmsApproximate algorithms

INTRODUCTIONRELATED WORKPROBLEM STATEMENTTECHNICAL APPROACHWORK TO DATECONCLUSION 8 Vehicle Routing Problem Transportation of passengers / distribution of goods between depots and final users RELATED WORK i D j l k

INTRODUCTIONRELATED WORKPROBLEM STATEMENTTECHNICAL APPROACHWORK TO DATECONCLUSION 9 Vehicle Routing Problem Mathematical models – mixed integer linear programming 3-index models (e.g. Cordeau, 2006) 2-index / set-partitioning models (e.g. Savelsbergh & Sol, 1998) RELATED WORK i D j l k r Indicates if an agent a traverses the edge from node i to node j Indicates if an agent a performs route r

INTRODUCTIONRELATED WORKPROBLEM STATEMENTTECHNICAL APPROACHWORK TO DATECONCLUSION 10 Vehicle Routing Problem Exact approaches Mathematical Programming Heuristic Approaches Construction and improvement heuristics Tabu Search, Genetic Algorithms, Simulated Annealing, etc RELATED WORK

INTRODUCTIONRELATED WORKPROBLEM STATEMENTTECHNICAL APPROACHWORK TO DATECONCLUSION 11 Vehicle Routing Problem Problem features not addressed: Precedence & simultaneity constraints with penalization of waiting/idle time Location choice for tasks Capacity constraints on locations Proximity constraints Mutual exclusion constraints RELATED WORK

INTRODUCTIONRELATED WORKPROBLEM STATEMENTTECHNICAL APPROACHWORK TO DATECONCLUSION 12 Multi-Robot Task Allocation In a multi-robot system, which robot should execute which task? RELATED WORK

INTRODUCTIONRELATED WORKPROBLEM STATEMENTTECHNICAL APPROACHWORK TO DATECONCLUSION 13 Multi-Robot Task Allocation Market-Based Approaches RELATED WORK A victim needs to be rescued at location (4, 2) I can do it for $80 It will cost me $54$73 $101

INTRODUCTIONRELATED WORKPROBLEM STATEMENTTECHNICAL APPROACHWORK TO DATECONCLUSION 14 Multi-Robot Task Allocation Related problem (Jones, 2007) Planning, allocating and scheduling agent interactions for team tasks with precedence and simultaneity constraints Solution approach: Market-based approach with combinatorial bidding and tiered auctions Limitations for this thesis: Does not include all problem features No optimality guarantees RELATED WORK

INTRODUCTIONRELATED WORKPROBLEM STATEMENTTECHNICAL APPROACHWORK TO DATECONCLUSION 15 Multi-Robot Task Allocation Related problem (Koes, 2005) Planning and Execution for Multirobot Coordination Solution approach: Mathematical programming (constraint optimization) Limitations for this thesis: Does not include all problem features Relies on off-the-shelve solvers, solution algorithm not customized to problem domain RELATED WORK

INTRODUCTIONRELATED WORKPROBLEM STATEMENTTECHNICAL APPROACHWORK TO DATECONCLUSION 16 Problem Statement Given A set of tasks (could be multi-step) A team of heterogeneous agents A set of locations A set of constraints (compatibility, precedence, simultaneity, proximity, mutual exclusion) Problem is to determine optimal: Task allocation Schedule Routes PROBLEM STATEMENT

INTRODUCTIONRELATED WORKPROBLEM STATEMENTTECHNICAL APPROACHWORK TO DATECONCLUSION 17 Technical Approach Mathematical programming To guarantee bounded optimality Specifics: Create a set-partitioning formulation of the problem, with side constraints Design a branch-and-price algorithm to solve it TECHNICAL APPROACH

INTRODUCTIONRELATED WORKPROBLEM STATEMENTTECHNICAL APPROACHWORK TO DATECONCLUSION 18 Technical Approach: Overview Background Set-partitioning formulation Branch-and-bound Branch-and-price Application to thesis problem Set-partitioning formulation with side constraints Formulation of pricing sub-problem TECHNICAL APPROACH

INTRODUCTIONRELATED WORKPROBLEM STATEMENTTECHNICAL APPROACHWORK TO DATECONCLUSION 19 Example Set-Partitioning Formulation 1P B 3D 3P 1D 2D 2P 4D A 4P TECHNICAL APPROACH

INTRODUCTIONRELATED WORKPROBLEM STATEMENTTECHNICAL APPROACHWORK TO DATECONCLUSION 20 Set-Partitioning Formulation Example 1P B 3D 3P 1D 2D 2P 4D A 4P TECHNICAL APPROACH Possible route r0r0

INTRODUCTIONRELATED WORKPROBLEM STATEMENTTECHNICAL APPROACHWORK TO DATECONCLUSION 21 Set-Partitioning Formulation Example 1P B 3D 3P 1D 2D 2P 4D A 4P TECHNICAL APPROACH Possible route r1r1

INTRODUCTIONRELATED WORKPROBLEM STATEMENTTECHNICAL APPROACHWORK TO DATECONCLUSION 22 Set-Partitioning Formulation Example 1P B 3D 3P 1D 2D 2P 4D A 4P TECHNICAL APPROACH Possible route r2r2

INTRODUCTIONRELATED WORKPROBLEM STATEMENTTECHNICAL APPROACHWORK TO DATECONCLUSION 23 Set-Partitioning Formulation Example 1P B 3D 3P 1D 4D 4P r0r0 r1r1 r2r2 r3r3 r4r4 r5r5 A:A:00  1 B:B:0000  1 Each agent assigned to at most 1 route: r0r0 r1r1 r2r2 r3r3 r4r4 r5r5 1:000=1 2:00=1 ………………… Each task assigned to exactly 1 route: TECHNICAL APPROACH Set Partitioning Formulation: r0r0 r1r1 r2r2 r3r3 r4r4 2D A 2P r1r1

INTRODUCTIONRELATED WORKPROBLEM STATEMENTTECHNICAL APPROACHWORK TO DATECONCLUSION 24 Set of all agents Set-Partitioning Formulation Example r0r0 r1r1 r2r2 r3r3 r4r4 r5r5 A:A:00  1 B:B:0000  1 Each agent assigned to at most 1 route: r0r0 r1r1 r2r2 r3r3 r4r4 r5r5 1:000=1 2:00=1 ………………… Each task assigned to exactly 1 route: TECHNICAL APPROACH Set Partitioning Formulation: (for each agent, k) (for each task, i) Set of all feasible routes for agent k Indicates if request i is on route r

INTRODUCTIONRELATED WORKPROBLEM STATEMENTTECHNICAL APPROACHWORK TO DATECONCLUSION 25 Branch-and-Bound TECHNICAL APPROACH P P1P1 P2P min: subject to: … route costs

INTRODUCTIONRELATED WORKPROBLEM STATEMENTTECHNICAL APPROACHWORK TO DATECONCLUSION 26 Solution 1P B 3D 3P 1D 2D 2P 4D A 4P TECHNICAL APPROACH r0r0 r1r1

INTRODUCTIONRELATED WORKPROBLEM STATEMENTTECHNICAL APPROACHWORK TO DATECONCLUSION 27 Branch-and-Price But, there might be too many routes to enumerate explicitly. What to do? Column generation at each node of branch-and- bound tree: TECHNICAL APPROACH P P1P1 P2P2 P 11 P 12 At each node: Compute the linear programming (LP) relaxation with a subset of the routes Solve a pricing sub-problem to find a profitable route to include Re-compute the LP relaxation Repeat until no profitable routes r0r0 r1r1 r r 3. r0r0 r1r1 r 2. r 3. r0r0 r1r1 r 2. r r r 5.

INTRODUCTIONRELATED WORKPROBLEM STATEMENTTECHNICAL APPROACHWORK TO DATECONCLUSION 28 Background: Branch-and-Price Pricing sub-problem for this example: TECHNICAL APPROACH Cost of route r for agent k Cost modification for each task (node) on the route Cost modification for entire route Dual variables Shortest route problem in a graph with modified node costs! 1 iff task i is on route r  k

INTRODUCTIONRELATED WORKPROBLEM STATEMENTTECHNICAL APPROACHWORK TO DATECONCLUSION 29 Recap Thesis Problem Problem Features: Tasks: Agents: Locations: Technical Approach: Branch-and-price algorithm on set-partitioning formulation with side constraints TECHNICAL APPROACH Spatially distributed, Multi-step, Location choice, Time windows, Precedence, Simultaneity, Mutual exclusion, Proximity constraints Heterogeneous, Capacity constraints Capacity constraints, Mutual exclusion constraints

INTRODUCTIONRELATED WORKPROBLEM STATEMENTTECHNICAL APPROACHWORK TO DATECONCLUSION 30 Example Problem 1 A 2 3a 4a 3b 4b B C Silo 1 Silo 2 Field 1 Field 2 5 3b 4b5 6 6 D TECHNICAL APPROACH Harvest field 1 Harvest field 2 Move field 1 grain to silo Move field 2 grain to silo Monitor field 1 grain unloading Monitor field 2 grain unloading 1 2 3a3b 4a4b 5 6

INTRODUCTIONRELATED WORKPROBLEM STATEMENTTECHNICAL APPROACHWORK TO DATECONCLUSION 31 Example Problem TaskSubtasksCompatible locations Compatible agents Harvest field 1Field 1 Harvest field 2Field 2 3. Transport field 1 grain Load Unload Field 1 Silo 1, Silo 2 4. Transport field 2 grain Load Unload Field 2 Silo 1, Silo 2 Monitor field 1 grain unloading Monitor field 2 grain unloading A A BC BC 3a 3b 4a 4b D D TECHNICAL APPROACH

INTRODUCTIONRELATED WORKPROBLEM STATEMENTTECHNICAL APPROACHWORK TO DATECONCLUSION 32 Example Problem Precedence constraints: Harvest field 1 before Load grain from field 1 Harvest field 2 before Load grain from field 2 Simultaneity constraints: Monitor field 1 grain unloading concurrent with Unload field 1 grain Monitor field 2 grain unloading concurrent with Unload field 2 grain Proximity constraints: Monitor field 1 grain unloading collated with Unload field 1 grain Monitor field 2 grain unloading collated with Unload field 2 grain 1 2 3a 3b 3a 4b b 4b TECHNICAL APPROACH

INTRODUCTIONRELATED WORKPROBLEM STATEMENTTECHNICAL APPROACHWORK TO DATECONCLUSION 33 Example Problem: Feasible Route B C A Silo 1 Silo D TECHNICAL APPROACH Field 1 Field 2

INTRODUCTIONRELATED WORKPROBLEM STATEMENTTECHNICAL APPROACHWORK TO DATECONCLUSION 34 Example Problem: Feasible Route B C A Silo 1 Silo D TECHNICAL APPROACH Field 1 Field 2

INTRODUCTIONRELATED WORKPROBLEM STATEMENTTECHNICAL APPROACHWORK TO DATECONCLUSION 35 Example Problem: Feasible Route B C A Silo 1 Silo D TECHNICAL APPROACH Field 1 Field 2

INTRODUCTIONRELATED WORKPROBLEM STATEMENTTECHNICAL APPROACHWORK TO DATECONCLUSION 36 Example Problem: Feasible Route B C A Silo 1 Silo D TECHNICAL APPROACH Field 1 Field 2

INTRODUCTIONRELATED WORKPROBLEM STATEMENTTECHNICAL APPROACHWORK TO DATECONCLUSION 37 Set Partitioning Formulation with Side Constraints Objective Function 1 route per agent 1 route per task Valid start time for taskValid arrival time for agent for task Valid idle time for agent for task Synchronization constraints Proximity constraints Precedence constraints Non-overlapping constraints Mutual exclusion constraints Location capacity constraints Maximize: Subject to: TECHNICAL APPROACH Side constraints Standard set-partitioning formulation

INTRODUCTIONRELATED WORKPROBLEM STATEMENTTECHNICAL APPROACHWORK TO DATECONCLUSION 38 Objective Function Maximize Rewards for completed tasks Travel costs for selected routes Costs for agents’ idle/waiting time TECHNICAL APPROACH

INTRODUCTIONRELATED WORKPROBLEM STATEMENTTECHNICAL APPROACHWORK TO DATECONCLUSION 39 Objective Function Maximize Rewards for completed tasks Travel costs for selected routes Costs for agents’ idle/waiting time Route selection variable: 1 if agent k is assigned route r Idle/waiting time of agent k for subtask i at location l TECHNICAL APPROACH

INTRODUCTIONRELATED WORKPROBLEM STATEMENTTECHNICAL APPROACHWORK TO DATECONCLUSION 40 Set Partitioning Formulation with Side Constraints Objective Function 1 route per agent 1 route per task Valid start time for taskValid arrival time for agent for task Valid idle time for agent for task Synchronization constraints Proximity constraints Precedence constraints Non-overlapping constraints Mutual exclusion constraints Location capacity constraints Maximize: Subject to: TECHNICAL APPROACH Side constraints Standard set-partitioning formulation

INTRODUCTIONRELATED WORKPROBLEM STATEMENTTECHNICAL APPROACHWORK TO DATECONCLUSION 41 Pricing Sub-problem TECHNICAL APPROACH Reward for each completed task ∆cost for entire route ∆cost for each subtask linear ∆cost for each subtask ∆cost for each subtask in a precedence/simultaneity constraint ∆cost for each subtask at a capacity-constrained location ∆cost for each subtask in a non-overlapping constraint ∆cost for each subtask at a location with mutual-exclusion Find route (sequence of subtask/location pairs) that maximizes:  + route - Travel cost Subject to constraints: Each route includes only 1 location per subtask All subtasks of a given task are on the same route ∆reward for each completed task - ∆cost for each subtask in a proximity constraint

INTRODUCTIONRELATED WORKPROBLEM STATEMENTTECHNICAL APPROACHWORK TO DATECONCLUSION 42 Work to Date Implemented branch-and-bound algorithm for simplified version of thesis problem Includes precedence constraints Does not include other side constraints Explicitly enumerates all routes – does not involve column generation WORK TO DATE

INTRODUCTIONRELATED WORKPROBLEM STATEMENTTECHNICAL APPROACHWORK TO DATECONCLUSION 43 Example Problem: 12 single-step tasks, 5 agents, max 3 tasks/agent WORK TO DATE Tasks Agents

INTRODUCTIONRELATED WORKPROBLEM STATEMENTTECHNICAL APPROACHWORK TO DATECONCLUSION 44 Example Problem: Optimal Solution 12 single-step tasks, 5 agents, max 3 tasks/agent No precedence constraints WORK TO DATE time Timeline:

INTRODUCTIONRELATED WORKPROBLEM STATEMENTTECHNICAL APPROACHWORK TO DATECONCLUSION 45 Example Problem: Optimal Solution 12 single-step tasks, 5 agents, max 3 tasks/agent 2 precedence constraints: t4<t8, t9<t8 WORK TO DATE time Timeline:

INTRODUCTIONRELATED WORKPROBLEM STATEMENTTECHNICAL APPROACHWORK TO DATECONCLUSION 46 Example Problem: Optimal Solution 12 single-step tasks, 5 agents, max 3 tasks/agent 4 precedence constraints: t4<t8, t9<t8, t8<t5, t5<t3 WORK TO DATE time Timeline: Waiting/idle time

INTRODUCTIONRELATED WORKPROBLEM STATEMENTTECHNICAL APPROACHWORK TO DATECONCLUSION 47 Quantitative Results WORK TO DATE

INTRODUCTIONRELATED WORKPROBLEM STATEMENTTECHNICAL APPROACHWORK TO DATECONCLUSION 48 Bounded Optimality: Example with 4 precedence constraints # Iterations ,000 25,000 20,000 15,000 10,000 5,000 0 Solution Value Legend Best solution found Upper bound Found optimal solution Proved optimality of solution

INTRODUCTIONRELATED WORKPROBLEM STATEMENTTECHNICAL APPROACHWORK TO DATECONCLUSION 49 Overview of Remaining Work Develop branch-and-price algorithm Extend implementation to include all side constraints Develop algorithms to solve pricing sub-problem Implement column generation Develop heuristic algorithms for comparison Evaluate branch-and-price against heuristic algorithms Solution quality Solution time WORK TO DATE

INTRODUCTIONRELATED WORKPROBLEM STATEMENTTECHNICAL APPROACHWORK TO DATECONCLUSION 50 Schedule CONCLUSION Dec ’08 – Jan ‘09Formulate benchmark problems Jan ‘09Implement omitted side constraints Feb – Apr ’09Design and implement algorithms for pricing sub-problem Apr – May ’09Implement column generation Jun – Jul ’09Evaluate branch-and-price algorithm, develop comparison heuristic approaches Aug – Sep ’09Refine algorithms and continue evaluation Oct – Nov ’09Write thesis Dec ’09Defend Thesis

INTRODUCTIONRELATED WORKPROBLEM STATEMENTTECHNICAL APPROACHWORK TO DATECONCLUSION 51 Contributions First branch-and-price approach to optimal robotic team coordination Set-partitioning formulation of novel complex coordination problem First optimal task allocation approach enabling cooperation in human-robot teams CONCLUSION

INTRODUCTIONRELATED WORKPROBLEM STATEMENTTECHNICAL APPROACHWORK TO DATECONCLUSION 52 Acknowledgements Sponsors: Qatar National Research Fund (QNRF) NASA Jet Propulsion Laboratory (JPL) Thesis Committee: Anthony Stentz, M. Bernardine Dias, Michael Trick, Nicola Muscettola rCommerce Lab Members Friends and Family CONCLUSION

Questions?

INTRODUCTIONRELATED WORKPROBLEM STATEMENTTECHNICAL APPROACHWORK TO DATECONCLUSION 54 Image Credits “Motivation” Slide: Construction site: Combine harvester: Tractor-trailer: Disaster response (with people & cranes): leaves-students.html Earthquake:

Extra slides

INTRODUCTIONRELATED WORKPROBLEM STATEMENTTECHNICAL APPROACHWORK TO DATECONCLUSION 56 Vehicle Routing Problem Transportation of passengers / distribution of goods between depots and final users Some variants: Capacitated vehicle routing with time-windows Multi-depot vehicle routing problems Pickup and delivery / dial-a-ride RELATED WORK i D j l k

INTRODUCTIONRELATED WORKPROBLEM STATEMENTTECHNICAL APPROACHWORK TO DATECONCLUSION 57 Multi-Robot Task Allocation Gerkey and Matarić’s (2004) categorization of MRTA problems RELATED WORK ST MT SR MR IA TA Single versus multi task robots Instantaneous versus time-extended assignment Single versus multi robot tasks Our problem: (ST-SR-TA or ST-MR-TA) + inter-task constraints + location choice

INTRODUCTIONRELATED WORKPROBLEM STATEMENTTECHNICAL APPROACHWORK TO DATECONCLUSION 58 Set-Partitioning Formulation Example 1P B 3D 3P 1D 2D 2P 4D A 4P TECHNICAL APPROACH Possible route r3r3

INTRODUCTIONRELATED WORKPROBLEM STATEMENTTECHNICAL APPROACHWORK TO DATECONCLUSION 59 Set-Partitioning Formulation Example 1P B 3D 3P 1D 2D 2P 4D A 4P TECHNICAL APPROACH Possible route r4r4

INTRODUCTIONRELATED WORKPROBLEM STATEMENTTECHNICAL APPROACHWORK TO DATECONCLUSION 60 Set-Partitioning Formulation Example 1P B 3D 3P 1D 2D 2P 4D A 4P TECHNICAL APPROACH Possible route r5r5

INTRODUCTIONRELATED WORKPROBLEM STATEMENTTECHNICAL APPROACHWORK TO DATECONCLUSION 61 Example Problem: Feasible Route B C A Silo 1 Silo D TECHNICAL APPROACH Field 1 Field 2

INTRODUCTIONRELATED WORKPROBLEM STATEMENTTECHNICAL APPROACHWORK TO DATECONCLUSION 62 Example Problem: Feasible Route B C A Silo 1 Silo D TECHNICAL APPROACH Field 1 Field 2

INTRODUCTIONRELATED WORKPROBLEM STATEMENTTECHNICAL APPROACHWORK TO DATECONCLUSION 63 Example Problem: Feasible Route B C A Silo 1 Silo D TECHNICAL APPROACH Field 1 Field 2

INTRODUCTIONRELATED WORKPROBLEM STATEMENTTECHNICAL APPROACHWORK TO DATECONCLUSION 64 Example Problem: Feasible Route B C A Silo 1 Silo D TECHNICAL APPROACH Field 1 Field 2

INTRODUCTIONRELATED WORKPROBLEM STATEMENTTECHNICAL APPROACHWORK TO DATECONCLUSION 65 Example Problem: Feasible Route B C A Silo 1 Silo D TECHNICAL APPROACH Field 1 Field 2

INTRODUCTIONRELATED WORKPROBLEM STATEMENTTECHNICAL APPROACHWORK TO DATECONCLUSION 66 Dual Variables

INTRODUCTIONRELATED WORKPROBLEM STATEMENTTECHNICAL APPROACHWORK TO DATECONCLUSION 67 Details of Pricing Sub-problem