Optimal Multi-Agent Path Finding under the Sum of Costs Objective Ariel Felner Ben-Gurion University Israel Much of the work here is joint with Roni Stern, Guni Sharon, Pavel Surynek, Eli Boyarski

Multi-agent path finding (MAPF) Input A graph with N states A set of K agents – each with start and goal state Actions An agent can move or wait Task – a solution A path for each agent (offline) Constraints Paths shouldn’t conflict Agents cannot be in the same location at the same time (Edge constraints, Following policies) Target Minimize the cost of the solution Centralized solver Offline task

Motivation Robotics Video games Transportation applications Warehouse management Product assembly

Automated junction Example from Peter Stone’s web site

Different cost functions =Individual path for agent ai Cost 1: sum of costs Cost 2: Makepan µ≤ ξ ≤ m µ

Complexity The problem was proved to be NP-hard [J. Yu and S. M. LaValle, AAAI-2013] The 15-puzzle is a special case of MAPF

Complexity

Main approaches MAPF solvers Suboptimal Optimal Reduction Solvers Search based Procedure based

Suboptimal solvers

Searched-based suboptimal solvers Agents are planned individually Then, conflicts and deadlocks are resolved Attributes: Fast Easy to understand/implement Forfeit optimality/completeness

Cooperative A* [Silver 2005] Initialize the reservation table T For each agent do{ Find a path (do not conflict with T) Reserve the path in T } Enhancements: Hierarchical CA* (HCA) [Sliver 2005] Windowed-Hierarchical CA* (WHCA*) [Silver 2005] Conflict Oriented WHCA* [Banya and Felner, ICRA 2014]

Relaxing Optimal Solvers Any optimal solver can be relaxed. WA* of any A*-based algorithm Suboptimal A*+OD+ID [Standley and Korf, IJCAI 2011] Suboptimal ICTS [Aljaloud and Sturtevant, SoCS 2013] CBS-e [Barrer et al. SoCS-2014]

Procedure-based sub-optimal solvers Have specific movement rules (e.g., go on highway) Complete! Very fast! Far from optimal Can solve very large problems

Procedure-based MAPF solvers A complete polynomial-time algorithm to the pebble motion problem was already introduced by [Kornhauser, FOCS 1984] It was recently implemented by Surynek. Agents move one at a time. Far from optimal.

Procedure-based MAPF solvers Slidable Multi-Agent Path Planning, [Wang & Botea, IJCAI, 2009] Complete for slidable grids Push and Swap [Luna & Bekris, IJCAI, 2011] Parallel push and swap Push and Rotate [de Wilde et al. AAMAS 2013] Macro-based Complete for graphs where at least two vertices are always unoccupied BIBOX [Surynek 2013] Tree-based agent swapping strategy, [Khorshid at el. SOCS, 2011] Complete for tree type graphs

Optimal solvers

Main approaches Optimal MAPF solvers Search based Reduction based A*, M* ICTS, CBS SAT ASP COP

Sum-of costs Makespan Main approaches Optimal MAPF solvers Search based Reduction based A*, M* ICTS, CBS SAT ASP COP

iii. Reduction solvers Reduce MAPF to other known problems in computer science. SAT [Surynek 2012] Integer Linear Programing [Yu et al. ICRA 2013] Answer Set Programming [Erdem et al, AAAI-2013] Work extremely fast for small graphs May be very slow for large graphs

SAT solver for makespan [Surynek] For i=1 to infinity Create a SAT formula that answers: “Is there a solution to the problem of cost i” Solve that formula Main challenge: How do we do this for sum-of-costs?

Integer Programing [Yu and La-Valle (2013a] Model MAPF as a network flow problem. Depths of the flow are associated with the different time steps. Used Integer Linear Programming (ILP) to provided a set of equations and an objective function which yield the optimal solution. Done for Makespan

Answer Set Programming [Erden et al, 2013] Used the declarative programming paradigm of Answer Set Programming (ASP) for optimally solving MAPF. They represent the path finding problem for each agent and the inter-agent constraints as a program P in ASP. The answer sets of P correspond to solutions of the problem.

Sum-of costs Makespan Main approaches Optimal MAPF solvers Search based Reduction based Our current work A*, M* ICTS, CBS SAT ASP COP Very easy

Sum-of costs Makespan Main approaches SAT Optimal MAPF solvers Search based Reduction based Our current work A*, M* ICTS, CBS SAT TODAY

Search-based optimal solvers

Optimal Search-based MAPF solvers A*-based algorithm A* EPEA* A*+OD+ID M* Other search algorithms ICTS CBS MA-CBS

A* approaches State space: Permutations of K agents into N locations=O(NK) Operators: Locations of all agent in the next time step Heuristic function: Sum of Individual Costs (SIC) Wait SIC = 3+3 = 6 Optimal = 3+4 = 7

Problems with A* Problem 1: State space is too large Solution: Let’s abstract the underlying graph Ryan [2008,2011] abstracted the underlying into known shapes such as halls, rings and corridors. Have specific expansion schedule for each of these cases. Sometimes not optimal.

Problems with A* Problem 2: The state space is exponential O(NK) On a 10x10 grid with 10 agents: =1020 Solution: let’s reduce the number of agents! i) Independence detection (ID) [Standely 2010] Divide the agents into independent groups Solve each group separately S3 G3

Problems with A* Problem 3: The branching factor is exponential: bglobal=bK On a grid with 20 agents: bk= 520= 95,367,431,640,625 Solution: let’s reduce the branching factor! M* [Wagner 2011] Dynamically change the branching factor based on conflicts. Works on the global search space but starts with single moves of agents When a conflict occurs between two agents M* moves back to all ancestors and generates ALL possible children.

M* S1,S2 A1,B1 X C,C X

M* S1,S2 … A1,B1 Am,Bm

Problems with A* Problem 4: Surplus nodes (those with f>C) Solution: let’s avoid them i) Operator Decomposition (OD) [Standley AAAI-2010] Intermediate states Each level in the tree moves a single agent Every K levels we have a full state (as A*) ii) Enhanced partial expansion A* (EPEA*) [AAAI-2012]  [Goldenberg et al. 2012] studied combinations of these approaches

New non-A* algorithms 1) The Increasing Cost Tree Search (ICTS) [Sharon et al. IJCAI-2011, AIJ-2012] 2) Conflict-Based Search (CBS) [Sharon et al. AAAI-2012] 3) Meta-agent Conflict-Based Search (MA-CBS) [Sharon et al. SoCS-2012] These algorithms are exponential in different parameters 34

Algorithm 1: ICTS [IJCAI-2011] Is there a solution with costs two level algorithm High-level Is there a solution with costs ? 10 10 10 NO! Low-level

Algorithm 1: ICTS [IJCAI-2011] two level algorithm High-level What about this? 10 10 11 YES! Low-level 3 10 11 10

ICTS: High level No solution Find a solution SIC 30 ∆ 31 32

Experiments: Dragon-Age Origin [Sturtevant] #problems solved under 5 minutes number of agents

ICTS: pathological case SIC =2 Optimal solution =74 Δ=72 A*: solved in 51ms ICTS: solved in 36,688ms

… A* Algorithm 2: Conflict-based Search (CBS) [AAAI-2012] Motivation: cases with bottlenecks: A* f=6: All m2 combinations of (Ai,Bj) will be generated and expanded - all will generate (C,C) which is illegal f=7: 3 states are expanded S1,S2 … A1,B1 A1,B2 Am,Bm A1,C C,C C,G2 G1,G2

CBS plans for single agents CBS – underlying idea A* and ICTS work in a K-agent search space CBS plans for single agents but under constraints

Conflicts and constraints Conflict: [agent A, agent B, location X, time T ] Constraint: [agent A, location X, time T] Conflict is resolved by adding either [A,X,T] or [B,X,T] CBS: general idea Plan for each agent individually Validate plans If the plans of agents A and B conflict Constrain A to avoid the conflict or Constrain B to avoid the conflict

The constraint tree Nodes: A set of individual constraints for each agent A set of paths consistent with the constraints Goal test: Are the paths conflict free. Expand Root Conflict {1,2,C,2} OK OK Replan 1 Expand Not Goal Goal Test Replan 2 Goal

When m > 4 CBS will examine fewer states than A* Analysis: example 1 A* : m2+3 = O(m2) states CBS: 2m+14 = O(m) states When m > 4 CBS will examine fewer states than A*

Analysis: example 2 Trends seen In open spaces: use A* 4 optimal solutions for each agent, each pair of solutions has a conflict CBS: exponential in #conflicts = 54 states A*: exponential in #agents = 8 states Trends seen In open spaces: use A* In bottlenecks: use CBS What if I have both?

Algorithm 3: Meta-agent CBS (MA-CBS) 1. Plan for each agent individually 2. Validate plans 3. If the plans of agents A and B conflict 5 Constrain A to avoid the conflicts or Constrain B to avoid the conflict 4 If (should merge(A,B)) merge A and B into a meta-agent and solve with A* Else Should merge(A,B): meta-reasoning rules Should merge(A,B) (simple rule) When the number of conflicts already seen between A,B > T T= ∞ (never merge) basic CBS T=0 (always merge) Standley’s ID MA-CBS

Experiments: choosing T for MA-CBS Many bottlenecks Few bottlenecks Many bottlenecks  High T (closer to CBS) More agents  Low T (closer to A*) Faster single-agent search  lower T (close to A*)

More Enhancements: Improved-CBS 1. Plan for each agent individually 2. Validate plans 3. If the plans of agents A and B conflict 5 Constrain A to avoid the conflicts or Constrain B to avoid the conflict 4 If (should merge(A,B)) merge A and B into a meta-agent and solve with A* Else Bypass conflicts Prioritize conflicts Merge and restart

Experiments: Improved-CBS Higher is better Lower is better ICBS Dragon Age: Origins map with many corridors and bottlenecks

Summary: No universal winner We observed the following tendencies: A* - exponential in #agents (NK) best in areas dense with agents ICTS - exponential in Δ best in open areas with few agents CBS – exponential in #conflicts best in areas with many bottlenecks MA-CBS continuum MA-CBS continuum

Reduction-based solvers Reduce MAPF to other known problems in computer science. SAT [Surynek 2012] Integer Linear Programing [Yu et al. ICRA 2013] Answer Set Programming [Erdem et al, AAAI-2013]

Sum-of costs Makespan Main approaches SAT Optimal MAPF solvers Search based Reduction based Our current work A*, M* ICTS, CBS SAT TODAY

SAT solver for makespan [Surynek] For i=1 to infinity Create a SAT formula that answers: “Is there a solution to the problem of cost i” Solve that formula Main challenge: How do we do this for sum-of-costs?

Connecting makespan to sum of costs ξ 0 ( 𝑎 1 )=3 , ξ 0 ( 𝑎 2 )=3 ξ 0 =6 µ 0 = 3 ξ =7,Δ=1 Definitions: ξ 0 ( 𝑎 𝑖 )= shortest path for 𝑎 𝑖 ξ 0 = 𝑖 . ξ 0 ( 𝑎 𝑖 ) µ 0 = longest ξ 0 ( 𝑎 𝑖 ) ξ = ξ 0 +Δ =SIC Claim: all solutions of cost ξ are done within µ ≤ µ 0 + Δ time steps

Time-Expansion Graph (TEG)

Time-Expansion Graph (TEG) For each agent 𝑎 𝑖 Standard edges: at time < ξ 0 ( 𝑎 𝑖 ) Extra edges: at time ≥ ξ 0 ( 𝑎 𝑖 )

Basic-SAT Is there a solution with exactly Δ extra edges for TEGs of length µ

Using MDDs We use MDDs instead of TEGs

MDD-SAT vs basic SAT MDD-SAT is much better than basic SAT

Experiments

Experiments

MAP Brc202d [from Sturtevant 2012]

Summary No universal winner Search-based methods are faster for easier problems SAT methods are faster for harder problems Future: a better comparison is needed

New setting for MAPF Classical setting Agents: fixed (all) Time: minimize New setting Time: Fixed Agents: solve as many Agents may have priorities

Evacuation C Everybody needs to exit a dangerous area either to a global goal or to an group- specific goal B A

New setting for MAPF Robots nay have different priorities Optimize the reward for saving robots What about priority to people? People are self interested (morally correct or not?) Will people obey the algorithm? Nearest door policy

Fairness, Social welfare Algorithm (that people will obey) Simulate the near door policy Mark those who are saved (lucky) Lucky people have priority of infinity Other people have priority of 1 Save ALL lucky people and as many other people In an A* search prune a node if there is at least one lucky agent that will not reach its goal