1. Optimal Multi-Agent Path Finding under the Sum of Costs Objective
Ariel Felner
Ben-Gurion University
Israel
Much of the work here is joint with
Roni Stern, Guni Sharon, Pavel Surynek, Eli Boyarski

2. Multi-agent path finding (MAPF)
Input
A graph with N states
A set of K agents – each with start and goal state
Actions
An agent can move or wait
Task – a solution
A path for each agent (offline)
Constraints
Paths shouldn’t conflict
Agents cannot be in the same location at the same time
(Edge constraints, Following policies)
Target
Minimize the cost of the solution
Centralized solver
Offline task

3. Motivation Robotics Video games Transportation applications
Warehouse management
Product assembly

4. Automated junction Example from Peter Stone’s web site

5. Different cost functions
=Individual path for agent ai
Cost 1: sum of costs
Cost 2: Makepan
µ≤ ξ ≤ m µ

6. Complexity The problem was proved to be NP-hard
[J. Yu and S. M. LaValle, AAAI-2013]
The 15-puzzle is a special case of MAPF

7. Complexity

8. Main approaches MAPF solvers Suboptimal Optimal Reduction Solvers
Search based
Procedure based

9. Suboptimal solvers

10. Searched-based suboptimal solvers
Agents are planned individually
Then, conflicts and deadlocks are resolved
Attributes:
Fast
Easy to understand/implement
Forfeit optimality/completeness

11. Cooperative A* [Silver 2005]
Initialize the reservation table T
For each agent do{
Find a path (do not conflict with T)
Reserve the path in T }
Enhancements:
Hierarchical CA* (HCA) [Sliver 2005]
Windowed-Hierarchical CA* (WHCA*) [Silver 2005]
Conflict Oriented WHCA* [Banya and Felner, ICRA 2014]

12. Relaxing Optimal Solvers
Any optimal solver can be relaxed.
WA* of any A*-based algorithm
Suboptimal A*+OD+ID [Standley and Korf, IJCAI 2011]
Suboptimal ICTS [Aljaloud and Sturtevant, SoCS 2013]
CBS-e [Barrer et al. SoCS-2014]

13. Procedure-based sub-optimal solvers
Have specific movement rules (e.g., go on highway)
Complete!
Very fast!
Far from optimal
Can solve very large problems

14. Procedure-based MAPF solvers
A complete polynomial-time algorithm to the pebble motion problem was already introduced by [Kornhauser, FOCS 1984]
It was recently implemented by Surynek.
Agents move one at a time.
Far from optimal.

15. Procedure-based MAPF solvers
Slidable Multi-Agent Path Planning, [Wang & Botea, IJCAI, 2009]
Complete for slidable grids
Push and Swap [Luna & Bekris, IJCAI, 2011]
Parallel push and swap
Push and Rotate [de Wilde et al. AAMAS 2013]
Macro-based
Complete for graphs where at least two vertices are always unoccupied
BIBOX [Surynek 2013]
Tree-based agent swapping strategy, [Khorshid at el. SOCS, 2011]
Complete for tree type graphs

16. Optimal solvers

17. Main approaches Optimal MAPF solvers Search based Reduction based
A*, M*
ICTS, CBS
SAT
ASP
COP

18. Sum-of costs Makespan Main approaches Optimal MAPF solvers
Search based
Reduction based
A*, M*
ICTS, CBS
SAT
ASP
COP

19. iii. Reduction solvers
Reduce MAPF to other known problems in computer science.
SAT [Surynek 2012]
Integer Linear Programing [Yu et al. ICRA 2013]
Answer Set Programming [Erdem et al, AAAI-2013]
Work extremely fast for small graphs
May be very slow for large graphs

20. SAT solver for makespan [Surynek]
For i=1 to infinity
Create a SAT formula that answers:
“Is there a solution to the problem of cost i”
Solve that formula
Main challenge:
How do we do this for sum-of-costs?

21. Integer Programing [Yu and La-Valle (2013a]
Model MAPF as a network flow problem.
Depths of the flow are associated with the different time steps.
Used Integer Linear Programming (ILP) to provided a set of equations and an objective function which yield the optimal solution.
Done for Makespan

22. Answer Set Programming [Erden et al, 2013]
Used the declarative programming paradigm of Answer Set Programming (ASP) for optimally solving MAPF.
They represent the path finding problem for each agent and the inter-agent constraints as a program P in ASP.
The answer sets of P correspond to solutions of the problem.

23. Sum-of costs Makespan Main approaches Optimal MAPF solvers
Search based
Reduction based
Our current work
A*, M*
ICTS, CBS
SAT
ASP
COP
Very easy

24. Sum-of costs Makespan Main approaches SAT Optimal MAPF solvers
Search based
Reduction based
Our current work
A*, M*
ICTS, CBS
SAT
TODAY

25. Search-based optimal solvers

26. Optimal Search-based MAPF solvers
A*-based algorithm A*
EPEA*
A*+OD+ID M*
Other search algorithms
ICTS
CBS
MA-CBS

27. A* approaches
State space: Permutations of K agents into N locations=O(NK) Operators: Locations of all agent in the next time step Heuristic function: Sum of Individual Costs (SIC)
Wait
SIC = 3+3 = 6
Optimal = 3+4 = 7

28. Problems with A* Problem 1: State space is too large
Solution: Let’s abstract the underlying graph
Ryan [2008,2011] abstracted the underlying into known shapes such as halls, rings and corridors.
Have specific expansion schedule for each of these cases.
Sometimes not optimal.

29. Problems with A* Problem 2: The state space is exponential O(NK)
On a 10x10 grid with 10 agents: =1020
Solution: let’s reduce the number of agents!
i) Independence detection (ID) [Standely 2010]
Divide the agents into independent groups
Solve each group separately S3 G3

30. Problems with A*
Problem 3: The branching factor is exponential: bglobal=bK
On a grid with 20 agents: bk= 520= 95,367,431,640,625
Solution: let’s reduce the branching factor!
M* [Wagner 2011]
Dynamically change the branching factor based on conflicts.
Works on the global search space but starts with single moves of agents
When a conflict occurs between two agents M* moves back to all ancestors and generates ALL possible children.

31. M*
S1,S2
A1,B1 X
C,C X

32. M*
S1,S2 …
A1,B1
Am,Bm

33. Problems with A* Problem 4: Surplus nodes (those with f>C)
Solution: let’s avoid them
i) Operator Decomposition (OD) [Standley AAAI-2010]
Intermediate states
Each level in the tree moves a single agent
Every K levels we have a full state (as A*)
ii) Enhanced partial expansion A* (EPEA*) [AAAI-2012]
 [Goldenberg et al. 2012] studied combinations of these approaches

34. New non-A* algorithms
1) The Increasing Cost Tree Search (ICTS) [Sharon et al. IJCAI-2011, AIJ-2012]
2) Conflict-Based Search (CBS) [Sharon et al. AAAI-2012]
3) Meta-agent Conflict-Based Search (MA-CBS) [Sharon et al. SoCS-2012]
These algorithms are exponential in different parameters 34

35. Algorithm 1: ICTS [IJCAI-2011] Is there a solution with costs
two level algorithm
High-level
Is there a solution with costs ? 10 10 10
NO!
Low-level

36. Algorithm 1: ICTS [IJCAI-2011]
two level algorithm
High-level
What about this? 10 10 11
YES!
Low-level 3 10 11 10

37. ICTS: High level
No solution
Find a solution
SIC 30 ∆ 31 32

38. Experiments: Dragon-Age Origin [Sturtevant]
#problems solved
under 5 minutes
number of agents

39. ICTS: pathological case
SIC =2
Optimal solution =74
Δ=72
A*: solved in ms
ICTS: solved in 36,688ms

40. … A* Algorithm 2: Conflict-based Search (CBS) [AAAI-2012]
Motivation: cases with bottlenecks: A*
f=6: All m2 combinations of (Ai,Bj) will be generated and expanded - all will generate (C,C) which is illegal
f=7: 3 states are expanded
S1,S2 …
A1,B1
A1,B2
Am,Bm
A1,C
C,C
C,G2
G1,G2

41. CBS plans for single agents
CBS – underlying idea
A* and ICTS work in a
K-agent search space
CBS plans for single agents
but under constraints

42. Conflicts and constraints
Conflict: [agent A, agent B, location X, time T ]
Constraint: [agent A, location X, time T]
Conflict is resolved by adding either [A,X,T] or [B,X,T]
CBS: general idea
Plan for each agent individually
Validate plans
If the plans of agents A and B conflict
Constrain A to avoid the conflict or
Constrain B to avoid the conflict

43. The constraint tree Nodes:
A set of individual constraints for each agent
A set of paths consistent with the constraints
Goal test:
Are the paths conflict free.
Expand
Root
Conflict
{1,2,C,2} OK OK
Replan 1
Expand
Not Goal
Goal Test
Replan 2
Goal

44. When m > 4 CBS will examine fewer states than A*
Analysis: example 1
A* : m = O(m2) states
CBS: 2m+14 = O(m) states
When m > 4 CBS will examine fewer states than A*

45. Analysis: example 2 Trends seen In open spaces: use A*
4 optimal solutions for each agent, each pair of solutions has a conflict
CBS: exponential in #conflicts = 54 states
A*: exponential in #agents = 8 states
Trends seen
In open spaces: use A*
In bottlenecks: use CBS
What if I have both?

46. Algorithm 3: Meta-agent CBS (MA-CBS)
1. Plan for each agent individually
2. Validate plans
3. If the plans of agents A and B conflict
5 Constrain A to avoid the conflicts or
Constrain B to avoid the conflict
If (should merge(A,B))
merge A and B into a meta-agent
and solve with A*
Else
Should merge(A,B): meta-reasoning rules
Should merge(A,B) (simple rule)
When the number of conflicts already seen between A,B > T
T= ∞ (never merge) basic CBS
T=0 (always merge) Standley’s ID
MA-CBS

47. Experiments: choosing T for MA-CBS
Many bottlenecks
Few bottlenecks
Many bottlenecks  High T (closer to CBS)
More agents  Low T (closer to A*)
Faster single-agent search  lower T (close to A*)

48. More Enhancements: Improved-CBS
1. Plan for each agent individually
2. Validate plans
3. If the plans of agents A and B conflict
5 Constrain A to avoid the conflicts or
Constrain B to avoid the conflict
If (should merge(A,B))
merge A and B into a meta-agent
and solve with A*
Else
Bypass conflicts
Prioritize conflicts
Merge and restart

49. Experiments: Improved-CBS
Higher is better
Lower is better
ICBS
Dragon Age: Origins map with many corridors and bottlenecks

50. Summary: No universal winner
We observed the following tendencies:
A* - exponential in #agents (NK)
best in areas dense with agents
ICTS - exponential in Δ
best in open areas with few agents
CBS – exponential in #conflicts
best in areas with many bottlenecks
MA-CBS continuum
MA-CBS continuum

51. Reduction-based solvers
Reduce MAPF to other known problems in computer science.
SAT [Surynek 2012]
Integer Linear Programing [Yu et al. ICRA 2013]
Answer Set Programming [Erdem et al, AAAI-2013]

52. Sum-of costs Makespan Main approaches SAT Optimal MAPF solvers
Search based
Reduction based
Our current work
A*, M*
ICTS, CBS
SAT
TODAY

53. SAT solver for makespan [Surynek]
For i=1 to infinity
Create a SAT formula that answers:
“Is there a solution to the problem of cost i”
Solve that formula
Main challenge:
How do we do this for sum-of-costs?

54. Connecting makespan to sum of costs
ξ 0 ( 𝑎 1 )=3 , ξ 0 ( 𝑎 2 )=3
ξ 0 =6
µ 0 = 3
ξ =7,Δ=1
Definitions:
ξ 0 ( 𝑎 𝑖 )= shortest path for 𝑎 𝑖
ξ 0 = 𝑖 . ξ 0 ( 𝑎 𝑖 )
µ 0 = longest ξ 0 ( 𝑎 𝑖 )
ξ = ξ 0 +Δ
=SIC
Claim: all solutions of cost ξ are done within
µ ≤ µ 0 + Δ time steps

55. Time-Expansion Graph (TEG)

56. Time-Expansion Graph (TEG)
For each agent 𝑎 𝑖
Standard edges: at time < ξ 0 ( 𝑎 𝑖 )
Extra edges: at time ≥ ξ 0 ( 𝑎 𝑖 )

57. Basic-SAT
Is there a solution with exactly Δ extra edges for TEGs of length µ

58. Using MDDs
We use MDDs instead of TEGs

59. MDD-SAT vs basic SAT
MDD-SAT is much better than basic SAT

60. Experiments

61. Experiments

62. MAP Brc202d [from Sturtevant 2012]

63. Summary No universal winner
Search-based methods are faster for easier problems
SAT methods are faster for harder problems
Future: a better comparison is needed

64. New setting for MAPF Classical setting Agents: fixed (all)
Time: minimize
New setting
Time: Fixed
Agents: solve as many
Agents may have priorities

65. Evacuation C
Everybody needs to exit a dangerous area either to a global goal or to an group- specific goal B A

66. New setting for MAPF Robots nay have different priorities
Optimize the reward for saving robots
What about priority to people?
People are self interested (morally correct or not?)
Will people obey the algorithm?
Nearest door policy

67. Fairness, Social welfare Algorithm (that people will obey)
Simulate the near door policy
Mark those who are saved (lucky)
Lucky people have priority of infinity
Other people have priority of 1
Save ALL lucky people and as many other people
In an A* search prune a node if there is at least one lucky agent that will not reach its goal