Guni Sharon, Roni Stern, Meir Goldenberg, Ariel Felner. Ben-Gurion University of The Negev Department of Information Systems Engineering Israel T HE INCREASING COST TREE SEARCH FOR OPTIMAL MULTI - AGENT PATHFINDING 1

B ACKGROUND In Multi-Agent Path Finding we would like to find  A path for each agent, such that  The different paths won’t overlap  Task: Minimize the total travel cost 2 Background Previous work ICTS formalization Theoretical analysis Do it faster Summery

M OTIVATION  Robotics  Video games  Vehicle routing  Air/Train traffic control 3 Background Previous work ICTS formalization Theoretical analysis Do it faster Summery

P REVIOUS WORK  Decoupled approach: Every agent plans separately. + Fast - Non optimal - Many times not complete ([Dresner and Stone, 2008], [Jansen and Sturtevant, 2008], [Silver, 2005], [Wang and Botea, 2008])  Centralized approach: agents are planned together + Can be optimal + Complete - Exponentially hard ([Ryan, 2008], [Ryan, 2010], [Standley, 2010], [Wang and Botea, 2008]) 4 Background Previous work ICTS formalization Theoretical analysis Do it faster Summery

A* SEARCH  Previous work used A* to solve this problem [Standley, 2010]  The heuristic used to guide the A* search is the Sum of Individual Costs (SIC).  SIC is the sum of shortest paths of each agent assuming that no other agent exist.  For the 15 puzzle, assuming each tile is an agent, this is Manhattan Distance. 5 Background Previous work ICTS formalization Theoretical analysis Do it faster Summery

A* SEARCH Stay! Expanding this state results in 25 new states! 5 possible moves 5 possible moves State – a set of locations, one per agent. Stay! 6 Background Previous work ICTS formalization Theoretical analysis Do it faster Summery

abc def ghi S EARCH T REE G ROWTH a,ia,i Root b,fb,fd,id,id,hd,hd,fd,f... g,ig,ig,cg,cg,eg,ea,fa,fa,ia,ia,ca,c 7 Background Previous work ICTS formalization Theoretical analysis Do it faster Summery

P ROBLEM !  The branching factor is (k is the number of agents)  On a problem with only 20 agents:  Branching factor = 95,367,431,640,625  A* can’t expand even the root!!!  Even given a perfect heuristic – A* is not feasible! 8 Background Previous work ICTS formalization Theoretical analysis Do it faster Summery

S TATE - OF - THE - ART A* APPROACH  Recently, two major enhancements to the A* approach were presented [Standley, 2010]  Operator Decomposition (OD)  Independence Detection (ID) – relevant in our case too. problem Independent sub-problem 9 Background Previous work ICTS formalization Theoretical analysis Do it faster Summery Independent sub-problem

O UR NEW ALGORITHM The increasing cost tree search (ICTS) 10

ICTS: O BSERVATION  A solution for MAPF is composed of single agent solutions (one per agent).  we decompose the total solution cost to cost per agent Solution’s cost Background Previous work ICTS formalization Theoretical analysis Do it faster Summery

T HE INCREASING COST SEARCH  The Increasing Cost tree Search (ICTS) is conceptually different from A*. It consist of two levels.  The high level:  What is the cost for every agent?  The low level:  Is there a valid solution based on a vector of costs (Given by the high level)? c1c1 c2c2 c33c33 12 Background Previous work ICTS formalization Theoretical analysis Do it faster Summery

What about this? T HE INCREASING COST APPROACH Background Previous work ICTS formalization Theoretical analysis Do it faster Summery High-level Low-level NO! YES! 3 Is there a solution with costs ? 3 3 3

H IGH L EVEL  The high level searches the Increasing Cost Tree (ICT) - defined as follow:  Node – a cost vector (cost per agent)  Operators – one agent’s cost is increased by one.  Root – The minimal individual costs (SIC). 14 Background Previous work ICTS formalization Theoretical analysis Do it faster Summery

H IGH LEVEL SEARCH  Search the ICT in a breadth-first manner.  For each node: is there a solution restricted to the given costs set?  The first solution found is surely optimal. ∆ Tree size=O( ) 15 No solution Find a solution Background Previous work ICTS formalization Theoretical analysis Do it faster Summery

L OW - LEVEL : G OAL TEST  The low level performs a goal test - Is there a valid solution for a given ICT node?  ICT node represents all paths for all agents given the costs. Low level:  1) For each agent enumerate all paths of its given cost.  2) Search for a valid set of paths. 16 Background Previous work ICTS formalization Theoretical analysis Do it faster Summery c1c1 c2c2 c33c33

E NUMERATING PATHS FOR AN AGENT  Problem: the number of different paths for a single agent is itself exponential. start goal 17 Background Previous work ICTS formalization Theoretical analysis Do it faster Summery

Linear in the number of steps StartAB CDE FGgoal S OLUTION - M ULTI VALUE DECISION DIAGRAM Start AC FDB GE Goal 4-steps MDD Exponential in the number of steps 18  Each level represents a step  Each level has no more then |v| nodes  Compact representation for all possible paths Background Previous work ICTS formalization Theoretical analysis Do it faster Summery

 Two single-agent MDDs can be merged into a two- agent MDD.  Representing all possible locations for two agents. O BSERVATION - M ERGING MDD S Start 1 MDD1 MDD2MDD(1,2) Start 2 Start 1,2 ABAC A,CB,AB,C Goal 1 Goal 2 Goal 1,2 19 A,A Background Previous work ICTS formalization Theoretical analysis Do it faster Summery

L OW LEVEL FORMALIZATION  The low level (goal test) works as follow:  For each agent, build an MDD according the given cost.  Merge all single agent MDDs to one k-agents-MDD.  Search the k-agents-MDD search space for a solution. 20 Background Previous work ICTS formalization Theoretical analysis Do it faster Summery Start 1 MDD1MDD2MDD(1,2) Start 2 Start 1,2 ABAC A,CB,AB,C Goal 1 Goal 2 Goal 1,2

T HEORETICAL ANALYSIS  A* expands the minimal nodes necessary  A* generates many unnecessary nodes (that will never be expanded)  Amount of unnecessary nodes generated is huge! S G Generated 21 Background Previous work ICTS formalization Theoretical analysis Do it faster Summery

ICTS V S A*  Assume A* expands nodes  A* will generate ( ) nodes  The runtime for A* is O( )  ICTS runtime is composed of  Expanding all the ICT nodes with cost <= optimal cost (O( ))  Performing a goal test on each of these nodes (O( ) )  The total runtime of ICTS is O( )  The question is what is bigger or ? 22 ∆ Background Previous work ICTS formalization Theoretical analysis Do it faster Summery

kk' ∆ A*+OD+ID ICTS , ,22149,2603,0926, ,19771,921,2534,67968,859  3X3 grid,no obstacles  50 random start and goal positions A* V S. ICTS TRADEOFF >> 23 Background Previous work ICTS formalization Theoretical analysis Do it faster Summery <<

S PEEDING UP ICTS  Only the last ICT node is a goal  Verifying that an ICT node is not a goal is hard  identifying a non goal node faster -> significant speedup. Check pairNo Solution! There is no solution for the entire problem! Background Previous work ICTS formalization Theoretical analysis Do it faster Summery

Start P AIR WISE PRUNING For every pair of agents (a1,a2) If no solution exists for a1,a2 Halt Else //A solution exists remove all MDD nodes that can not be part of a solution Start A Goal BA MDD1MDD2 B Goal MDD2’ 25 Background Previous work ICTS formalization Theoretical analysis Do it faster Summery Sparser MDDs will result in a smaller search space further on (in the low level).

kk'A*+OD+IDICTSICTS+P , ,371.42, ,000.0<20, ,000.0<29, EXPERIMENTS 26  8X8 grid, No obstacles  50 random start and goal positions X43 X1,683

“D RAGON A GE : O RIGINS ” MAPS  x – number of agents  y – number of problems solved (under 5 minutes) 27 Background Previous work ICTS formalization Theoretical analysis Do it faster Summery

28

S UMMARY  The relative performance between A* and ICTS depends on K and  On many practical cases ICTS outperforms A*+OD+ID. 29 Background Previous work ICTS formalization Theoretical analysis Do it faster Summery ∆

W HAT NEXT ?  Better pruning techniques:  Reuse – remember bad costs sub-combinations  n-wise pruning [Sharon et al., SoCS 2011]  Anytime/suboptimal version of ICTS  Generalization of the ICTS to other problems  Reducing node generations in the A* approach 30 Background Previous work ICTS formalization Theoretical analysis Do it faster Summery

T HE END 31 Any questions?