1. Cooperative Pathfinding
David Silver

2. Overview The problem Solution methods Live demo (!?) Results
Local repair A*
Cooperative A*
Hierarchical, Cooperative A*
Windowed, Hierarchical, Cooperative A*
Live demo (!?)
Results

3. The Problem N agents N goals Find N paths
Such that no two paths overlap

4. The Problem with the Problem
Search space is enormous
e.g. 100x100 gridworld
~(10,000)N states
Branching factor of 5N
{N, E, S, W, wait} for each agent
Need to solve in real-time
e.g. complete search within 1ms

5. Reformulating the Problem
Err… isn’t this course called Single Agent Search?
So simplify the problem to a series of single agent searches.
Two approaches considered:
A* with local repair
Cooperative A*

6. 1. A* with Local Repair Search for route to goal Ignore other agents
If collision is imminent, route again
Increase agitation level with each reroute
Cross fingers and hope for the best

7. Problems with Local Repair
Failure to reach goal
Long solution lengths
Frequent recalculation
Appearance of ‘stupidity’

8. 2. Cooperative A* Consider each agent in turn, greedily
Search for a route to goal, avoiding reserved states
Mark the agent’s route in a reservation table
Basic heuristic uses Manhattan distance

9. Search Space The new search space has 3 dimensions
2 dimensional grid
Time dimension
Reservation table marks illegal states
Mark each state on any agent’s path
Sparse data structure
Implemented using hash table

10. Problems with Cooperative A*
Poor heuristic, many nodes expanded
Need to improve heuristic
Problems with agents at destination
Need to continue searching
Sensitive to agent order
Dynamically rotate through agent orders

11. How to improve the heuristic?
Pattern databases no good
Search space too large
Goal may be different each time
Map may change dynamically
So use hierarchical A* (Holte)
Search for goal at abstract level
Use abstract distance as heuristic
3 ideas for reusing abstract search

12. 3. Hierarchical, Cooperative A*
Domain abstraction
Ignore time dimension
Ignore reservation table
Basic gridworld search
Cooperative A* as before
But using abstract distance as heuristic
Abstract distance computed on demand

13. Reverse, Resumable A* Search backwards from Goal Search towards Start
But only terminate when requested node is reached
Keep Open, Closed lists
Resume search whenever a new node is requested
Consistent heuristic required

14. 4. Windowed, Hierarchical, Cooperative A*
Break up search into manageable pieces
Like an intermediate abstraction layer
Full search up to N steps
Ignore time/reservations after N steps
After N steps this is the same as abstract layer
So use abstract distance to complete search

15. Windowing S
window N
abstract edge
d(N,G) = h(N,G) G

16. Continuing the search Continue search after reaching destination
May need to get out of the way
Use edge costs:
0 if sitting on destination
1 for other edges
Abstract distance to goal for final edge

17. Using Windows Compute window for all agents initially
Then recompute each agent when half way through window
Stagger computation to spread out processing time
Can keep abstract distances until destination changes

18. Demo
I hope this works!

19. Results: nodes expanded

20. Results: path length

21. Further ideas Additional layers in the hierarchy Prioritising agents
e.g. ignore half the reservations
Prioritising agents
Overriding low priority agents
Abstracting in space as well as time
e.g. Using N-cliques