1. Graphs, Goals and NPCs Advanced Programming for 3D Applications CE00383-3

2. Path Planning and Search Spatial Environment Grid of cells mapped on the environment Search Tree Path discretisation search Solution Path

3. Overview Alternative Search Space Representations Alternative Search Space Representations –Grids –Graphs –Meshes… When we know what the world looks like, how can we move through it? When we know what the world looks like, how can we move through it? –A* –Precomputed Pathfinding with Navigation Sets –Potential Fields

4. Introduction In many video games, artificial characters have to find their way through spatial layouts: In many video games, artificial characters have to find their way through spatial layouts: –rooms (DOOM-like games) –terrain models, maps, etc. (strategy games)

5. Search & Path Planning A significant number of games involve moving in buildings, battlefields, etc. A significant number of games involve moving in buildings, battlefields, etc. Choosing the best path within a determined space is a classic AI problem Choosing the best path within a determined space is a classic AI problem Algorithms have been developed that “construct” the best path by exploring possible directions on the basis of individual cells’ properties Algorithms have been developed that “construct” the best path by exploring possible directions on the basis of individual cells’ properties

6. Discretisation grid cellshexagonal cells S G 3D discretisation

7. Regular Grids

8. How do we generate? How do we generate? Advantages Advantages –Random access lookup (O(1)) to determine what tile lies at any coordinate –Complete Negatives Negatives –Usually requires large number of nodes to accurately represent world –Path Quality - Agent can only walk in four cardinal directions? That’s no fun. - Let them walk diagonals! Still not much fun

9. Regular Grids NWSE Diagonals String-Pulling Catmull-Rom Spline

10. Grids as Graphs Everything else we look at will be a graph Everything else we look at will be a graph Grids are graphs too Grids are graphs too –Each cell is a node and edges are adjoining cells Maybe we should just handle grid as a graph Maybe we should just handle grid as a graph –Undirected graph could tell us about topography, etc, whereas array can’t

11. Grids as Graphs

12. Corner Graphs Waypoints around obstacles Waypoints around obstacles

13. Corner Graphs How do we generate? How do we generate? –Identify convex corners, can character walk in straight line between? Add edge if possible Advantages Advantages –Less memory –Faster generation Negatives Negatives –Character will “walk on a rail” hugging edges of obstacles instead of walking through open space –And what about different sized characters? –Lookup is O(n 2 ), have to check every node in graph against every other. Imagine a world with a boat load or corners!

14. Corner Graphs

16. Waypoint Graphs Place nodes in middle of rooms instead of at convex corners Place nodes in middle of rooms instead of at convex corners

17. How do we generate? How do we generate? –Place nodes wherever we want (suits 3-D worlds But requires hand tuning to be effective  ) Advantages Advantages –Reduce memory footprint from regular grids, reduce wall hugging from corner graphs –Work well in “human” architectures Negatives Negatives –Still O(n 2 ) –Path Quality vs. Simplicity –Works poorly in open areas Waypoint Graphs

19. Circle-Based Waypoint Graphs Add radius parameter to indicate open space near waypoint Add radius parameter to indicate open space near waypoint

20. Circle-Based Waypoint Advantages Advantages –Only look at overlapping circles, alleviating O(n 2 ) problem from before –Easier to obtain optimal paths –Works well in open terrain Negatives Negatives –Doesn’t work as well in human architectures that aren’t circle friendly

21. Space-Filling Volumes Use rectangles or 3-D Boxes instead of circles Use rectangles or 3-D Boxes instead of circles

22. Space-Filling Volumes How do we generate? How do we generate? –Seed and Grow –Make Grid and Merge Very similar to circle-based, but handles angles better Very similar to circle-based, but handles angles better

23. Navigation Meshes Let’s try and cover walk able surfaces with convex polygons Let’s try and cover walk able surfaces with convex polygons Character can travel between adjoining polygons Character can travel between adjoining polygons

24. Navigation Meshes

25. How do we generate? How do we generate? –By hand (time consuming) Automated tools to analyze and optimize geometry of world Automated tools to analyze and optimize geometry of world Too complex and not represented as a single polygon mesh, instead may be overlapping, etc Too complex and not represented as a single polygon mesh, instead may be overlapping, etc

26. Navigation Meshes Advantages Advantages –Quickly find optimal paths independent of character shapes and capabilities –Handle indoor and outdoor terrains well Negatives Negatives –Can become complex and expensive memory wise –Difficult to generate

27. Problem with N-Sided Meshes

28. Interacting with Pathfinding What about dynamic objects in world? What about dynamic objects in world? All the representations discussed have been static and obviously can’t handle a dynamic world directly All the representations discussed have been static and obviously can’t handle a dynamic world directly These representations need to be able to provide information to system determining path These representations need to be able to provide information to system determining path –Waypoint Graphs and Corner Graphs don’t illustrate walk able surfaces –Meshes and Grids do map every walk able surface –Space-filling volumes and Circle Waypoints provide some representation of walk able areas

29. Path finding Now that we have world represented, how do we plan movement? Now that we have world represented, how do we plan movement? –A*, Depth-First, Dijkstra Dynamic path finding is expensive Dynamic path finding is expensive –Precompiled solutions can eliminate runtime cost, but memory expensive Article suggests technique of Navigation Set Hierarchy Article suggests technique of Navigation Set Hierarchy

30. Path finding Determine best paths leading from source node to boundary of source set Determine best paths leading from source node to boundary of source set Determine best path from source set boundary to goal set boundary Determine best path from source set boundary to goal set boundary Determine best path from goal set boundary to goal node Determine best path from goal set boundary to goal node Compile list of complete paths and choose one with least cost Compile list of complete paths and choose one with least cost

31. Transition Table Main element in pre computed solutions is a lookup table Main element in pre computed solutions is a lookup table Each entry represents next step to take from one node to some goal node Each entry represents next step to take from one node to some goal node

32. Transition Table Transition Table

33. Transition Table Do not need full search of nodes at run time, just series of lookups Do not need full search of nodes at run time, just series of lookups Fast, but becomes memory expensive as size of world grows Fast, but becomes memory expensive as size of world grows –How expensive? n 2 …solution to shrink transition tables? …solution to shrink transition tables?Hierarchy!

34. Navigation Set Self-contained collection of nodes that requires no links to external nodes to complete a path Self-contained collection of nodes that requires no links to external nodes to complete a path Nodes can be members of more than one set Nodes can be members of more than one set Goal: Find someway to partition large Navigation Sets into smaller ones Goal: Find someway to partition large Navigation Sets into smaller ones

35. Complete Hierarchy

36. Interface Nodes and Sets Need to account for paths that cross navigation sets Need to account for paths that cross navigation sets Any node that connects to a node in another navigation set is an interface node Any node that connects to a node in another navigation set is an interface node Have a second layer of nodes in addition to navigation sets, called interface set Have a second layer of nodes in addition to navigation sets, called interface set Interface set itself is a navigation set Interface set itself is a navigation set –Therefore, can make transition table for it too

37. Complete Hierarchy 21 nodes 21 nodes –1 Navigation Set = 441 Table Entries (21*21) –4 Navigation Sets = 183 Table Entries (7*7 + 7*7 + 7*7 + 6*6)

38. Constructing the Hierarchy Two goals to process –How many tables to create? Amount of data needed is 1/n of original size + interface set Amount of data needed is 1/n of original size + interface set As size of navigation sets increase, cost of interface set becomes less a factor As size of navigation sets increase, cost of interface set becomes less a factor –Where to place boundaries? Keep interface nodes as low as possible Keep interface nodes as low as possible

39. Constructing the Hierarchy

40. Potential Fields Reactive approach to path finding Reactive approach to path finding –Setup virtual potential or force field around objects –Must define field and agents reaction to field Interactions in general are explicitly stated and movement “emerges” Interactions in general are explicitly stated and movement “emerges” –Paths are not planned explicitly

41. Potential Fields Think of world as matrix Think of world as matrix –Each point tells has value to represent strength of field under it –Possibly assign potential based on distance from goal Might get stuck behind obstacle Might get stuck behind obstacle –Result/Goal: “Glide down gradient/path of least resistance”

42. Potential Fields Advantages? Advantages? –Works in continuous space! No need to make grid, place waypoints, etc Disadvantages? Disadvantages? –Can be trapped in local minima –It’s emergent, not sure what it will do

43. Path planning on a grid (1/2) Going from A to B avoiding obstacles Going from A to B avoiding obstacles It is only possible to move from one cell to an adjacent cell It is only possible to move from one cell to an adjacent cell How to find: How to find: –the shortest path? –In the shortest time? –At the lesser cost? A B

44. Path planning on a grid (2/2) From each node of the grid, it is possible to go only to neighbouring nodes From each node of the grid, it is possible to go only to neighbouring nodes Path planning is the process by which a path is assembled from A to B through authorised moves Path planning is the process by which a path is assembled from A to B through authorised moves A B

45. Search Trees From a given node, it is possible to generate a set of neighbours. For instance a given node on the grid has for neighbours corresponding to the N, S, E, W, NW, NE, SE, SW moves From a given node, it is possible to generate a set of neighbours. For instance a given node on the grid has for neighbours corresponding to the N, S, E, W, NW, NE, SE, SW moves The process of generating all the neighbours of a given node is called node expansion. The process of generating all the neighbours of a given node is called node expansion.

46. Grid Moves and Search Trees A B path

47. Search Algorithms and Path Planning Because the path on the grid is formally equivalent to a path on the tree, tree- searching algorithms can be used for path planning Because the path on the grid is formally equivalent to a path on the tree, tree- searching algorithms can be used for path planning The path found on the tree does correspond to a path on the grid, hence in the game environment The path found on the tree does correspond to a path on the grid, hence in the game environment

48. Why Search? Search is a key technique for AI, in particular real-time AI Search is a key technique for AI, in particular real-time AI Applications in computer games include: Applications in computer games include: –Path planning –Moving Target Search –Two-player games (othello, chess, etc.) –Search-based planning for animation or intelligent actor behaviour

49. Search Search consists of a set of technique that explore graphs and extract solutions, i.e. sub-graphs satisfying some desirable properties Search consists of a set of technique that explore graphs and extract solutions, i.e. sub-graphs satisfying some desirable properties Solutions can be: Solutions can be: –Nodes (e.g. n-queens) –Paths (e.g. n-puzzle)

50. State-space Search In the state space representation of a problem, the nodes correspond to partial problem solution states and the arcs correspond to steps in the problem solving process (or operators) In the state space representation of a problem, the nodes correspond to partial problem solution states and the arcs correspond to steps in the problem solving process (or operators) This makes possible to apply the same graph-search algorithms for finding a solution to these problems (various optimisation problems, games, etc.) This makes possible to apply the same graph-search algorithms for finding a solution to these problems (various optimisation problems, games, etc.)

51. Search Problems Many problems can be described as search problems (though this is not always the best description) Many problems can be described as search problems (though this is not always the best description) Some traditional search problems are used to teach search algorithms and assess the performance of search methods Some traditional search problems are used to teach search algorithms and assess the performance of search methods

52. Search Algorithms These algorithms are based on graph traversal through node expansion, i.e. the generation of successors of a given node These algorithms are based on graph traversal through node expansion, i.e. the generation of successors of a given node There are several formulations for search algorithms (e.g., recursive vs. non- recursive ones) There are several formulations for search algorithms (e.g., recursive vs. non- recursive ones)

53. Search Algorithms (2) Traditionally, search algorithms are described using two specific data structures, OPEN and CLOSED Traditionally, search algorithms are described using two specific data structures, OPEN and CLOSED OPEN is the list of candidate nodes OPEN is the list of candidate nodes CLOSED is the list of explored nodes CLOSED is the list of explored nodes

54. Uninformed Search “Brute force” search “Brute force” search … could take more than the age of universe on some problems (e.g. Chess) … could take more than the age of universe on some problems (e.g. Chess) One solutions is to use heuristics to prune the search space: this is called informed search or heuristic search One solutions is to use heuristics to prune the search space: this is called informed search or heuristic search All the search algorithms we will describe are heuristic search algorithms All the search algorithms we will describe are heuristic search algorithms

55. Breadth-first Search OPEN a b c d e c d e f g h d e f g h i j …

56. Breadth-first Search Put S (the Start node) on OPEN Put S (the Start node) on OPEN While OPEN is not empty: While OPEN is not empty: –Select the first node from OPEN –If n is a goal node, exit on success, else Expand node n Expand node n Put children at the end of OPEN (eliminating those already in OPEN or CLOSED) Put children at the end of OPEN (eliminating those already in OPEN or CLOSED) Note: this is a FIFO strategy Note: this is a FIFO strategy

57. Depth-first Search a b c d e f g h c d e n o g h c d e p h c d e … OPEN

58. Depth-first Search Put S on OPEN Put S on OPEN While OPEN is not empty do: While OPEN is not empty do: –Select first node from OPEN –If n is a goal node, exit with solution Else expand node n, put children of n at the beginning of OPEN (eliminating those already on OPEN or CLOSED) Else expand node n, put children of n at the beginning of OPEN (eliminating those already on OPEN or CLOSED) Note: this is a LIFO strategy Note: this is a LIFO strategy

59. Heuristic Search General idea: to use some form of information to reduce the number of nodes explored by the search process General idea: to use some form of information to reduce the number of nodes explored by the search process This saves time and space (memory) This saves time and space (memory) introduction to heuristic search: A* introduction to heuristic search: A*

60. Heuristic Search Principle: using information to guide the search of the space state, reducing the number of nodes actually explored Principle: using information to guide the search of the space state, reducing the number of nodes actually explored This information characterises how a given node might lead to a solution This information characterises how a given node might lead to a solution

61. Heuristic Search Depth-first search and breadth-first search select the first node from the OPEN list as the next node to be explored Depth-first search and breadth-first search select the first node from the OPEN list as the next node to be explored The first node might not be the best The first node might not be the best How to extract the best node from OPEN? How to extract the best node from OPEN? An evaluation function, f(n), measures the quality of each node An evaluation function, f(n), measures the quality of each node A heuristic is an estimate of the cost to the goal node A heuristic is an estimate of the cost to the goal node It is not “mathematically accurate”, it is a guess It is not “mathematically accurate”, it is a guess

62. Best-first Search Put S on OPEN Put S on OPEN While OPEN is not empty: While OPEN is not empty: –Select the node n from OPEN for which f(n) is minimal, put n in CLOSED; Expand n, generating all its successors n i –For all n i do: Compute f(n i ) Compute f(n i ) If n i is neither in OPEN or CLOSED, put it in OPEN If n i is neither in OPEN or CLOSED, put it in OPEN If n i is already in OPEN or CLOSED, compare the new value of f(n i ) to the old one. If the new one is lower (= better), substitute the new value to the old, redirecting its pointers to n If n i is already in OPEN or CLOSED, compare the new value of f(n i ) to the old one. If the new one is lower (= better), substitute the new value to the old, redirecting its pointers to n

63. Best-first Search The problem with BF search is that is has “no memory” The problem with BF search is that is has “no memory” It can wander in the graph without keeping track of the cost of the path it generated so far It can wander in the graph without keeping track of the cost of the path it generated so far

64. The A* Algorithm A* is a heuristic search algorithm whose evaluation function, f, takes into account both the value of a node for reaching the goal and the cost: f(n) = g(n) + h(n) A* is a heuristic search algorithm whose evaluation function, f, takes into account both the value of a node for reaching the goal and the cost: f(n) = g(n) + h(n) If n’ is a successor of n: If n’ is a successor of n: g(n’) = c(n, n’) + g(n) n’ c(n, n’) n

65. The f function in A* f’(n) = h’(n) + g(n) “Best-first” component“Breadth-first” component (breadth-first search if h(n) = 0 for all n)

66. The A* Algorithm (1/3) (adapted from J. Pearl) 1. Put initial node in OPEN 2. if OPEN is empty, exit on failure 3. Take out of OPEN the node n for which f is minimum and put it in CLOSED 4. If n is a goal node, exit returning the path obtained by tracing back pointers from n to s

67. A* Algorithm (2/3) 5. Otherwise expand n, generating all successors nodes n’, and keep pointers from these to n. For each n’ (successor of n): –if n’ is neither in OPEN or CLOSED, compute h(n’) and f(n’) = h(n’) + g(n’) –if n’ is already either in OPEN or CLOSED, re- direct its pointers along the path giving minimal g(n’)

68. A* Algorithm (3/3) 6. If n’ has required changing pointers and was previously on CLOSED, put it back in OPEN 7. Go to step 2

69. The A* Algorithm (pseudo-code) While OPEN is not empty { pop node n from OPEN for which f is minimal if n is a goal node return successful path for each successor n’ of n { if n’ is already in OPEN or CLOSED with a lower cost g(n’), consider next successor compute f(n’) if n’ is in CLOSED, remove it from CLOSED if n’ is not yet in OPEN, push n’ on OPEN } push n onto CLOSED } Return failure if no solution found

70. Distance as Heuristic Function Cell “x” is closer to B than cell “z” Cell “x” is closer to B than cell “z” As the goal is to find the shortest path, distance can be used to select the most promising nodes As the goal is to find the shortest path, distance can be used to select the most promising nodes A B x z Euclidean distance “Manhattan distance”

71. Example S G 1 2 3 4 5 3241 5 S f = g + h f’ = g + h’ (estimate) h’ = d (n, G) c =1 g(n’) = g(n) + c(n, n’)

72. Example (2) S G 1 2 3 4 5 3 241 5 S 3.13.23.3 3.13.23.3

73. Example (3) 2 4153 S 2.22.32.1 S G 1 2 3 4 5 3.13.23.3 2.1 2.2 2.3 1 11 1 1 2 2 22222 2 2 3 3 3 3 3 4 4 4 4 4 4 5 555 6666 6 644 5 55 666

74. Example S G g(n) = steps h(n) = Manhattan distance S G

75. Example 1+3=4 S 1+5=6 G S G

76. Example 1+3=4 S 1+5=6 G S G

77. Example 2+4=6 1+3=4 S 1+5=6 2+2=4 G S G

78. Example 2+4=6 1+3=4 S 1+5=6 2+2=4 G S G

79. Example 2+4=6 1+3=4 S 1+5=6 3+3=6 2+2=4 G Could choose any one of the 6’s next S G

80. Example 2+4=6 1+3=4 S 1+5=6 3+3=6 2+2=4 G Let’s choose this one … S G

81. Example 2+4=6 1+3=4 S 1+5=6 3+3=6 2+2=4 G 4+4=8 Could choose either of the 6’s S G

82. Example 2+4=6 1+3=4 S 1+5=6 3+3=6 2+2=4 G 4+4=8 Let’s choose this one … S G

83. Example 2+4=6 1+3=4 S 1+5=6 3+3=6 2+2=4 G 3+5=84+4=8 S G

84. Example 2+4=6 1+3=4 S 1+5=6 3+3=6 2+2=4 G 3+5=84+4=8 S G

85. Example 2+4=6 1+3=4 S 1+5=6 3+3=6 2+2=4 2+4=6 G 3+5=84+4=8 S G

86. Example 2+4=6 1+3=4 S 1+5=6 3+3=6 2+2=4 2+4=6 G 3+5=84+4=8 S G

87. Example 2+4=6 1+3=4 S 1+5=6 3+3=6 2+2=4 2+4=63+3=6 G 3+5=84+4=8 S G

88. Example 2+4=6 1+3=4 S 1+5=6 3+3=6 2+2=4 2+4=63+3=6 G 3+5=84+4=8 S G

89. Example 2+4=6 1+3=4 S 1+5=6 3+3=6 2+2=4 2+4=63+3=6 G 4+2=6 3+5=84+4=8 S G

90. Example 2+4=6 1+3=4 S 1+5=6 3+3=6 2+2=4 2+4=63+3=6 G 4+2=6 3+5=84+4=8 S G

91. Example 2+4=6 1+3=4 S 1+5=6 3+3=6 2+2=4 2+4=63+3=6 G 5+1=6 4+2=6 3+5=84+4=8 S G

92. Example 2+4=6 1+3=4 S 1+5=6 3+3=6 2+2=4 2+4=63+3=6 G 5+1=6 4+2=6 3+5=84+4=8 S G

93. Example 2+4=6 1+3=4 S 1+5=6 3+3=6 2+2=4 2+4=63+3=6 G 5+1=6 4+2=6 3+5=84+4=8 S G

94. Implementing Path Planning (1) Discretise your graphic environment to obtain a grid of cells Discretise your graphic environment to obtain a grid of cells Determine the contents of each cell, this will define the cost for each cell (whether it can be traversed or not, and at which terrain cost) Determine the contents of each cell, this will define the cost for each cell (whether it can be traversed or not, and at which terrain cost) For a static environment, this can be done off-line For a static environment, this can be done off-line

95. Implementing Path Planning (2) Choose the data structures for OPEN and CLOSED depending on estimated size of the problem Choose the data structures for OPEN and CLOSED depending on estimated size of the problem –get relevant source code for handling data structures Implement the A* algorithm itself Implement the A* algorithm itself Map the resulting path to the animation procedure Map the resulting path to the animation procedure

96. Dynamic Environments A* assumes that the goal is fixed A* assumes that the goal is fixed Hence it cannot be used in dynamic environments, i.e. environments in which the goal is moving Hence it cannot be used in dynamic environments, i.e. environments in which the goal is moving Solutions: Solutions: –A* to last known location + local search –Real-time A* (Algorithm versions of real-time search)

97. The A* “Family” RTA*: Real-Time A*. The algorithm commits to a move every unit of time RTA*: Real-Time A*. The algorithm commits to a move every unit of time LRTA*: Learning RTA* (variants can be used to chase moving targets) LRTA*: Learning RTA* (variants can be used to chase moving targets) A  *:  -admissible A* A  *:  -admissible A* IDA*: Iterative-Deepening A* (f threshold) IDA*: Iterative-Deepening A* (f threshold) D*: Dynamic A*, allows “replanning” D*: Dynamic A*, allows “replanning” R  *: A variant which uses information on “how good” the heuristic function is R  *: A variant which uses information on “how good” the heuristic function is

98. A* search A* search is a heuristically guided graph search A* search is a heuristically guided graph search Under certain conditions, it is guaranteed to be optimal and no other search algorithm can find the solution while exploring fewer vertices Under certain conditions, it is guaranteed to be optimal and no other search algorithm can find the solution while exploring fewer vertices For example, better than Dijkstra’s algorthim because A* is guided by heuristics For example, better than Dijkstra’s algorthim because A* is guided by heuristics Inputs: graph representation, start vertex and goal vertex, cost function, heuristic estimate function Inputs: graph representation, start vertex and goal vertex, cost function, heuristic estimate function Output: best path from start to goal, total cost Output: best path from start to goal, total cost

99. Cost and heuristic functions Cost function g(n) Cost function g(n) –g(n) is the cost from the start vertex to vertex n –Cost could mean distance, resources, risk, etc. –Use distance as cost for shortest path Heuristic function h(n) Heuristic function h(n) –h(n) is the estimated cost from n to the goal –E.g., Euclidean distance, Manhattan distance Total cost f(n) = g(n) + h(n) Total cost f(n) = g(n) + h(n)