1. Multiagent Probabilistic Smart Terrain Dr. John R. Sullins Youngstown State University

2. John Sullins Youngstown State University Multiagent Probabilistic Smart Terrain CGAMES 2011 2 Multi-Agent Search in Games Player in some room on this level Multiple guards searching for player Some rooms more likely than others

3. John Sullins Youngstown State University Multiagent Probabilistic Smart Terrain CGAMES 2011 3 Multi-Agent Search in Games Guards must “divide up” rooms in plausible way Focus on most likely rooms While making sure all rooms searched

4. John Sullins Youngstown State University Multiagent Probabilistic Smart Terrain CGAMES 2011 4 Outline Goals of multi-agent probabilistic search Background: Probabilistic smart terrain Estimating global expected distances to targets that meet goals of group Matching agents to targets Demonstration on examples

5. John Sullins Youngstown State University Multiagent Probabilistic Smart Terrain CGAMES 2011 5 Multi-Agent Smart Terrain Assumptions: –“Teams” of NPCs with same goal (such as “find player”) –One NPC finds target that meets goal  entire team succeeds Must be fast solution –No time for complex negotiations among characters –Plausible behavior from POV of player sufficient

6. John Sullins Youngstown State University Multiagent Probabilistic Smart Terrain CGAMES 2011 6 Plausibility Benchmarks Cooperative behavior: –Room 1 closer to G1 and more probable than room 2 –G1 should still move to room 2 –G2 can cover room 1 –Both rooms searched quickly

7. John Sullins Youngstown State University Multiagent Probabilistic Smart Terrain CGAMES 2011 7 Plausibility Benchmarks Probability as a factor: –Room 1 much more probable than room 2 –G1 should move directly to room 1 –Player overwhelmingly likely to be found there Main purpose: Find goal, not search all targets

8. John Sullins Youngstown State University Multiagent Probabilistic Smart Terrain CGAMES 2011 8 Plausibility Benchmarks Divide and conquer: –G1 closer to both rooms and could explore both –G2 should still move to the rooms also even though closer to neither –If G1 moves to one room, G2 can quickly cover the other

9. John Sullins Youngstown State University Multiagent Probabilistic Smart Terrain CGAMES 2011 9 Outline Goals of multi-agent probabilistic search Background: Probabilistic smart terrain Estimating global expected distances to targets that meet goals of group Matching agents to targets Demonstration on examples

10. John Sullins Youngstown State University Multiagent Probabilistic Smart Terrain CGAMES 2011 10 Smart Terrain Target meets goals  transmits “signal” –Signal moves around objects, weakens with distance Character follows signal to target

11. John Sullins Youngstown State University Multiagent Probabilistic Smart Terrain CGAMES 2011 11 Limits of Smart Terrain Normal smart terrain not appropriate for all situations “Guard search” example: –Player “transmits signal” –Guards follow directly to player –Obvious cheat! Guards should have to search for player based on probabilities

12. John Sullins Youngstown State University Multiagent Probabilistic Smart Terrain CGAMES 2011 12 Probabilistic Smart Terrain Targets broadcast signal of form “I meet goal” “I may meet goal with probability P ”

13. John Sullins Youngstown State University Multiagent Probabilistic Smart Terrain CGAMES 2011 13 Expected Distances Expected number of tiles character must travel from tile x to target that meets goal d max Dist(x) = Σ  (1 – p i ) d=0 d i < d Probability no target within d tiles of x meets goal (assumption of conditional independence) Summed over all distances up to some maximum d max (otherwise sum could be infinite)

14. Compute Dist(x) for adjacent tiles x Move to adjacent tile with lowest Dist(x) John Sullins Youngstown State University Multiagent Probabilistic Smart Terrain CGAMES 2011 14 Expected Distances p = 0.6 distance = 8 p = 0.7 distance = 6

15. John Sullins Youngstown State University Multiagent Probabilistic Smart Terrain CGAMES 2011 15 Outline Goals of multi-agent probabilistic search Background: Probabilistic smart terrain Estimating global expected distances to targets that meet goals of group Matching agents to targets Demonstration on examples

16. John Sullins Youngstown State University Multiagent Probabilistic Smart Terrain CGAMES 2011 16 Global Expected Distances Each agent A j estimates moves until goal found by some agent (not necessarily itself) –Distances to targets A j is moving towards –Distances of other agents to targets A j is moving away from

17. John Sullins Youngstown State University Multiagent Probabilistic Smart Terrain CGAMES 2011 17 Finding Closest Targets Step 1: Each agent A j determines set of targets T j that it is closer to than any other agent

18. John Sullins Youngstown State University Multiagent Probabilistic Smart Terrain CGAMES 2011 18 Partitioning by Direction Step 2: For each possible next tile for A j, determine which targets T j direction  T j would be closer in that direction T j left T j right T j up T j down

19. John Sullins Youngstown State University Multiagent Probabilistic Smart Terrain CGAMES 2011 19 Global Expected Distances Step 3: Compute global expected distance for each possible next tile x based on targets t i  T j d max Dist(x) = Σ  (1 – p i ) d=0 d i < d t i  T j direction  d i = distance(x, t i ) + 1 t i  T j direction  d i = min(distance(A k, t i )) j ≠ k (distance to closest other uncommitted agent)

20. John Sullins Youngstown State University Multiagent Probabilistic Smart Terrain CGAMES 2011 20 Global Expected Distances T j left T j right T j up T j down

21. John Sullins Youngstown State University Multiagent Probabilistic Smart Terrain CGAMES 2011 21 Global Expected Distances Example: Expected global distance for A 1 moving left –Neither room reached in < 4 moves –A 1 reaches room 2 (probability 0.4) in 4 moves –A 2 reaches room 1 (probability 0.5) in 5 moves

22. Each agent computes global expected distances for surrounding tiles Each agent then moves to tile with the lowest global expected distance John Sullins Youngstown State University Multiagent Probabilistic Smart Terrain CGAMES 2011 22 Moving Agents A1 moves right A1 moves left

23. John Sullins Youngstown State University Multiagent Probabilistic Smart Terrain CGAMES 2011 23 Global Expected Distances Probability of targets is also an important factor A1 covers R2, A2 covers R1 A1 covers R1, A2 covers R2

24. John Sullins Youngstown State University Multiagent Probabilistic Smart Terrain CGAMES 2011 24 Outline Goals of multi-agent probabilistic search Background: Probabilistic smart terrain Estimating global expected distances to targets that meet goals of group Matching agents to targets Demonstration on examples

25. John Sullins Youngstown State University Multiagent Probabilistic Smart Terrain CGAMES 2011 25 Assigning Agents to Targets Closest targets in direction of tile x with minimum Dist(x) now assigned to that agent May not be considered by any other agent this move R 2 assigned to A 1 R 2 cannot be considered by A 2

26. Agent now committed to those targets That agent will not be used by other agents to compute their Dist(x) John Sullins Youngstown State University Multiagent Probabilistic Smart Terrain CGAMES 2011 26 Committing Agents to Targets A 1 committed to R 2 and will not move to R 1 Cannot be used by A 2 to determine global expected distance to R 1

27. Some agents may not be closest to any targets Agents “subsumed” by other agents –Initially, no move chosen –Are reconsidered after other agents choose directions John Sullins Youngstown State University Multiagent Probabilistic Smart Terrain CGAMES 2011 27 Subsumed Agents A1 closer to both A2 closest to neither

28. Targets agents moving away from are released to subsumed agents –t i  T j, t i  T jx where Dist(x) is minimum –A 1 chooses to move towards R 2 –R 1 released for consideration by other agents –A 2 now uses R 1 to move left John Sullins Youngstown State University Multiagent Probabilistic Smart Terrain CGAMES 2011 28 Subsumed Agents

29. John Sullins Youngstown State University Multiagent Probabilistic Smart Terrain CGAMES 2011 29 Overall Algorithm Structure for each move { while (unassigned targets && uncommitted agents) { –find closest uncommitted agent to each unassigned target (agents with no targets are subsumed) –for (each non-subsumed agent A) { –A computes Dist(x) based only on other uncommitted agents –A committed to move in direction with minimum Dist(x); –A’s targets in that direction assigned to A; –Targets not in that direction released for next cycle of loop } } }

30. John Sullins Youngstown State University Multiagent Probabilistic Smart Terrain CGAMES 2011 30 Subsumed Agents Agents may still be subsumed if all targets assigned to other agents –Loop ends without all agents being assigned targets –A 1 best move is down –Both R 1 and R 2 closer in that direction –Both R 1 and R 2 assigned to A 1 –No remaining targets for A 2

31. John Sullins Youngstown State University Multiagent Probabilistic Smart Terrain CGAMES 2011 31 Subsumed Agents Base subsumed agent move on all targets regardless of what other agents are doing –Use original single-agent probabilistic smart terrain formula –Gives agent appearance of doing something

32. John Sullins Youngstown State University Multiagent Probabilistic Smart Terrain CGAMES 2011 32 Subsumed Agents Agents may eventually not be subsumed –Agent moves to area with multiple targets –Will move towards one target and away from others –Those other targets now available to subsumed agents in area

33. John Sullins Youngstown State University Multiagent Probabilistic Smart Terrain CGAMES 2011 33 Outline Goals of multi-agent probabilistic search Background: Probabilistic smart terrain Estimating global expected distances to targets that meet goals of group Matching agents to targets Demonstration on examples

34. John Sullins Youngstown State University Multiagent Probabilistic Smart Terrain CGAMES 2011 34 Performance on Initial Example Player may be in one of 7 rooms, 3 with “treasure” 3 guards searching for player Probability player in a given “treasure” room = 0.2 Probability player in a “non-treasure” room = 0.1

35. John Sullins Youngstown State University Multiagent Probabilistic Smart Terrain CGAMES 2011 35 Dynamic Targets “Target” tiles placed one step inside each room –Gives guards appearance of “looking” in a room Probabilities change when guard reaches tile –Player not in room  probability set to 0 –Guard now influenced by other rooms

36. John Sullins Youngstown State University Multiagent Probabilistic Smart Terrain CGAMES 2011 36 Performance on Initial Example

37. John Sullins Youngstown State University Multiagent Probabilistic Smart Terrain CGAMES 2011 37 Initial Example Modified “Gold” room moved to upper left Guard 2 moves to jewel room instead Guard 1 path also altered as result

38. John Sullins Youngstown State University Multiagent Probabilistic Smart Terrain CGAMES 2011 38 Ongoing Work Testing with actual players –Implement algorithm as game (Unreal engine) –Goal: steal treasure while avoiding guards –Player can see guard movement, guards use algorithm to search for player Do NPC guard actions appear plausible to players?

39. Multiagent Probabilistic Smart Terrain Dr. John R. Sullins Youngstown State University

40. John Sullins Youngstown State University Multiagent Probabilistic Smart Terrain CGAMES 2011 40 Player Found in Large Example Player tile probability set to 1 Other target probabilities set to 0 All guards will now converge on player