1. Probabilistic Smart Terrain Dr. John R. Sullins Youngstown State University

2. John Sullins Youngstown State University Probabilistic Smart Terrain ICTAI 2009 2 Outline What is Smart Terrain? Why do we need to add probabilities? Estimating expected distances to objects that meet character needs Plausibility benchmarks and experimental results Adding learned knowledge during exploration Hierarchical application to games

3. John Sullins Youngstown State University Probabilistic Smart Terrain ICTAI 2009 3 Smart Terrain Solves complex navigation problems in real time

4. John Sullins Youngstown State University Probabilistic Smart Terrain ICTAI 2009 4 Smart Terrain Characters have “needs” –Example: hunger Objects in world meet needs –Example: refrigerator with food inside Characters move towards objects that meet needs

5. John Sullins Youngstown State University Probabilistic Smart Terrain ICTAI 2009 5 Smart Terrain Objects meets needs  transmits “signal” –Signal weakens with distance –Signal moves around objects

6. John Sullins Youngstown State University Probabilistic Smart Terrain ICTAI 2009 6 Smart Terrain Characters follow signal to objects –Move in direction of increasing signal –Only need to compute map once when level created –No need for complex navigation!

7. John Sullins Youngstown State University Probabilistic Smart Terrain ICTAI 2009 7 Need for Probabilities Smart terrain can result in implausible actions –Room character has never visited –Contains empty refrigerator Does not transmit signal Character ignores it –Not plausible behavior!

8. John Sullins Youngstown State University Probabilistic Smart Terrain ICTAI 2009 8 Probabilistic Smart Terrain Objects broadcast signal of form “I meet need n” “I may meet need n with probability P ” Probability = uncertainty that object meets need Character might explore uncertain objects along path

9. John Sullins Youngstown State University Probabilistic Smart Terrain ICTAI 2009 9 Probabilistic Smart Terrain Theoretical goal: Move to closest object with highest probability Problem: Optimizing two separate criteria! Actual Goal: Plausible behavior for characters Meets “hunger” need with P = 0.7 At distance 8 Meets “hunger” need with P = 0.6 At distance 6

10. John Sullins Youngstown State University Probabilistic Smart Terrain ICTAI 2009 10 Expected Distances Expected number of tiles character must travel to reach object that fulfills need Use to determine which tile to move to next –Compute expected distance for four surrounding tiles –Move to surrounding tile with lowest value for expected number of tiles to travel

11. John Sullins Youngstown State University Probabilistic Smart Terrain ICTAI 2009 11 Expected Distances P(t): probability no objects within t tiles meet need P(t) =  (1 – p i ) (Equation 1) where d i < t Based on: –d i: distances to each object i –p i: probabilities each object i meets need –Assumption of conditional independence

12. John Sullins Youngstown State University Probabilistic Smart Terrain ICTAI 2009 12 Expected Distances t < 6: P(t) = 1 6 ≤ t < 8: P(t) = (1 – 0.6) = 0.4 t ≥ 8: P(t) = (1 – 0.6)(1 – 0.7) = 0.12 Distance: 6 Prob: 0.6 Distance: 8 Prob: 0.7

13. John Sullins Youngstown State University Probabilistic Smart Terrain ICTAI 2009 13 Expected Distances Expected distance from tile T to tile that meets need E(T) = Σ P(t) (Equation 2) t t < 6: P(t) = 1 6 ≤ t < 8: P(t) = 0.4 t ≥ 8: P(t) =0.12

14. John Sullins Youngstown State University Probabilistic Smart Terrain ICTAI 2009 14 Expected Distances Problem: Sum could be infinite Solution: Limit t to some t max t max > d i  i t max E(T) = Σ P(t) (Equation 3) t

15. John Sullins Youngstown State University Probabilistic Smart Terrain ICTAI 2009 15 Expected Distances Compute expected distance E(T) for all tiles T Character moves to adjacent tile with lowest E(T)

16. John Sullins Youngstown State University Probabilistic Smart Terrain ICTAI 2009 16 Plausibility Benchmarks Goal for games: Non-player characters should behave plausibly –Move in direction that “makes sense” to player Benchmarks for plausible behavior: –Objects similar in either distance or probability –Group of objects in same direction –Objects that meet need with complete certainty

17. John Sullins Youngstown State University Probabilistic Smart Terrain ICTAI 2009 17 Plausibility Benchmarks Objects at same distance  move to higher probability Objects with same probability  move to closer one

18. John Sullins Youngstown State University Probabilistic Smart Terrain ICTAI 2009 18 Plausibility Benchmarks Nearly same distance  move to much higher probability Nearly same probability  move to much closer object

19. John Sullins Youngstown State University Probabilistic Smart Terrain ICTAI 2009 19 Plausibility Benchmarks Aggregate probabilities benchmark: Multiple objects > single object with higher probability –Assumption of conditional independence

20. John Sullins Youngstown State University Probabilistic Smart Terrain ICTAI 2009 20 Plausibility Benchmarks Complete Certainty benchmark: Single object with probability = 1 > multiple objects with probability < 1

21. John Sullins Youngstown State University Probabilistic Smart Terrain ICTAI 2009 21 Learned Knowledge Probabilities changed when object reached –Object meets need  probability becomes 1 –Does not meet need  probability becomes 0 Should affect future actions Refrigerator empty Move towards another goal

22. John Sullins Youngstown State University Probabilistic Smart Terrain ICTAI 2009 22 Learned Knowledge Changing global map affects all characters –Will also appear to have learned this knowledge New character enters room Also ignores empty refrigerator

23. John Sullins Youngstown State University Probabilistic Smart Terrain ICTAI 2009 23 Learned Knowledge Each character stores own world model –Belief object meets needs –Initially based on probabilities –Modified when objects explored Refrigerator R170% Refrigerator R280% ObjectBelief object meets need 0%

24. John Sullins Youngstown State University Probabilistic Smart Terrain ICTAI 2009 24 Learned Knowledge Each object propagates raw data to tiles –Probability it meets need –Distance to that tile

25. John Sullins Youngstown State University Probabilistic Smart Terrain ICTAI 2009 25 Learned Knowledge Character examines surrounding tiles –Modify probabilities using world model –Compute expected distances for each

26. John Sullins Youngstown State University Probabilistic Smart Terrain ICTAI 2009 26 Hierarchical Smart Terrain More realistic scenario: –Know whether objects meet needs –Don’t know if object is present in given area Go to entrance of most likely area If object present, move to it. Otherwise, move to another area

27. John Sullins Youngstown State University Probabilistic Smart Terrain ICTAI 2009 27 Hierarchical Smart Terrain “Area attractors” at entrances to rooms –Broadcast to entire level –Probability object that meets need is in room –Probability set to 0 when reached by character Objects in room –Signal range = size of room –Probability = 1 if present in room

28. John Sullins Youngstown State University Probabilistic Smart Terrain ICTAI 2009 28 Hierarchical Smart Terrain Compute expected distances from area attractors Move to “best” room

29. John Sullins Youngstown State University Probabilistic Smart Terrain ICTAI 2009 29 Hierarchical Smart Terrain Object is present in area: –Now in range of object, probability meets need = 1 –Character will move directly to object

30. John Sullins Youngstown State University Probabilistic Smart Terrain ICTAI 2009 30 Hierarchical Smart Terrain Object is not present in area: –Set probability of area attractor = 0 –Character will move to next plausible attractor

31. John Sullins Youngstown State University Probabilistic Smart Terrain ICTAI 2009 31 Conclusions Probabilities added to Smart Terrain algorithm Characters move to adjacent tile with shortest expected distance to a tile that meets need Algorithm produces plausible behavior for benchmarks Probabilities overridden by learned knowledge Hierarchical algorithm for realistic play

32. John Sullins Youngstown State University Probabilistic Smart Terrain ICTAI 2009 32 Ongoing Work Algorithm modification to avoid local minima Characters with multiple needs at different levels –Low-probability object that meets critical need –High-probability object that meets less critical need –Which to move towards? Objects that change over time –Empty refrigerator now may be restocked in future

33. John Sullins Youngstown State University Probabilistic Smart Terrain ICTAI 2009 33 Local Minima Caused when paths to low probability objects overlap Overlap in paths to P=0.23 objects Tiles nearer to object appear farther away

34. John Sullins Youngstown State University Probabilistic Smart Terrain ICTAI 2009 34 Local Minima Solution: Weight estimated tiles by distance t max E(T) = Σ P(t) k t t Nearby objects “appear” even closer Any weight k > 0 seems to work

35. John Sullins Youngstown State University Probabilistic Smart Terrain ICTAI 2009 35 Multiple Needs Characters can have multiple needs –Hunger, Fun Some needs more critical than others –Hunger = 10Fun = 5 Objects may only partially fulfill needs –Donuts: Hunger-7 –Cookies: Hunger-3 –TV: Fun-6 Needs increase over time (each tile traversed) –Hunger += 0.5 per tileFun += 0.2 per tile

36. John Sullins Youngstown State University Probabilistic Smart Terrain ICTAI 2009 36 Multiple Needs Goal: Minimize total “discontentment” Σ (need j ) 2 j Problem: Balancing different factors

37. John Sullins Youngstown State University Probabilistic Smart Terrain ICTAI 2009 37 Multiple Needs Terminology: –n j = current level of need j –d i j = distance to object i –p i j = probability object i meets need j –a i j = amount that need j decreased by if it meets need) –c j = increase in need j for each tile traversed Expected decrease in need j caused by all objects i = Σ p i j a i j within t tiles i

38. John Sullins Youngstown State University Probabilistic Smart Terrain ICTAI 2009 38 Multiple Needs Expected level of need j if move t tiles: max (0, n j + tc j - Σ p i j a i j ) where d i < t i Expected discontentment if move t tiles: Σ ( max (0, n j + tc j - Σ p i j a i j ) ) 2 where d i < t j i

39. John Sullins Youngstown State University Probabilistic Smart Terrain ICTAI 2009 39 Multiple Needs Total expected discontentment at given tile: t max Σ Σ ( max (0, n j + tc j - Σ p i j a i j ) ) 2 t j i Compute for surrounding tiles Move to tile with lowest expected discontentment