Probabilistic Smart Terrain Dr. John R. Sullins Youngstown State University

John Sullins Youngstown State University Probabilistic Smart Terrain ICTAI Outline What is Smart Terrain? Why do we need to add probabilities? Estimating expected distances to objects that meet character needs Plausibility benchmarks and experimental results Adding learned knowledge during exploration Hierarchical application to games

John Sullins Youngstown State University Probabilistic Smart Terrain ICTAI Smart Terrain Solves complex navigation problems in real time

John Sullins Youngstown State University Probabilistic Smart Terrain ICTAI Smart Terrain Characters have “needs” –Example: hunger Objects in world meet needs –Example: refrigerator with food inside Characters move towards objects that meet needs

John Sullins Youngstown State University Probabilistic Smart Terrain ICTAI Smart Terrain Objects meets needs  transmits “signal” –Signal weakens with distance –Signal moves around objects

John Sullins Youngstown State University Probabilistic Smart Terrain ICTAI Smart Terrain Characters follow signal to objects –Move in direction of increasing signal –Only need to compute map once when level created –No need for complex navigation!

John Sullins Youngstown State University Probabilistic Smart Terrain ICTAI Need for Probabilities Smart terrain can result in implausible actions –Room character has never visited –Contains empty refrigerator Does not transmit signal Character ignores it –Not plausible behavior!

John Sullins Youngstown State University Probabilistic Smart Terrain ICTAI Probabilistic Smart Terrain Objects broadcast signal of form “I meet need n” “I may meet need n with probability P ” Probability = uncertainty that object meets need Character might explore uncertain objects along path

John Sullins Youngstown State University Probabilistic Smart Terrain ICTAI Probabilistic Smart Terrain Theoretical goal: Move to closest object with highest probability Problem: Optimizing two separate criteria! Actual Goal: Plausible behavior for characters Meets “hunger” need with P = 0.7 At distance 8 Meets “hunger” need with P = 0.6 At distance 6

John Sullins Youngstown State University Probabilistic Smart Terrain ICTAI Expected Distances Expected number of tiles character must travel to reach object that fulfills need Use to determine which tile to move to next –Compute expected distance for four surrounding tiles –Move to surrounding tile with lowest value for expected number of tiles to travel

John Sullins Youngstown State University Probabilistic Smart Terrain ICTAI Expected Distances P(t): probability no objects within t tiles meet need P(t) =  (1 – p i ) (Equation 1) where d i < t Based on: –d i: distances to each object i –p i: probabilities each object i meets need –Assumption of conditional independence

John Sullins Youngstown State University Probabilistic Smart Terrain ICTAI Expected Distances t < 6: P(t) = 1 6 ≤ t < 8: P(t) = (1 – 0.6) = 0.4 t ≥ 8: P(t) = (1 – 0.6)(1 – 0.7) = 0.12 Distance: 6 Prob: 0.6 Distance: 8 Prob: 0.7

John Sullins Youngstown State University Probabilistic Smart Terrain ICTAI Expected Distances Expected distance from tile T to tile that meets need E(T) = Σ P(t) (Equation 2) t t < 6: P(t) = 1 6 ≤ t < 8: P(t) = 0.4 t ≥ 8: P(t) =0.12

John Sullins Youngstown State University Probabilistic Smart Terrain ICTAI Expected Distances Problem: Sum could be infinite Solution: Limit t to some t max t max > d i  i t max E(T) = Σ P(t) (Equation 3) t

John Sullins Youngstown State University Probabilistic Smart Terrain ICTAI Expected Distances Compute expected distance E(T) for all tiles T Character moves to adjacent tile with lowest E(T)

John Sullins Youngstown State University Probabilistic Smart Terrain ICTAI Plausibility Benchmarks Goal for games: Non-player characters should behave plausibly –Move in direction that “makes sense” to player Benchmarks for plausible behavior: –Objects similar in either distance or probability –Group of objects in same direction –Objects that meet need with complete certainty

John Sullins Youngstown State University Probabilistic Smart Terrain ICTAI Plausibility Benchmarks Objects at same distance  move to higher probability Objects with same probability  move to closer one

John Sullins Youngstown State University Probabilistic Smart Terrain ICTAI Plausibility Benchmarks Nearly same distance  move to much higher probability Nearly same probability  move to much closer object

John Sullins Youngstown State University Probabilistic Smart Terrain ICTAI Plausibility Benchmarks Aggregate probabilities benchmark: Multiple objects > single object with higher probability –Assumption of conditional independence

John Sullins Youngstown State University Probabilistic Smart Terrain ICTAI Plausibility Benchmarks Complete Certainty benchmark: Single object with probability = 1 > multiple objects with probability < 1

John Sullins Youngstown State University Probabilistic Smart Terrain ICTAI Learned Knowledge Probabilities changed when object reached –Object meets need  probability becomes 1 –Does not meet need  probability becomes 0 Should affect future actions Refrigerator empty Move towards another goal

John Sullins Youngstown State University Probabilistic Smart Terrain ICTAI Learned Knowledge Changing global map affects all characters –Will also appear to have learned this knowledge New character enters room Also ignores empty refrigerator

John Sullins Youngstown State University Probabilistic Smart Terrain ICTAI Learned Knowledge Each character stores own world model –Belief object meets needs –Initially based on probabilities –Modified when objects explored Refrigerator R170% Refrigerator R280% ObjectBelief object meets need 0%

John Sullins Youngstown State University Probabilistic Smart Terrain ICTAI Learned Knowledge Each object propagates raw data to tiles –Probability it meets need –Distance to that tile

John Sullins Youngstown State University Probabilistic Smart Terrain ICTAI Learned Knowledge Character examines surrounding tiles –Modify probabilities using world model –Compute expected distances for each

John Sullins Youngstown State University Probabilistic Smart Terrain ICTAI Hierarchical Smart Terrain More realistic scenario: –Know whether objects meet needs –Don’t know if object is present in given area Go to entrance of most likely area If object present, move to it. Otherwise, move to another area

John Sullins Youngstown State University Probabilistic Smart Terrain ICTAI Hierarchical Smart Terrain “Area attractors” at entrances to rooms –Broadcast to entire level –Probability object that meets need is in room –Probability set to 0 when reached by character Objects in room –Signal range = size of room –Probability = 1 if present in room

John Sullins Youngstown State University Probabilistic Smart Terrain ICTAI Hierarchical Smart Terrain Compute expected distances from area attractors Move to “best” room

John Sullins Youngstown State University Probabilistic Smart Terrain ICTAI Hierarchical Smart Terrain Object is present in area: –Now in range of object, probability meets need = 1 –Character will move directly to object

John Sullins Youngstown State University Probabilistic Smart Terrain ICTAI Hierarchical Smart Terrain Object is not present in area: –Set probability of area attractor = 0 –Character will move to next plausible attractor

John Sullins Youngstown State University Probabilistic Smart Terrain ICTAI Conclusions Probabilities added to Smart Terrain algorithm Characters move to adjacent tile with shortest expected distance to a tile that meets need Algorithm produces plausible behavior for benchmarks Probabilities overridden by learned knowledge Hierarchical algorithm for realistic play

John Sullins Youngstown State University Probabilistic Smart Terrain ICTAI Ongoing Work Algorithm modification to avoid local minima Characters with multiple needs at different levels –Low-probability object that meets critical need –High-probability object that meets less critical need –Which to move towards? Objects that change over time –Empty refrigerator now may be restocked in future

John Sullins Youngstown State University Probabilistic Smart Terrain ICTAI Local Minima Caused when paths to low probability objects overlap Overlap in paths to P=0.23 objects Tiles nearer to object appear farther away

John Sullins Youngstown State University Probabilistic Smart Terrain ICTAI Local Minima Solution: Weight estimated tiles by distance t max E(T) = Σ P(t) k t t Nearby objects “appear” even closer Any weight k > 0 seems to work

John Sullins Youngstown State University Probabilistic Smart Terrain ICTAI Multiple Needs Characters can have multiple needs –Hunger, Fun Some needs more critical than others –Hunger = 10Fun = 5 Objects may only partially fulfill needs –Donuts: Hunger-7 –Cookies: Hunger-3 –TV: Fun-6 Needs increase over time (each tile traversed) –Hunger += 0.5 per tileFun += 0.2 per tile

John Sullins Youngstown State University Probabilistic Smart Terrain ICTAI Multiple Needs Goal: Minimize total “discontentment” Σ (need j ) 2 j Problem: Balancing different factors

John Sullins Youngstown State University Probabilistic Smart Terrain ICTAI Multiple Needs Terminology: –n j = current level of need j –d i j = distance to object i –p i j = probability object i meets need j –a i j = amount that need j decreased by if it meets need) –c j = increase in need j for each tile traversed Expected decrease in need j caused by all objects i = Σ p i j a i j within t tiles i

John Sullins Youngstown State University Probabilistic Smart Terrain ICTAI Multiple Needs Expected level of need j if move t tiles: max (0, n j + tc j - Σ p i j a i j ) where d i < t i Expected discontentment if move t tiles: Σ ( max (0, n j + tc j - Σ p i j a i j ) ) 2 where d i < t j i

John Sullins Youngstown State University Probabilistic Smart Terrain ICTAI Multiple Needs Total expected discontentment at given tile: t max Σ Σ ( max (0, n j + tc j - Σ p i j a i j ) ) 2 t j i Compute for surrounding tiles Move to tile with lowest expected discontentment