2. RECAP CSE 348 AI Game Programming Héctor Muñoz-Avila

3. Course Goal Our goal was to understand the connections and the misconceptions from both sides AI research A C B ABC AC B C B A B A C B A C BC A C A B A C B B C A AB C A B C A B C “AI” as game practitioners implemented it projects

4. Controlling the AI Opponent: FSMs FSM: States, Events and Actions Stack Based FSM’s Polymorphic FSM Multi-tier FSM Spawn D Wander ~E,~S,~D ~E D Attack E,~D ~E E E D ~S Chase S,~E,~D E S S D Soldier RiflemanOfficer BritishSoviet AmericanGerman Machine Gunner BritishSoviet AmericanGerman BritishSoviet AmericanGerman Robocode Planning Operators Patrol  Preconditions: No Monster  Effects: patrolled Fight  Preconditions: Monster in sight  Effects: No Monster PatrolFight Monster In Sight No Monster FSM: A resulting plan: Patrol patrolled Fight No Monster Monster in sight

5. Controlling the AI Opponent: HFSMs Start Turn Right Go-through Door Pick-up Powerup Wander Attack Chase Spawn ~E E ~S S D ~E UT task: Domination Strategy: secure most locations UT action: move Bot1 to location B

6. Controlling AI Opponent: Scripting (Nick Haynes) Wargus  Autonomous agents calculate their action based on… Desires Sensory Input Proximity to items of interest (Jon Martin) 1 1 1 1 1 Space reservation: quasi-coordination

7. Controlling AI Opponent: Team Multi-layered approach Line of sight (player, npcs) GOAP: Agent can dynamically find alternate solutions to problems (Eric Lease) Dead Reckoning Predicting future state For games: Newton physics Estimate future trajectory: Kinematics Dayne Mickelson (Dayne Mickelson) Team sports Identify high-level decisions Unreal tournament

8. Learning: Adaptive Behavior (Megan Vasta) Neural networks Dynamic scripting: Reinforcement learning But sometimes the problem resides in the scripts not the ordering Use evolutionary computation to improve scripts Combat team controlled by human player team controlled by computer A B  Evolve a population (each member is a candidate solution) …

9. Learning: Adaptive Behavior (2) User model Flexibility beyond predefined difficulty levels When/what to update (Brigette Swan) Allegiance Defense Friendly Enemy 0.4-0.3 WeakStrong 0.1 Medium Induced from a collection of data Based on information gain formulas Assume discrete values (Jeff Storey)

10. Learning: Adaptive Behavior (3) Pattern recognition 1.Symbols 2.Optimization: balancing units in an RTS game 2. Curse of dimensionalit Analysis of Machine learning Usage 1. Cheap to recognize what to learn from? 2. Cheap to store the knowledge? 3. Cheap to use the knowledge? 4. Does game benefit from learning? (Chris Kramer)

11. Spatial Analysis Terrain analysis: Concepts: borders, corridors Selection of new colonies Spatial Analysis: (Jay Shipper ) Random map generator: Location of players Map is generated step-wise by adding clumps Transport units in RTS games: (Russell Kuchar)

12. Spatial Analysis (2) Wall generation Graph representation: (tiles, connections) Greedy algorithm Hierarchy in RTS games Rami Khouri (Rami Khouri)

13. Path-Finding (1) A*: minimize f(n) = g(n) + h(n) Dan Bader (Dan Bader) Rep. simplicity versus optimality Grid Graphs Meshes String pulling - Can be used to compute AI (Tom Gianos) Navigation set hierarchy Interface tables Reduction memory Increase performance

14. Path-Finding (2) (Owen Cummings) Path Look-Up tables Several times faster than A*Several times faster than A* But memory consumption is highBut memory consumption is high Solution: Area-based Look-up tables Notion of portalsNotion of portals Very fastVery fast (Tom Schaible) Flying Edge Door Edge Vault Edge Jump Edge Rappelling Edge Throwing a grenade is not so simple! Add information to nodes Add behavior info in edges Hunting players in a convincing manner

15. Path-Finding (3) a Obstacle RaRa DaDa VaVa Sidestep Repulsion (Don DeLorenzo) Avoiding obstacles Should be smooth Crucial in dynamic worlds Intelligent Steering Use error correction: current error + history error + rate error (Adam Balgach) Racing vehicle control Multi-layer system Each layer defines behavior Optimal racing line Use of Newton physics

16. Game theory Spectimax kind of search Declarative Knowledge A C BC B A Initial state Goals HTN approach for declarer play –Use HTN planning to generate a game tree in which each move corresponds to a different strategy, not a different card Reduces average game-tree size to about 26,000 leaf nodes Compute expectimax and expectimin Evaluation functions Pruning search space poker

17. Game Design (Peter Shankar) “Meaningful play” Outcome is discernable and integratedOutcome is discernable and integrated Elements for meaningful play: SemioticsSemiotics SystemsSystems InteractivityInteractivity ChoiceChoice Cultural System Experiential System Formal System Sid Mier says: “personal touch” is needed

18. Hall of Fame Winners Project 1: Tournament: Adam Balgach, Tom Gianos. Bot: Yankees Innovation: Tom Shaible, Don Delorenzo. For: "meta-level" FSM design of code. Winners Project 2: Tournament: Adam Balgach, Tom Gianos. Team: Yankees (continuing champions!) Innovation: Swan, Brigette L, Vasta, Megan E., and Khouri, Rami H. For: a number of interesting ideas: predicting next place for firing, distributing battlefield, training examples. Winners Project 3: Project # 3 was no tournament. Winners Project 4: Tournament: Adam Balgach, Tom Gianos. Team: Yankees (unbeatable!) Winners Project 5: Tournament: Tom Schaible, Don Delorenzo. Team: DDTS (new champions!) Winners Project 6: Tournament:. Owen Cummings, Dayne Mickelson Team: Tony Wonder (new champions!) Innovation: Tom Shaible, Don Delorenzo. For: decision trees and reinforcement learning

19. Acknowledgements Jon Martin and Eric Lease All of you: –Presentations were very good –Projects were worked well (despite difficulties) –Changes: 4 projects: robocode, UT, MadRTS, poker UT: 2 bots only Poker: use downloadable version

20. The End…