--Lu Xu REAL-TIME STRATEGY GAMES (RTS)

Outline History of RTS Architecting an RTS AI An Economic Approach to Goal-Directed Reasoning in an RTS Transport Unit AI for Strategy Games Wall Building for RTS Games Prioritizing Actions in a Goal-Based RTS AI RTS Citizen Unit AI RTS Terrain Analysis

HISTORY OF RTS

History – The Genesis of RTS 1 st instance of real-time strategy: Herzog Zwei (Sega Genesis) – 1989  a two-player game in which the object is to destroy the enemy's base 1 st PC real-time strategy game Dune II (Westwood) Improvement upon Herzog Zwei  you can build bases anywhere  the game introduces structure dependencies  To produce harvesters and tanks, you have to build a heavy factory. But to build the heavy factory, you first need a light factory. And so on.

History – The First Wave Warcraft: Orcs & Humans (Blizzard) – 1994 Warcraft opts for a more personal combat motif: hand-to-hand fighting. Provide multiplayer capability Random map generator (1 st for RTS) Command & Conquer (Westwood Studios) – 1995 Online-only spin-off called Sole Survivor, in which you play a single unit and fight other players in a deathmatch-style competition. Can find power-ups such as health, invisibility, armor, and the like.

History – The Sequels Warcraft II: Tides of Darkness (Blizzard) – 1995 SVGA graphics AI improved Add fog of war had the ability to be a superb multiplayer game that allowed for surprisingly lag-free gaming, even on slow connections Command & Conquer: Red Alert (Westwood)

History – the Second Generation Total Annihilation (Cavedog) – st game to have 3D terrain and units Units can be given multiple orders Personalizes the game with the introduction of a "commander" unit No real story, only some spoken intros to the various missions Dark Reign (Auran) – 1997 A very sophisticated waypoint system, as well as individual unit AI settings. Starcraft (Blizzard) – 1998 the most successful of early real-time strategy games

History – The Early Fruits of RTS Evolution Close Combat series (1995) introduced the frantic action of real-time gaming to the realistic weapon modeling of wargames the game was based on the simulation of unit morale Baldur’s Gate (1998) RTS  RPGs 1 st take the lasso-your-units-and-click paradigm Myth series (1997) all the resource management and concentrated on simple unit tactics. Myth II -- a niche in the real-time market for those more interested in combat than in resource gathering.

ARCHITECTING AN RTS AI

Architecting an RTS AI The components that make up an RTS game architecture consist of what some call “managers” – each responsible for a specific area in the management of a computer player RTS Game Components Civilization Build Unit Resource Research Combat

Civilization Highest-level manager for responsible for development of the computer player’s economy Responsible for coordination between the build, unit, resource, and research managers, as well as setting spending limits Handles computer player expansion, upgrading of buildings and units, and epoch advancement

Build Responsible for placement of the structures and towns Receives requests from the unit manager for training buildings and from the civilization manager for support buildings All site evaluation occurs here Most buildings have requirements on where they can and cannot be placed

Unit Responsible for training units Keeps track of what units are in training at various buildings Monitors the computer player’s population limit Prioritizes unit training requests Communicate with the build manager if a training building is not currently available Attempt to maintain a unit count similar to the human players in the game – not allowing the AI unit counts to get too high

Resource & Research Resource Responsible for tasking citizens to gather resources in response to requests from both the unit and build managers Load balances gathering duties and governs the expansion of the computer player to newly explored resource sites Research Responsible for the directed research of the computer player Technologies are examined and selected based on their usefulness and cost

Combat Responsible for directing military units on the battlefield Requests units to be trained via the unit manager and deploys them in whatever offensive or defensive position is most beneficial Keeps track of units via a personnel manager, and handles offensive and defensive duties via similarly named managers Periodically updates the status, movement, and actions of various attack groups, while these attack groups have subgroups that comprise like units

Difficulty Levels Create a hard AI first and reduce its intelligence for lower levels Going the other way is much more difficult to achieve Decide that modifications to make to differentiate your difficulty levels Brainstorming  narrow down list have the most impact on the effectiveness of the AI  test Cheating

Challenges Random Maps Most troubling of the sticky spots, account for as much as half of the development time Wall Building Keep enemies away from your economic centers Island Hopping Resource management Stalling Labyrinthine interactions between buildings, units, and resources that can cause the computer player to get stuck waiting for one thing or another, resulting in a computer player that cannot advance

AN ECONOMIC APPROACH TO GOAL- DIRECTED REASONING IN AN RTS

An Economic Approach to Goal-Directed Reasoning in an RTS Goal-Directed Reasoning Reasoning in a state space is conducted either forward from an initial state, or backward from a goal Start with a given goal and locate all of the moves that could generate the goal state  repeat the process for each states Unpredictability Large number of possible moves The nature of human competition Solution -- Reactive system

The Utility Model Utility – the amount of satisfaction that a consumer receives from a product, or how desirable the product is Heuristic function – estimates whether a given state is on the path to the goal by approximating the desirability of the states Utility Model’s key concept – using a measurement of utility as the heuristic for evaluating and choosing goals Marginal Utility (MU) – additional satisfaction gained from the consumption of one addition item Theory of Diminishing Marginal Utility

An Example RTS: Rock-Paper-Scissors

Production Possibilities

Cost of Goals versus Cost of Tasks

The Combat Domain Can be modeled by a Utility Model that is chained to the economic Utility Model The outputs of the economic model – units – become the inputs to the combat model Example Suppose combat goal is to defeat the player’s army of Papers The utility of combat objects would be tied to their combat seats’ units that compare favorably to Papers in combat (Scissors) would have a higher utility than those that compare poorly (Rocks)

TRANSPORT UNIT AI FOR STRATEGY GAMES

Transport Unit AI for Strategy games Transports move passengers around the map Passengers are land-based units Unit AI  Micro level for specific piece Challenges: Many units must work together to achieve common goal. Attempt to minimize player frustration. Adhere to expected RTS transport standards

Transport Class Hierarchy 4 main types of transports Transport Air Transport Building Transport Land Transport Naval Transport helicopter balloon bunker city hall horse armored personnel carrier ships subs

Basic Solution All units have high level goals. Goals are broken down and implemented through a finite-state machine (FSM) Loading describes the high-level goal that units are given when a player instructs those units to board a transport Unloading is the high-level goal given to a transport unit when a player instructs that transport to disembark its passenger

Passenger FSM Cannot unload Transports handle unloading their cargo

Transport FSM Load Rendezvous point Unload Unload point

Loading Loading is the high-level goal for units that need to enter a transport Both passengers and transports are given a load goal, although they each have their own role to play for this goal Loading is simpler than uploading To load, all units involved must agree on some rendezvous point where they will meet

Loading (Cont.) Rendezvous Point Testing for Passengers in Loading Range Air Transports Building Transports Land Transports Naval Transports

Unloading Unloading is the high-level goal given to transports to disembark their passengers Players give the transport a particular location in which to unload The unload point is then modified so that it is a location that the transport can actually reach The transport then tries to get as close as it can to the unload point Once the transport is with its unload range from the unload point, the passengers can exit the transport Use sub-tile placement to keep them from colliding with other passengers, other units, and terrain features

Unload Point Unload point (UP) is key to unloading process Given the transport point (T) and the player’s clicked point (CP) There are actually three unload points: The Transport destination point (TDP) The first passenger point (FPP) The unload point Air Transports Building Transports Land Transports Naval Transports

WALL BUILDING FOR RTS GAMES

Wall Building for RTS Games A Wall is a set of wall segments connected together in such a way that: Every wall segment has exactly two unique adjacent wall segments (neighbors). It might not be obvious. But this requirement implies that wall segments are linked in circular fashion There is at least one interior tile All interior tiles are connected through edges (diagonal connections do not count)

Examples of Walls and Wall Segments that do and do not form Walls

Wall-building Algorithm Greedy Algorithms Usually fast and fairly easy to implement But there is no guarantee that a series of locally optimal steps will yield a globally optimal final solution  Dijkstra’s Single-Source Shortest Path Algorithm, guarantee an optimal solution

A Move in Greedy Method A move is the process of removing an existing wall segment and placing in it at a different location in such a way that: There is a net gain of exactly one tile in an area that is walled off (# of interior tiles increases by one) The resulting wall still meets the definition Definition: All moves have outward direction (1 st requirement) It implies that some moves might require addition of extra wall segments (2 nd requirement)

Example Group 1Group 2 Group 3Group 4 Any given wall segment and its two neighbors, a wall segment triplet, can be arranged only in a limited number of ways Case 1Case 2Case 3Case 1Case 2Case 3 Case 1Case 2Case 3 Case 1Case 2 Case 3

Example (Cont.) Group 1, Case 1: four permutations Group 1, Case 2: nine permutations Group 1, Case 3: four permutations Each group could generate 17 wall permutations Adding the new wall segments to the existing triplets

Problem with This Paradigm To push the wall outward, we must have a case for every possible wall configuration. 68 cases if tiles are squares, much more if tiles are hexagonal, more if their octagonal Special cases if the wall is adjacent to a natural barrier. Thinking of the problem in terms of pushing the wall is complex

Change the paradigm Instead of pushing the walls, expand the inner area Represent the tiles as vertices, and connections between tiles as edges Now we can have as many edges as we want coming out of a node

Using the Algorithm Keep three lists: List of all nodes & edges, graph Closed list: all nodes in the interior space of the wall Open list: all nodes that are candidates for expansion, in practice this is the perimeter of the enclosed area, i.e. spaces that the wall is occupying Closed Open Graph

Using the Algorithm Understanding its three controlling functions Traversal function  Finds successor nodes Heuristic function  Ranks open nodes Acceptance function – Control the algorithm and determines when to stop  Decides when the wall meets given criteria Taking advantage of natural barriers Lower the cost of walling off a certain area Allow us to wall of large areas for the same cost Control the interior area expansion

PRIORITIZING ACTIONS IN A GOAL-BASED RTS AI

Prioritizing Actions in a Goal-Based RTS AI The Goal Engine Architecture Responsible for making all strategic-level decisions in the opposing player AI Decides what action to take, where to take those actions, and what resources to apply to each Basic elements of the engine: Actors Goals Egos The Think Cycle The Reactive AI

Actors The things in the game that are controlled by human or AI players, or that can have an effect on the game Buildings are immobile actors Defensive structures are buildings that can attack things Economic building are buildings that provide some nonmilitary advantage Units are actors that can move Military units are units that can attack things Builders are units that construct and/or repair buildings Actor’s combat value (CV)

Goals Represent every action a player can take in the game Common types include attack, defend, explore, recruit, construct, and repair Each goal has a status Inactive Active Selected Finished Priority Base priority Current priority

Egos & The Think Cycle Ego A single set of all the values that define the AI’s behavior Each AI can have multiple egos, which it switches between depending on the current state of the game The Think Cycle A goal engine think, as one iteration of this cycle is called, is expensive Select ego  identify inactive and finished goals  calculate the base priority for each active goal  assign initial resources to the goals  optimize resource allocation

Generating Goal Priorities Some common tools Multipliers Exponents Minimum values Maximum values Inversion (1/x) Bonuses One-time bonuses Repeat penalties Random bonuses

Some techniques to generate good goal priorities Fuzzy Factor, Goal Inertia, and Goal commitment The Value of Money Actor Value Build Templates Map Analysis Regions High-level pathfinding and path caching Military influence

The Attack Goal Calculation of base priority and current priority CV Ratio

The Attack Goal (Cont.) If CV ration < target CV, apply this formula: where exp > 1, and is specified in the ego Otherwise, if CV ration < Max [CV], apply: If neither of those cases holds, the CV ratio >= max[cv], we apply MaxMult

RTS CITIZEN UNIT AI

RTS Citizen Unit AI Goals and Unit AI FSMs Building and Repairing

RTS Citizen Unit AI (Cont.) Build site selection A unit is in range of another unit if the two collision discs touch or overlap

RTS Citizen Unit AI (Cont.) Citizen build queues Queue of goals Allows the player to issue multiple commands to the citizen with a few mouse clicks Three-phase building construction Foundation  Only visible to the owning player and has no interaction with the world Scaffolding  Can be seen by all players Completed building  When the scaffolding reaches the maximum hit point value  Fully functional: train units, store resources, & research technology

Gathering Resource gather location Waiting area Finding storage Gathering Smart citizens Building, not repairing Auto build Wall building Finding storage revisited

RTS TERRAIN ANALYSIS

RTS Terrain Analysis: An Image-Processing Approach Determine what information is needed by the AI Are my troops together in a single logical area of the map? Are there enemy forces in the same area as my units? Does my area have connections to others? When defending an area, how should I place my troops? When attacking, how should I approach the enemy? Are there strategically important elements in my area (e.g., bunkers, cities, or bridges)?

Geometric Terrain Analysis Techniques Analyze the map Build a graph representation using geometric information Generalized Voronoi diagrams Very useful for dividing the map into different areas that can later be classified by other algorithms Not a definitive solution

Pathfinding Simulation Analysis Techniques Try to detect important paths on the map Find avenues of approach ANTS Fail to provide all the needed information No information about logical areas No information about connections No information from the paths about which of their waypoints are strategically important

Basic Concepts of Algorithm The passability Map A black-and-white image in which each pixel represents 1 square meter on the terrain map Regions, connections, and hotspots Regions – areas of passable terrain that are delimited either by impassable terrain or by other regions of a different type  Game zones  Chokepoints Connections – logical links between chokepoints and game zones Hotspot – the best position to defend a game zone from attacks coming through a particular chokepoint

The Terrain Analysis Algorithm

Noise reduction

The Terrain Analysis Algorithm (Cont.) Random Pathfinding Iterative Image Processing Combining image processing and pathfinding data Analysis selection Data extraction

The Terrain Analysis Algorithm (Cont.)