CIS 488/588 Bruce R. Maxim UM-Dearborn Target Selection CIS 488/588 Bruce R. Maxim UM-Dearborn 12/6/2018

Case Study - 1 Important problem space attributes affecting target selection Distance form origin to target Distance between enemy position and target Distance between target and estimated enemy position Relative angle between projectile trajectory and velocity of target 12/6/2018

12/6/2018

Case Study - 2 Important animat attributes affecting target selection Angular divergence from animat heading to target as well as the angular velocities involved Speed of travel and direction of movement affect ability to hit targets Should be possible to create a perceptron that can guess how much damage a projectile will inflict on the enemy and use this to assist in target detection 12/6/2018

Design Rationale - 1 A fully connected, feed-forward MLP module should do the job It would be nice for the animat to learn target selection on-line A good compromise would be to gather game data on-line, but actually is done off-line This allows the designer some control over features to emphasize and provide some guidance to the learning process 12/6/2018

Design Rationale - 2 Having lots of data is good, since it allows assessment of the impact of noise on a game Since no two fights are ever the same in a good game, noise can be expected to affect learning a great deal Might need to create statistical profile for the MLP to learn from 12/6/2018

Module Design Initialization in XML <layer inputs=“4” units=“8”/> <layer units=“1”/> Interfaces (both incremental and batch) void Run(const vector<float>& input, const vector<float>& output); float Sample(const vector<float>& input, void Randomize( ); float Batch(const vector<Pattern>& inputs, const vector<Pattern>& outputs); 12/6/2018

Data Structures Array of layers Each layer contains a set of neurons plus output array for layer Training data stored separately Activation function derivative also stored along with the gradient error for each neuron Layers also store weight deltas used to allow momentum in steepest descent 12/6/2018

Simulation and Learning Simulation is set of nested loops processing the layers in turn Each loop is located in its own function (except for innermost loop) Learning algorithms perform forward simulation to determine derivatives and gradients Backpropagation of error is then used to adjust the neuron gradients 12/6/2018

Algorithm Outline // Selects best target by predicting damage // Obstacles are checked before calling function function select_target { repeat // propose target spot near enemy target = position + randvec( ); // perceptron predicts hit probability value = estimate_damage(target); until value > threshold; // stop if big enough return target; } 12/6/2018

Tracking Rockets - 1 AI must be able to answer the following Did projectile explode nearby? Was it another player’s projectile? Was there any damage from projectile? What conditions present when shot fired? To prevent misinterpretations each animat is only allowed to have one rocket in the air at a time 12/6/2018

Tracking Rockets - 2 Information gathering When rocket fired start tracking and remember initial conditions When sensor detects noise, see if it was caused by a rocket If rocket explosion detected set target to point of collision and expected time of collision to current time When collision time is past use data to train NN 12/6/2018

Dealing with Noise Simulation data contains lots of noise Game situations have been simplified Features gathered from situations are limited Environment is unpredictable since it contains autonomous agents Gathering on-line data is real tough Better to gather lots of data from several animats and log it for later analysis and off-line training 12/6/2018

Inputs and Outputs Chosen from distances and dot products using four points in space Player origin Enemy position Estimated position Chosen target Output is Boolean value indicating whether damage was inflicted or not 12/6/2018

12/6/2018

Training Bots are not given unlimited rockets They are expected to pick up a rocket launcher and forage for ammunition During testing rockets are freely available Animats without ammo serve as good targets any way Learning can be sped up by running quake in server mode without graphics 12/6/2018

Splash Predicts enemy position and generates a random target around it A perceptron is used to evaluate the likelihood of success Rocket is fired if the chances of success are acceptable Training data is gathered when expected collision time is past 12/6/2018

Evaluation - 1 Noisy environment, average error rate 25% Takes batch algorithms hundreds of training periods (epochs) to become competent The NN visibly improves the animat’s shooting abilities Perceptron often aims at floor near enemy tends to favor spots Close to estimate Close to enemy current position Close to rocket origin 12/6/2018

Evaluation - 2 Perceptron good at finding flaws in suggestions made by target generation mechanism Prevents kamikaze shots, by enforcing minimal distance to target Perceptron only improves target selection according to its experience Does not seem to learn “stupid” behaviors 12/6/2018