1. CIS 488/588 Bruce R. Maxim UM-Dearborn
Target Selection
CIS 488/588
Bruce R. Maxim
UM-Dearborn
12/6/2018

2. Case Study - 1
Important problem space attributes affecting target selection
Distance form origin to target
Distance between enemy position and target
Distance between target and estimated enemy position
Relative angle between projectile trajectory and velocity of target
12/6/2018

3. 12/6/2018

4. Case Study - 2 Important animat attributes affecting target selection
Angular divergence from animat heading to target as well as the angular velocities involved
Speed of travel and direction of movement affect ability to hit targets
Should be possible to create a perceptron that can guess how much damage a projectile will inflict on the enemy and use this to assist in target detection
12/6/2018

5. Design Rationale - 1
A fully connected, feed-forward MLP module should do the job
It would be nice for the animat to learn target selection on-line
A good compromise would be to gather game data on-line, but actually is done off-line
This allows the designer some control over features to emphasize and provide some guidance to the learning process
12/6/2018

6. Design Rationale - 2
Having lots of data is good, since it allows assessment of the impact of noise on a game
Since no two fights are ever the same in a good game, noise can be expected to affect learning a great deal
Might need to create statistical profile for the MLP to learn from
12/6/2018

7. Module Design Initialization in XML
<layer inputs=“4” units=“8”/>
<layer units=“1”/>
Interfaces (both incremental and batch)
void Run(const vector<float>& input,
const vector<float>& output);
float Sample(const vector<float>& input,
void Randomize( );
float Batch(const vector<Pattern>& inputs,
const vector<Pattern>& outputs);
12/6/2018

8. Data Structures Array of layers
Each layer contains a set of neurons plus output array for layer
Training data stored separately
Activation function derivative also stored along with the gradient error for each neuron
Layers also store weight deltas used to allow momentum in steepest descent
12/6/2018

9. Simulation and Learning
Simulation is set of nested loops processing the layers in turn
Each loop is located in its own function (except for innermost loop)
Learning algorithms perform forward simulation to determine derivatives and gradients
Backpropagation of error is then used to adjust the neuron gradients
12/6/2018

10. Algorithm Outline // Selects best target by predicting damage
// Obstacles are checked before calling function
function select_target {
repeat
// propose target spot near enemy
target = position + randvec( );
// perceptron predicts hit probability
value = estimate_damage(target);
until value > threshold; // stop if big enough
return target; }
12/6/2018

11. Tracking Rockets - 1 AI must be able to answer the following
Did projectile explode nearby?
Was it another player’s projectile?
Was there any damage from projectile?
What conditions present when shot fired?
To prevent misinterpretations each animat is only allowed to have one rocket in the air at a time
12/6/2018

12. Tracking Rockets - 2 Information gathering
When rocket fired start tracking and remember initial conditions
When sensor detects noise, see if it was caused by a rocket
If rocket explosion detected set target to point of collision and expected time of collision to current time
When collision time is past use data to train NN
12/6/2018

13. Dealing with Noise Simulation data contains lots of noise
Game situations have been simplified
Features gathered from situations are limited
Environment is unpredictable since it contains autonomous agents
Gathering on-line data is real tough
Better to gather lots of data from several animats and log it for later analysis and off-line training
12/6/2018

14. Inputs and Outputs
Chosen from distances and dot products using four points in space
Player origin
Enemy position
Estimated position
Chosen target
Output is Boolean value indicating whether damage was inflicted or not
12/6/2018

15. 12/6/2018

16. Training Bots are not given unlimited rockets
They are expected to pick up a rocket launcher and forage for ammunition
During testing rockets are freely available
Animats without ammo serve as good targets any way
Learning can be sped up by running quake in server mode without graphics
12/6/2018

17. Splash Predicts enemy position and generates a random target around it
A perceptron is used to evaluate the likelihood of success
Rocket is fired if the chances of success are acceptable
Training data is gathered when expected collision time is past
12/6/2018

18. Evaluation - 1 Noisy environment, average error rate 25%
Takes batch algorithms hundreds of training periods (epochs) to become competent
The NN visibly improves the animat’s shooting abilities
Perceptron often aims at floor near enemy tends to favor spots
Close to estimate
Close to enemy current position
Close to rocket origin
12/6/2018

19. Evaluation - 2
Perceptron good at finding flaws in suggestions made by target generation mechanism
Prevents kamikaze shots, by enforcing minimal distance to target
Perceptron only improves target selection according to its experience
Does not seem to learn “stupid” behaviors
12/6/2018