1. Neural Networks References: “Artificial Intelligence for Games” "Artificial Intelligence: A new Synthesis"

2. History In the 70’s it was *THE* approach to AI – Since then, not so much. Although it has crept back into some apps. – It still has it’s uses. Many different types of Neural Nets – (Multi-layer) Feed-forward (perceptron) – Hebbian – Recurrent

3. Overview Givens: – A set of n training cases (each represented by a m-bit “binary string”) – The “correct” p-bit answer for each training case. Supervised vs. Unsupervised Process: – Train a neural network to “learn” the training cases. Using the Neural Network. – When exposed to new (grey) inputs, it should pick the closest match. Better than decision trees in this respect. Basically: a classifier.

4. Example Application #1 Modeling a enemy AI. The AI can do one of these 4 actions: – Go for cover – Shoot nearest enemy – Run away – Idle Training cases: – 50 sample play-throughs, 40 samples (2000 cases) with: position of player (16 bits) health of player (6 bits) ammo-state of player (8 bits) – One of the following: Supervised training: The action the AI *should* have taken. Unsupervised training: A way to measure success (based on player/AI state)

5. Example Application #2 Training Cases: – 8 8x8 pixel images (black/white) (64 bits each) – For each a classification number (3 bits each) This is supervised learning. Use: – After training, feed it some images similar to one of the training cases (but not the same) – It *should* return the best classification number.

6. Perceptrons Modeled after a single neuron. Components: – Dendrites (Input) – Axon (Output) – Soma (Activation Function)

7. Part I: Perceptrons

8. Perceptron Activation functions

9. Perceptron Activation functions, cont. To convert the output of the sigmoid to a "crisp" value (when this is an output from the nnet), you count a value >= 0.5 as 1.

10. Perceptron Activation functions, cont.

11. Examples 1.0 I0I0 I1I1 Perceptron O0O0 w0w0 w1w1 w 2 =- δ

12. Why not XOR? The first two examples are linearly separable W2=δW2=δ In higher dimensions, the “dividing line” is a hyper-plane, not a plane. OUTPUT=0 OUTPUT=1

13. AND and OR δ=0.75 δ=0.4 AND OR

14. XOR You can’t draw a line to separate the “True”’s from the “False”’s

15. Multi-layer perceptron networks We can’t solve XOR with one perceptron, but we can solve it with 3. – The 0 th one in the first layer looks for [0, 1] – The 1 st one in the first layer looks for [1, 0] – The only one in the second layer OR’s the output of the first layer’s perceptrons. 0.2 0.3

16. XOR NNet 1.0 I0I0 I1I1 Perceptron 00 O0O0 Perceptron 01 Perceptron 10 1.0 Notation: W ij h = Weight to layer h, from input#j to perceptron#i W 00 0 W 10 0 W 01 0 W 11 0 W 02 0 W 12 0 W 00 1 W 01 1 W 02 1 hIjW ji h 000-0.5 0010.4 0100.5 011-0.6 020-0.2 021-0.3 1000.5 100 120-0.4

17. Weight matrices + Feed-Forward

18. Feed Forward Example

19. Feed Forward Example, cont.

20. Part II: Training

21. Training intro

22. Training intro, cont. Observations: – If d == o, no change in W – d – o could be -1 or +1. -1 means the output was too high. Decreases weight +1 means the output was too low. Increase weight. – The f’ term indicates how “changeable” the value is f’ of 0 (when f is nearly 1 or nearly 0) means it’s “locked-in” f’ of 0.25 (when f is near 0.5) means it’s very changeable.

23. Training example Single perceptron. 2 inputs, one output. – Weights initially [0.3, 0.7, 0.0] Scenario#1 In=[0, 1,1]. DesiredOutput=0 Feed forward. f(0.7)=0.668. f’(0.7)=0.224. Err=-0.668 W += 0.1 * (0 – 0.668) * 0.224 * [0, 1,1] W is now [0.3, 0.685,-0.015] Feed forward. f(0.685)=0.664. f'(0.685)=0.223. Err=-0.661 If we repeat this 299 more times: Err=-0.032 Scenario#2 In=[0,1,1]. DesiredOutput=1 Feed forward. f(0.7)=0.668. f'(0.7)=0.224. Err=0.332 W += 0.1 * (1 – 0.668) * 0.224 * [0,1,1] W is now [0.3, 0.707, 0.007] Feed forward. f(0.715)=0.671. f'(0.715)=0.22. Err=0.329 If we repeat this 299 more times: Err=0.055

24. Training example, cont. Basically, we're… – Adjusting weights: We subtract from the weight if the error is negative and we have a 1 for input – Negative Error == output is too high. We add to the weight if the error is positive and we have a 1 for input. – Positive Error == output is too low. If the input is 0, leave weight unchanged. – The threshold ( δ ) is a weight too, where the input is always 1. So even if the inputs are all 0, we could still change the threshold.

25. Training intro, cont. You repeatedly show examples, I – Perhaps in a random order Weight values will stabilize at “correct” values. – *IF* the answers are linearly separable (like AND; unlike XOR) – If they aren’t it, will at least minimize the error.

26. Training Hidden Layers A bit more complex… The last weight matrix ("below" the output) is trained just as with a single perceptron – Treat the rows as the weight vectors. – The error is the difference between the desired output and actual output (not "crispified") – Row i in the weight matrix is considered the weight vector For other matrices – The error is the error from the layer above weighted proportional to the (current) weight values feeding into that layer. Assigning blame

27. Training Hidden Layers Updating W i : If i is the last (top) matrix: Else: W Updating weights feeding into a hidden perceptron: – Divide up the error of perceptrons this one feeds into (assigning blame) – w k is the weight from the node we're feeding into to the kth node in the next layer up. – err k is the error of the kth node in the next layer up.

28. P 0,0 P 0,1 P 0,2 P 0,3 P 1,0 P 1,1 P 1,2 P 2,0 P 2,1 Training Hidden Layer example P 1,0 HErr 1,0 =ErrW 2 col(0) P 1,1 HErr 1,1 =ErrW 2 col(1) P 1,2 HErr 1,2 =ErrW 2 col(2) P 0,0 HErr 0,0 =HErr 1 W 1 col(0) P 0,1 HErr 0,1 =HErr 1 W 1 col(1) P 0,2 HErr 0,2 =HErr 1 W 1 col(2) P 0,3 HErr 0,3 =HErr 1 W 1 col(3) 1.0 P 2,0 Err 0 = d 0 – o 0 P 2,1 Err 1 = d 1 – o 1 I0I0 I1I1 I2I2 1.0