1. Hopefully a clearer version of Neural Network

2. I1 O2 O1 H1 H2I2

3. Layers of Weights We Name Sets of Weights between layers As W1 for weights between input Layer and First Hidden Layer W2 for weights between next 2 layers and WN-1 for Weights between N-1th and Nth Layer(i.e. Output Layer) In our example Net we just have 3 layers Input Hidden and Output So we have just W1 and W2

4. I1 O2 O1 H1 H2I2 W1 W2

5. Weights along Individual Links Convention Each Weight is named as follows WNij N refers to the Layer of Weights So Between Input and First Hiden Layer i.e. W2ij is the Reference Between Hidden and Output W2ij

6. Individual Weights within a layer Reference WNij WN refers to the Weight Layer ij refers to the indices of the source and destination nodes. So for example the weight between hidden node h1 and output node o2 It belongs to weight layer 2 so W2 i = 1 and j = 2 so Weight is W2 12

7. I1 O2 O1 H1 H2I2 W1 W2 W2 12

8. Full Naming of Weight Set

9. I1 O2 O1 H1 H2I2 W1 W2 W2 12 W1 12 W2 21 W2 11 W2 22 W1 21 W1 11 W1 22

10. With Actual Weights

11. I1 O2 O1 H1 H2I2 W1 W2 0 0 0 1 1

12. Inputs 1 and 0 Target outputs {0.7,0.6}

13. I1 O2 O1 H1 H2I2 W1 W2 0 0 0 1 1 1 0

14. Hidden Layer Computation Xi =iW1 = 1 * 1 + 0 * -1 = 1, 1 * -1 + 0 * 1 = -1 = { 1 - 1} = {Xi1,Xi2} = Xi

15. h = F(X) h1 = F(Xi1) = F(1) h2 = F(Xi2) = F(-1)

16. I1 O2 O1 H1 H2I2 W1 W2 0 0 0 1 1 1 0 0.73 0.27

17. Next Outputs

18. Output Layer Computation X = hW2 = 0.73 * -1 + 0.27 * 0 = -0.73, 0.73 * 0 + 0.27 * -1 = -0.27 = { -0.73 - 0.27} = {X1,X2} = X

19. O = F(X) O1 = F(X1) O2 = F(X2)

20. I1 O2 O1 H1 H2I2 W1 W2 0 0 0 1 1 1 0 0.73 0.27 0.325 0.433

21. Error D= Output(1 – Output)(Target – Output) Target T1 = 0.7, O1 = 0.325 = 0.33 d1 = 0.33( 1 -0.33)(0.7 -0.33 ) = 0.33 (0.67)(0.37) = 0.082 Target T2 = 0.6, O2 = 0.433 = 0.43 d2 = 0.43(1 - 0.43)(0.6-0.43) = 0.43(0.57)(0.17) = 0.42

22. Weight Adjustment △ W2t = α hd + Θ △ W2t-1 where α = 1 Time t = 1 so no previous time

23. Weight Adjustments

24. Weight Change

25. Equals

27. Putting these new weights in the diagram To get

28. I1 O2 O1 H1 H2I2 W1 W2 0.031 0.022 -0.94 -0.988 0 1 1

29. Next Calculate Change on W1 layer weights

30. Error Calculation e = h(1 - h)W2d

31. Another Way to write the error

32. What is this Outputs are O1 and O2 So k = {1,2} So if i = 1

34. I1 O2 O1 H1 H2I2 W1 W2 0.031 0.022 -0.94 -0.988 0 1 1

35. This equals e1 = (h1(1-h1)W11 D1 +W12D2 e2 = (h2(1-h2)) W21 D1 +W22D2 d1 = 0.082 d2 = = 0.042 e1 = (0.73(1-0.73))( -1* 0.082 +0*0.042) e2 =( 0.27(1-0.27)) (0 *0.082 +-1*0.042) e1 = (0.73(0.27)( -0.082)) e2 =( 0.27(0.73)) (-0.042) e1 = -0.016 e2 = -0.0083

36. Weight Adjustment △ W1t = α Ie + Θ △ W2t-1 where α = 1

37. Weight Adjustment

38. Existing W1

39. Weight Change W1

40. New W1

41. Changing Net

42. I1 O2 O1 H1 H2I2 W1 W2 -0.102 -1.0083 -0.04 -1.109 -1.038 0 0.884 1