1. CS 621 Artificial Intelligence Lecture 29 – 22/10/05
Prof. Pushpak Bhattacharyya
SOM, Theoretical Aspects Of Machine Learning, Probably Approximately Correct Learning.

2. A …
P neurons o/p layer Wp
… … .
n neurons
Clusters:
A : A
B : :
C :

3. Wp(n+1) = Wp(n) + η (I(n) – Wp(n))
Competitive learning: Weights to the winner neuron only are changed.

4. Generating function application to recurrence relation solving.
Example: Max. No of regions on 2-Dim plane produced by n lines.

5. L1 L3 y L2
R1 = 2
R2 = R1 + 2
= 2+ 2
= 4 L4
R3 = R2 + 3
= = 7
R4 = = Rn = Rn-1 + n R1 = 2 x

6. Rn = Rn-1 + n Rn-1 = Rn-2 + (n-1) Rn-2 = Rn-3 + (n-2) . R2 = R1 + 2

7. Rn = n + (n-1) + (n-2)+ …..+R1 + 2
= n+(n-1)+(n-2)+…..+2+2
=n(n+1)/

8. Generating Function Method
Key idea: Rns are generated by infinite series function ,i.e,
F(x) = R1 + R2x + R3x2 + ………………+ Rnxn-1+ ….. ∞
Rn = Coeff of xn-1

9. Rn = Rn-1 + n Rn - Rn-1 = n R2 - R1 = 2 R3 – R2 = 3 .
(Continued)
Rn = Rn-1 + n
Rn - Rn-1 = n
R2 - R1 = 2
R3 – R2 = 3 .

10. (Continued) F(x) = R1 + R2x + R3x2 + ………………+ Rnxn-1+ ….. ∞ (1)
xF(x) = R1x + R2x2 + R3x3 + ……………+ Rnxn+ ….. ∞ (2)
- 2 gives
(1-x) F(x) = R1 + (R2 – R1)x + (R3 – R2)x2 + ……………+ (Rn – Rn-1)xn-1+ ….. ∞

11. (Continued) (1-x) F(x) = 2 + 2x + 3x2 + ……………+ nxn-1+ ….. ∞
F(x) = (1-x)-1(2 + 2x + 3x2 + ……………+ nxn-1+ ….. ∞)
= ( 1 + x + x2 + x3 +…+ xn-1+…..)
(2 + 2x + 3x2 + 4x3 +…+ nxn-1+…..)

12. Co eff of Xn-1 from the expression for f(x)
(Continued)
Co eff of Xn-1 from the expression for f(x)
= ( n + n-1 + n-2 + … )
n(n+1)/ 2 + 1

13. Clustering Algos 1. Competitive learning 2. K – means clustering
3. Self organization / Kohonen net
4. Counter Propagation

14. Two packages 1. SNNS – Stuttgart Neural Net Simulator 2. WEKA Package
3. MATLAB

15. Data to train IRIS data: 3 - class data X- OR, parity, Majority

16. IRIS Data
Classification of flowers based on four attributes in 3 classes c1, c2 and C2
1 Petal length A1
2 Petal width A2
3 Sepal length A3
4 Sepal width A4

17. Google on “Machine learning Data”
IRIS
120 – data points
< A1, A2, A3, A4 >1 --- C1
< A1, A2, A3, A4 >2 --- C3
……….

18. C1, C2, C3 each has 50 data points in it
80 – 20 rule for ML
Training – 40 and Testing -- 10

19. What you have to do for IRIS choose 40 points randomly from C1, C2 and C3
1. Run BP on FF n/w
2. Run SOM assuming unlabelled data. See if you can discover 3 cluster

20. Two part assignment Supervised
Hidden layer to be found
4 neurons for 4 attributes

21. Very good performance on test data should be achieved.
BP for IRIS
error 1%
Training iterations
EPOCHS
Very good performance on test data should be achieved.

22. Cluster Discovery By
SOM/Kohenen Net
4 I/p neurons A1 A2 A3 A4

23. K – means clustering
K o/p neurons are required from the knowledge of k clusters being present. ……
26 neurons
Full connection ……
n neurons

24. Steps 1. Initialize the weights randomly.
2. Ik is the vector presented at kth iteration.
3. Find W* such that
|w* - Ik| < |wj - Ik| for all j

25. 4. make W*(new) = W* (old) + η(Ik - w* ).
5. k  K +1 ; go to 3.
6. Go to 2 until the error is below a threshold.

26. K means 1. Initialize weights randomly.
2. Ik in the I/p vector presented at kth.
step of one iteration over all the training data (EPOCH).
3. Find the W* such that |w* - Ik| is minimum.
4. Find W* for Ik+1..

27. 5. See the partitions at the end of the epoch.
6. Find centroid of each partition.
7. Make the weight equal to the centroid of the partition.
8. Take next epoch.

28. 9. Keep doing until the error is below a threshold.
Key idea form competitive learning: Weight change incorporate after every epoch.