1. Cellular Automata Evolution : Theory and Applications in Pattern Recognition and Classification Niloy Ganguly

2. Analysis Synthesis Evolution Associative Memory Pattern Classifier PPS Non Linear CA Aim of the Dissertation Additive CA – An important modeling tool Extremely interesting state transition behavior Can mimic complex operations Problem – How to find the exact CA rules which will model a particular application This thesis builds up the general framework and applies it to the special application of Pattern Recognition

3. Analysis Synthesis Evolution Associative Memory Pattern Classifier PPS Non Linear CA Coverage Additive Cellular Automata (CA) ? Analysis Synthesis Evolution Pattern Recognition/Classification  Associative Machine  Pattern Classifier  Classifying Prohibited Pattern Sets for VLSI Testing Associative Memory – More general class of CA

4. Analysis Synthesis Evolution Associative Memory Pattern Classifier PPS Non Linear CA Cellular Automata 50’s - J von Nuemann 80’s - Wolfram Work round the world America - Santafe Institute of Complexity Study Europe - Stephen Bandini, Bastein Chopard VLSI Domain India under Prof. P.Pal.Chaudhuri Late 80’s - Work at IIT KGP Late 90’s - Work at BECDU  Book - Additive Cellular Automata Vol I

5. Analysis Synthesis Evolution Associative Memory Pattern Classifier PPS Non Linear CA Cellular Automata  A computational Model with discrete cells updated synchronously ……….. output Input Combinatio nal Logic Clock From Left Neighbor From Right Neighbor 0/1 2 - State 3- Neighborhood CA Cell

6. Analysis Synthesis Evolution Associative Memory Pattern Classifier PPS Non Linear CA Cellular Automata Combinational Logic can be of 256 types each type is called a rule ……….. Each cell can have 256 different rules Q CL K D Combinatio nal Logic Clock From Left Neighbor From Right Neighbor 2 - State 3- Neighborhood CA Cell

7. Analysis Synthesis Evolution Associative Memory Pattern Classifier PPS Non Linear CA Cellular Automata Combinational Logic can be of 256 types each type is called a rule ……….. Each cell can have 256 different rules 98236226107 4 cell CA with different rules at each cell

8. Analysis Synthesis Evolution Associative Memory Pattern Classifier PPS Non Linear CA CA - State Transition 0011 0111 98236226107 0010 3 7 2 98236226107

9. Analysis Synthesis Evolution Associative Memory Pattern Classifier PPS Non Linear CA State Transition Diagram 915 6 13 712 314 11 5 28 14 10 0 5 15 10 0 4 14 11 1 2 7 13 8 3 6 12 9

10. Analysis Synthesis Evolution Associative Memory Pattern Classifier PPS Non Linear CA Additive Cellular Automata Combinational Logic can be of 15 types ……….. Each cell can have 15 different rules i-1ii+1 XNOR / XOR

11. Analysis Synthesis Evolution Associative Memory Pattern Classifier PPS Non Linear CA Additive Cellular Automata 15

12. Analysis Synthesis Evolution Associative Memory Pattern Classifier PPS Non Linear CA Additive Cellular Automata 60102150204 1 0 0 0 1 1 0 0 0 1 1 1 0 0 0 1 T = 6016551204 1 0 0 0 1 0 0 0 1 0 0 0 0 1 T = 0 1 1 0 F = Linear CA Additive CA

13. Analysis Synthesis Evolution Associative Memory Pattern Classifier PPS Non Linear CA Additive Cellular Automata - Analysis 60102150204 915 6 13 712 314 11 5 28 14 10 0 CA Rules Cycle Structure

14. Analysis Synthesis Evolution Associative Memory Pattern Classifier PPS Non Linear CA Additive Cellular Automata - Analysis 60102150204 CA Rules Cycle Structure and Depth 5 15 10 0 4 14 11 1 2 7 13 8 3 6 12 9

15. Analysis Synthesis Evolution Associative Memory Pattern Classifier PPS Non Linear CA Linear Cellular Automata - Analysis CA Rules 1 0 0 0 0 0 1 1 0 0 0 0 1 0 0 0 0 0 1 1 0 0 0 1 0 T = Characteristic Polynomial (x + 1). (x +1) 2. (x 2 +x + 1) [1(1), 1(1)] x [1(1), 1(1),1(2)] x [1(1), 1(3)] = [4(1), 2(2), 4(3), 2(6)] 204102204102 90 Elementary Divisor – (irreducible polynomial) p Primary Cycles (odd) – 1, 3. Secondary Cycles 2 p.k – (2, 4..), (6, 12,..). PFCS, PCS

16. Analysis Synthesis Evolution Associative Memory Pattern Classifier PPS Non Linear CA Additive Cellular Automata - Analysis Similarity between ACA and LCA The cycle structure of an Additive CA differs from its Linear Counterpart only if the characteristic polynomial contains a (x +1) factor. 51153204153 165 1 0 0 0 0 0 1 1 0 0 0 0 1 0 0 0 0 0 1 1 0 0 0 1 0 T = 1 1 0 1 1 F = CS = [2(4), 2(12))]

17. Analysis Synthesis Evolution Associative Memory Pattern Classifier PPS Non Linear CA Additive Cellular Automata - Analysis Compute the cycle structure of LCA. Characteristic Polynomial - (x + 1). (x +1) 2. (x 2 +x + 1) CS = [4(1), 2(2), 4(3), 2(6)] If factor (x+1) p is present Check the nature of F vector. If F vector belongs to Null Space of (x+1) p (here (x +1) 2 ), then merge all the cycles k to 2 p.k (here p = 2) k = 1 4 x 1 + 2 x 2 = 8 = 2(4), k = 3 4 x 3 + 2 x 6 = 24 = 2(12) Null Space (T + I) p. F = 0, (T + I) p-1. F ≠ 0

18. Analysis Synthesis Evolution Associative Memory Pattern Classifier PPS Non Linear CA Additive Cellular Automata - Synthesis CS = [4(1), 2(2), 4(3), 2(6)] 204102204102 90 Steps – Linear Cellular Automata 1.Express the CS as product of 2 PFCS [1(1), 3(1), 2(2)] x [1(1),1(3)] 2.Express PFCS as product of PCS (1,1) 1 x (1,1) 2 x (1,3) 1 3.Construct the elementary divisor of each PCS. (x+1). (x+1) 2. (x 2 +x+1) - characteristic polynomial. 4.Corresponding to each individual elementary divisor construct a submatrix and join the submatrix by placing them in Block Diagonal Form [1 ] 0 0 0 0 0 |1 1| 0 0 0 |0 1| 0 0 0 0 0 |1 1| 0 0 0 |1 0| T = (x+1) (x+1) 2 (x 2 +x+1) [1(1), 1(1), 1(2)]

19. Analysis Synthesis Evolution Associative Memory Pattern Classifier PPS Non Linear CA Additive Cellular Automata - Synthesis Steps – Additive Cellular Automata 1.Synthesis of T Matrix 2.Synthesis of F Vector Synthesis of T Matrix Find the corresponding linear cycle structure from the additive cycle structure. 51153204153 165 CS = [2(4), 2(12))] CS = [4(1), 2(2), 4(3), 2(6)] Synthesize the T Matrix Synthesis of F Vector – Probabilistic approach, Randomly pick a F vector and check whether it falls in the respective Null Space

20. Analysis Synthesis Evolution Associative Memory Pattern Classifier PPS Non Linear CA General Framework for CA evolution. 1. Form Population of (say) 50 CA 98236226107 11100010 Linear Cellular Automata - Evolution 4 cell CA needs 32 bit chromosome 3. Select 10 best solution 1110001000 1000001001 …. 0.8 0.7 … 5. Crossover between solutions and form 35 new solutions 10000110001110001000 1000001000 3240 24 4. Mutate 5 best chromosome 11100010001110011000 3248 2. Arrange the chromosomes with respect to their fitness value

21. Analysis Synthesis Evolution Associative Memory Pattern Classifier PPS Non Linear CA General Framework for CA evolution. 1. Form Population of (say) 50 CA 98236226107 11100010 Linear Cellular Automata - Evolution 4 cell CA needs 32 bit chromosome 1110001000 1100011010 …. 0.8 0.5 …. Population of 50 chromosomes at Generation 0 1110011100 1100000010 …. 0.95 0.75 …. Population of 50 chromosomes at Generation 1

22. Analysis Synthesis Evolution Associative Memory Pattern Classifier PPS Non Linear CA General Framework for CA evolution. 1. Form Population of (say) 50 CA 98236226107 11100010 Linear Cellular Automata - Evolution 4 cell CA needs 32 bit chromosome Problem – Huge search space 4 cell CA – search space = 2 32 100 cell CA – search space = 2 800 !!! For linear CA 100 cell CA – search space = 2 300 !!! Solution – Analytically reduce the search space. Identify a subclass of CA fit for the particular job and evolve it. Subclass – Group CA, Max-length CA, LCA with same characteristic polynomial

23. Analysis Synthesis Evolution Associative Memory Pattern Classifier PPS Non Linear CA Special class of Linear CA Characteristic polynomial x n-m (1+x) m Min. Polynomial x d (1+x) d - depth 01101 10011 01111 10001 1100111011 00111 00101 10010 01100 10000 01110 1100011010 00100 00110 01000 10100 01010 10110 1110011110 00010 00000 01001 10101 01011 10111 1111111101 00001 00011 Multiple Attractor Cellular Automata (MACA) Basin

24. Analysis Synthesis Evolution Associative Memory Pattern Classifier PPS Non Linear CA Select 10 best solution 1110001000 1000001001 …. 0.8 0.7 … Crossover between solutions and form 35 new solutions 10000110001110001000 1000001000 3240 24 Mutate 5 best chromosome 11100010001110011000 3248 Problem in using conventional genetic algorithm to arrive at the correct configuration of MACA Same rules in different sequence doesn’t produce the MACA 9060150906015090 MACANot an MACA MACA - Evolution

25. Analysis Synthesis Evolution Associative Memory Pattern Classifier PPS Non Linear CA  A special methodology of Genetic Algorithm is used  Consideration - After mutation and cross-over, the resultant is also a MACA  Pseudo Chromosome Format is introduced  All members of chromosomes has the characteristic polynomial x n-m (1+x) m  The characteristic polynomial of all MACA is x n-m (1+x) m MACA - Evolution

26. Analysis Synthesis Evolution Associative Memory Pattern Classifier PPS Non Linear CA Char Poly = x 3 (1+x) 2 Distribute the factors - x 2 (1+x) x (1+x) Resultant Matrix T102 x2x2 (1 + x) x 10000 00100 00100 00111 00011 T =102 Pseudo Chromosome Format MACA - Evolution

27. Analysis Synthesis Evolution Associative Memory Pattern Classifier PPS Non Linear CA Each x d is represented by d followed by d -1 zeros Each (1+x) represented by -1102 x2x2 (1 + x) x 10000 00100 00100 00111 00011 T =102 Pseudo Chromosome Format MACA - Evolution

28. Analysis Synthesis Evolution Associative Memory Pattern Classifier PPS Non Linear CA00031202 00210302 00031302 0302 0031 00031202 00031202 MACA - Evolution Crossover Technique MACA - 1 MACA - 2 MACA d followed by d-1 zero

29. Analysis Synthesis Evolution Associative Memory Pattern Classifier PPS Non Linear CA1100300311003003 31100003 3110003 MACA - Evolution Mutation Technique MACA - 1 Mutated MACA d followed by d-1 zero

30. Analysis Synthesis Evolution Associative Memory Pattern Classifier PPS Non Linear CA Multiple Attractor Cellular Automata - Applications Associative Memory Model Pattern Classifier AB C…ZAB C…Z Bookman Old Style A Comic Sans MS Conventional Approach - Compares input patterns with each of the stored patterns learn

31. Analysis Synthesis Evolution Associative Memory Pattern Classifier PPS Non Linear CA The Problem A Comic Sans MS A A B AB C…ZAB C…Z Bookman old Style Grid by Grid Comparison

32. Analysis Synthesis Evolution Associative Memory Pattern Classifier PPS Non Linear CA The Problem A A B Grid by Grid Comparison 0 0 1 0 0 1 1 1 1 0 0 1 0 1 1 0 1 0 0 1 No of Mismatch = 3

33. Analysis Synthesis Evolution Associative Memory Pattern Classifier PPS Non Linear CA The Problem A A B Grid by Grid Comparison 0 0 1 0 0 1 1 1 1 0 0 1 1 1 1 0 0 1 0 1 1 1 0 1 1 1 1 0 No of Mismatch = 9

34. Analysis Synthesis Evolution Associative Memory Pattern Classifier PPS Non Linear CA Associative Memory Time to recognize a pattern - Proportional to the number of stored patterns ( Too costly with the increase of number of patterns stored ) Solution - Associative Memory Modeling Entire state space - Divided into some pivotal points. State close to pivot - Associated with that pivot. Time to recognize pattern - Independent of number of stored patterns.

35. Analysis Synthesis Evolution Associative Memory Pattern Classifier PPS Non Linear CA Time to recognize a pattern - Proportional to the number of stored patterns ( Too costly with the increase of number of patterns stored ) Solution - Associative Memory Modeling Two Phase : Learning and Detection Time to learn is higherDriving a car Difficult to learn but once learnt it becomes natural Densely connected Network - Problems to implement in Hardware Solution - Cellular Automata (Sparsely connected machine) - Ideally suitable for VLSI application Associative Memory

36. Analysis Synthesis Evolution Associative Memory Pattern Classifier PPS Non Linear CA MACA – Can be made to act as an Associative Memory A B C D Hamming Hash Family - Patterns close to each other is more likely to fall in the same basin What follows – (for example) Different variations of A falls in same attractor basin MACA as Associative Memory

37. Analysis Synthesis Evolution Associative Memory Pattern Classifier PPS Non Linear CA Performance – Memorizing Capacity Given a set of patterns to be learned – P1, P2, ….Pk, Evolve an MACA which can classify the patterns in different attractor basin Pattern Size (n) Hopfield Network 10 20 50 90 100 9 13 25 34 36 2 3 8 14 15 Capacity – Theoretical Capacity – Experimental 8 13 24 33 37

38. Analysis Synthesis Evolution Associative Memory Pattern Classifier PPS Non Linear CA Performance – Recognition Capacity Recognition Capacity - The machine can identify 90% of all the patterns which are within one hamming distance from pivot point. The recognition capacity can be made perfect by using multiple MACA each classifying the same set of patterns.

39. Analysis Synthesis Evolution Associative Memory Pattern Classifier PPS Non Linear CA Classifying Several Related Patterns into one class Vehicle Another Vehicle !! Pattern Classification

40. Analysis Synthesis Evolution Associative Memory Pattern Classifier PPS Non Linear CA Human Brain Pattern Classification

41. Analysis Synthesis Evolution Associative Memory Pattern Classifier PPS Non Linear CA MACA - A NATURAL CLASSIFIER. 11 10 01 00 Class I Class II MACA Based Classification Strategy for Two Class Classifier Pattern Classification

42. Analysis Synthesis Evolution Associative Memory Pattern Classifier PPS Non Linear CA MACA - A NATURAL CLASSIFIER. MACA Based Classification Strategy for Two Class Classifier Forms Natural Cluster Closeness is measured in terms of hamming distance Pattern Classification

43. Analysis Synthesis Evolution Associative Memory Pattern Classifier PPS Non Linear CA MACA - A NATURAL CLASSIFIER. MACA Based Classification Strategy for Two Class Classifier Pattern Classification

44. Analysis Synthesis Evolution Associative Memory Pattern Classifier PPS Non Linear CA Distribution of patterns in class 1 and class2 a a’ bb’ c c’ Experimental Results

45. Analysis Synthesis Evolution Associative Memory Pattern Classifier PPS Non Linear CA a a’ Experimental Results

46. Analysis Synthesis Evolution Associative Memory Pattern Classifier PPS Non Linear CA bb’ Experimental Results

47. Analysis Synthesis Evolution Associative Memory Pattern Classifier PPS Non Linear CA c c’ Experimental Results

48. Analysis Synthesis Evolution Associative Memory Pattern Classifier PPS Non Linear CA d d’ Class 1 Class 2 Experimental Results Clusters Detection by two class classifier

49. Analysis Synthesis Evolution Associative Memory Pattern Classifier PPS Non Linear CA Prohibited Pattern Set Prohibited Pattern Set (PPS) – A set of patterns input of which sents the system into an unstable state. Example : Toggle State of a flip flop Design a TPG with the following features  It avoids the generation of such PPS  It maintains the randomness and fault coverage of a Pseudo Random Pattern Generator  Side by side it doesn’t add to any hardware cost

50. Analysis Synthesis Evolution Associative Memory Pattern Classifier PPS Non Linear CA Problem Definitions Non Max Length GF(2) Cellular Automata is employed to obtain the design criteria Design the CA in such a way so that it has large cycles free from PPS PPS can be of two types  Prohibited Random Patterns – Small number of patterns  Prohibited Functions – some combination of Primary Input can be detrimental

51. Analysis Synthesis Evolution Associative Memory Pattern Classifier PPS Non Linear CA Overview of Design Target Cycle(TC ) Redundant Cycle(RC ) Dmax Given PPS 000011000000100001001 000011100011110010100 110110110110010100100 0010001 Evolve a Non Maxlength CA Criterion for choosing Non-Max Length CA Large cycle of length close to a Max length Cycle Most members of PPS fall in smaller cycles Same Evolution Framework as before, population is built on group CA only

52. Analysis Synthesis Evolution Associative Memory Pattern Classifier PPS Non Linear CA Experimental Observation-I Real data of PPS is not available PPS randomly generated, no. of prohibited patterns assumed 10, 15 For a particular n, 10 different PPS are considered PPS = 10 PPS = 15 TC FreeSpace 8 14 17 19 22 217 59.7622544.14 15841 55.021584141.00 131071 57.788267734.80 458745 57.7045874545.70 4063201 65.62313805142.65 #cell

53. Analysis Synthesis Evolution Associative Memory Pattern Classifier PPS Non Linear CA Experimental Observations -II Study of randomness property Platform used is DiehardC Compared with corresponding maximal length CA Random Testn=24 Max TPG n=32 Max TPG n=48 Max TPG Overlap Sum pass passpass 3D Spherepass fail B’day Spacingfail Overlap 5- permut fail pass DNAfail pass fail Squeezefail passfail pass fail

54. Analysis Synthesis Evolution Associative Memory Pattern Classifier PPS Non Linear CA Experimental Observations -III Fault coverage of the proposed design (Compared with MaxLength CA) Fault Simulator used : Cadence `verifault’

55. Analysis Synthesis Evolution Associative Memory Pattern Classifier PPS Non Linear CA Generalized Multiple Attractor CA The State Space of GMACA – Models an Associative Memory Associative Memory and Non-Linear CA

56. Analysis Synthesis Evolution Associative Memory Pattern Classifier PPS Non Linear CA Generalized Multiple Attractor CA Pivot Points Dist =1Dist =3  The state transition diagram breaks into disjoint attractor basin  Each attractor basin of CA should contain one and only one pattern to be learnt in its attractor cycle  The hamming distance of each state with its attractor is lesser than that of other attractors.

57. Analysis Synthesis Evolution Associative Memory Pattern Classifier PPS Non Linear CA GMACA Evolution Fitness Function PjPj L max=4 If P j does not belongs to any attractor cycle after Maximum Iteration L max Fitness Function (F) = 0

58. Analysis Synthesis Evolution Associative Memory Pattern Classifier PPS Non Linear CA Fitness Function If P j does not belongs to any attractor cycle after Maximum Iteration L max Fitness Function (F) = 0 PjPj else Fitness Function: F = [1 - HD(P i - P j )/N] Desired Pivot Point GMACA Evolution

59. Analysis Synthesis Evolution Associative Memory Pattern Classifier PPS Non Linear CA Fitness Function Average fitness of 30 randomly chosen state GMACA Evolution

60. Analysis Synthesis Evolution Associative Memory Pattern Classifier PPS Non Linear CA Performance Observation : GMACA have much higher capacity than Hopfield Net Pattern Size (n) Hopfield Network 10 15 25 35 45 8 10 15 19 23 2245722457 Capacity – MACA Capacity – GMACA 4 4 6 8 10

61. Analysis Synthesis Evolution Associative Memory Pattern Classifier PPS Non Linear CA Comments  Memorizing Capacity of GMACA - Higher than Hopfield Net but less than MACA  Genetic Algorithm and Reverse Engineering Techniques is employed innovatively  Recognition Capacity higher than MACA  Rules lie in the edge of chaos

62. Analysis Synthesis Evolution Associative Memory Pattern Classifier PPS Non Linear CA Major Contributions  Analysis  Synthesis  Evolution  Pattern Recognition

63. Thank you