1. Polynomial Discrete Time Cellular Neural Networks Eduardo Gomez-Ramirez † Giovanni Egidio Pazienza‡ † LIDETEA, POSGRADO E INVESTIGACION Universidad La Salle – México, D.F. ‡ Department d’Electronica, EALS Universitat “Ramon Llull” – Barcelona, Spain

2. Outline Cellular Neural Networks (CNN) Introduction and Objective Genetic Algorithms (GA) Polynomial Discrete Time CNNs (PDTCNNs) XOR Problem Game of Life Learning vs Design Conclusions and future work IntroCNN & GAPolyn. CNNXORGoLConclusions

3. CNN: Introduction CNN for complex task (linearly nonseparable data)  Multilayer CNNs  Include more degrees of freedom for the output state of each layer  Search in a finite set of templates  Single layer: Polynomial CNNs IntroCNN & GAPolyn. CNNXORGoLConclusions

4. Improve the representation power of a single layer CNN including a simple nonlinear term to solve problems with linearly nonseparable data (XOR) IntroCNN & GAPolyn. CNNXORGoLConclusions Objective

5. CNN: mathematical model IntroCNN & GAPolyn. CNNXORGoLConclusions The simplified mathematical model is: where x c is the state of the cell, u c the input and y c the output

6. CNN: Activation Function IntroCNN & GAPolyn. CNNXORGoLConclusions

7. CNN: Block Diagram IntroCNN & GAPolyn. CNNXORGoLConclusions

8. CNN: Discrete Model IntroCNN & GAPolyn. CNNXORGoLConclusions Computing x(∞), the model can be represented as using the following activation function

9.  Steps:  Crossover C(Fg)  Mutation M(*)  Adding random parent Ag(  ) GA: main steps proposed IntroCNN & GAPolyn. CNNXORGoLConclusions

10. GA: Crossover IntroCNN & GAPolyn. CNNXORGoLConclusions

11. I=0 Individual 1 Individual 2 GA: Crossover IntroCNN & GAPolyn. CNNXORGoLConclusions a1a1 b1b1 c1c1 a2a2 b2b2 c2c2

12. GA: Mutation IntroCNN & GAPolyn. CNNXORGoLConclusions where r  U(0,1) is a random variable with uniform distribution defined on a probability space ( , ,P),   

13. GA: Mutation (resolution) IntroCNN & GAPolyn. CNNXORGoLConclusions I=0 Individual 1 Individual 2 a1a1 b1b1 c1c1 a2a2 b2b2 c2c2

14. GA: Selecting Parents IntroCNN & GAPolyn. CNNXORGoLConclusions

15. GA: Adding Random Parent IntroCNN & GAPolyn. CNNXORGoLConclusions

16. THEOREM 1: (Weierstrass’s Approximation Theorem) Let g be a continuous real valued function defined on a closed interval [a,b]. Then, given any  positive, there exists a polynomial y (which may depend on  ) with real coefficients such that: For every x  [a,b]. Polynomial Discrete Time Cellular Neural Network IntroCNN & GAPolyn. CNNXORGoLConclusions

17. THEOREM 2 *: Any Boolean Function of n-variables can be realized using a Polynomial Threshold gates of order s  n. The quadratic threshold gate can be defined: And s is the number of inputs and T is the threshold constant. Polynomial Discrete Time Cellular Neural Network IntroCNN & GAPolyn. CNNXORGoLConclusions * N. J. Nilsson. The Mathematical Foundations of Learning Machines. McGraw Hill, New York, 1990.

18. PDTCNN: the model (I) IntroCNN & GAPolyn. CNNXORGoLConclusions

19. PDTCNN: the model (II) IntroCNN & GAPolyn. CNNXORGoLConclusions

20. PDTCNN: Solving XOR problem Some papers:  Z. Yang, Y. Nishio, A. Ushida, Templates and algorithms for two-layer cellular neural networks. IJCNN’02, 2002.  F. Chen, G. He, G. Chen & X. Xu, Implementation of Arbitrary Boolean Functions via CNN. CNNA’06, 2006. IntroCNN & GAPolyn. CNNXORGoLConclusions

21. PDTCNN:Solving XOR problem  M. Balsi, Generalized CNN: Potentials of a CNN with Non-Uniform Weights. CNNA-92, 2002.  E. Bilgili, I. C. Göknar and O. N. Ucan, Cellular neural network with trapezoidal activation function. Int. J. Circ. Theor. Appl., 2005 IntroCNN & GAPolyn. CNNXORGoLConclusions

22. Learning parameters Initialpop=20000 Number of fathers=7 Maximum number of random parents to be add = 3 Kpro=0.8 Increment=1 Mutation Probability=0.15 IntroCNN & GAPolyn. CNNXORGoLConclusions

23. PDTCNN:First Scheme U:u ij =x ij  x ij+1 IntroCNN & GAPolyn. CNNXORGoLConclusions

24. PDTCNN:Second Scheme U:u ij =x ij  y ij b) c) IntroCNN & GAPolyn. CNNXORGoLConclusions

25. The Game of Life (I) The Game of Life (GoL) is a totalistic cellular automaton consisting in a two-dimensional grid cells, that may be either alive (black) or dead (white). IntroCNN & GAPolyn. CNNXORGoLConclusions

26. The Game of Life (II) The state of each cell varies according to the following rules:  Birth: a cell that is dead at time t becomes alive at time t + 1 only if exactly 3 of its neighbors were alive at time t;  Survival: a cell that was living at time t will remain alive at t + 1 if and only if it had exactly 2 or 3 alive neighbors at time t. IntroCNN & GAPolyn. CNNXORGoLConclusions

27. The Game of Life (III)  Every sufficient well-stated mathematical problem can be reduced to a question about Life;  It is possible to make a life computer (logic gates, storage etc.);  Life is universal: it can be programmed to perform any desired calculation;  Given a large enough Life space and enough time, self-reproducing animals will emerge...  The whole universe is a CA! (E.Fredkin, MIT). IntroCNN & GAPolyn. CNNXORGoLConclusions

28. The Game of Life – NOT gate IntroCNN & GAPolyn. CNNXORGoLConclusions A

29. CNN & GoL Multilayer CNN (Chua, Roska) – 1990 Activation function (Chua, Roska) – 1990 CNN-UM (Roska,Chua) -1990 CNN Universal Cells (Dogaru, Chua) – 1999 Simplicity vs. Computational power IntroCNN & GAPolyn. CNNXORGoLConclusions

30. Polynomial CNN (I) What’s g(u d,y d )? IntroCNN & GAPolyn. CNNXORGoLConclusions

31. Polynomial CNN (II) In the simplest case g(u d, y d ) is a second degree polynomial, whose general form is IntroCNN & GAPolyn. CNNXORGoLConclusions

32. Polynomial CNN (III) Thanks to some considerations we find that IntroCNN & GAPolyn. CNNXORGoLConclusions

33. Polynomial CNN (IV) u c and appear in the state equation direct link with totalistic Cellular Automata IntroCNN & GAPolyn. CNNXORGoLConclusions

34. GoL: Rules (I)

35. GoL: Rules (II)  Rule 1: a cell will be alive at least 3 of the 9 cells in its 3 × 3 neigh. are alive Black pixel= +1 White pixel= -1 pixel centr. = 1 (black) Σ neigh. = -2 (5 w, 2 b) next state = -1 (white)

36. GoL: Rules (III)  Rule 2: a cell will be alive if at most 3 of its 8 neighbors are alive Black pixel= +1 White pixel= -1 pixel centr. = 1 (black) Σ neigh. = -2 (5 w, 2 b) next state = 1 (black)

37. Design algorithm (I) First iteration: we try to perform the first rule (a cell will be alive at least 3 of the 9 cells in its 3 × 3 neigh. are alive) If Y(0)=0 b c =1 b p =1 i=3 IntroCNN & GAPolyn. CNNXORGoLConclusions

38. Design algorithm (II) Second iteration: we try to accomplish the second rule (a cell will be alive if at most 3 of its 8 neighbors are alive) IntroCNN & GAPolyn. CNNXORGoLConclusions

39. Design algorithm (III) Hyp: p c =0

40. Templates found using learning Coming soon... IntroCNN & GAPolyn. CNNXORGoLConclusions

41. Conclusions (I) In general: In some cases it is possible to reduce a multilayer DTCNN to a single layer PDTCNN Thanks to the GoL we can explore the capacity of PDTCNNs for Universal Machine IntroCNN & GAPolyn. CNNXORGoLConclusions

42. Conclusions (II) About learning:  The resolution used reduces the search space  The step “Add random parent” improves the behavior to avoid local minimas About design We give a simple algorithm to design templates for the Polynomial CNN

43. Future Work Implementations of mathematical morphology functions with PDTCNNs IntroCNN & GAPolyn. CNNXORGoLConclusions

44. Polynomial Discrete Time Cellular Neural Networks Eduardo Gomez-Ramirez Giovanni Egidio Pazienza egr@ci.ulsa.mx gpazienza@salle.url.edu