Polynomial Discrete Time Cellular Neural Networks Eduardo Gomez-Ramirez † Giovanni Egidio Pazienza‡ † LIDETEA, POSGRADO E INVESTIGACION Universidad La Salle – México, D.F. ‡ Department d’Electronica, EALS Universitat “Ramon Llull” – Barcelona, Spain

Outline Cellular Neural Networks (CNN) Introduction and Objective Genetic Algorithms (GA) Polynomial Discrete Time CNNs (PDTCNNs) XOR Problem Game of Life Learning vs Design Conclusions and future work IntroCNN & GAPolyn. CNNXORGoLConclusions

CNN: Introduction CNN for complex task (linearly nonseparable data)  Multilayer CNNs  Include more degrees of freedom for the output state of each layer  Search in a finite set of templates  Single layer: Polynomial CNNs IntroCNN & GAPolyn. CNNXORGoLConclusions

Improve the representation power of a single layer CNN including a simple nonlinear term to solve problems with linearly nonseparable data (XOR) IntroCNN & GAPolyn. CNNXORGoLConclusions Objective

CNN: mathematical model IntroCNN & GAPolyn. CNNXORGoLConclusions The simplified mathematical model is: where x c is the state of the cell, u c the input and y c the output

CNN: Activation Function IntroCNN & GAPolyn. CNNXORGoLConclusions

CNN: Block Diagram IntroCNN & GAPolyn. CNNXORGoLConclusions

CNN: Discrete Model IntroCNN & GAPolyn. CNNXORGoLConclusions Computing x(∞), the model can be represented as using the following activation function

 Steps:  Crossover C(Fg)  Mutation M(*)  Adding random parent Ag(  ) GA: main steps proposed IntroCNN & GAPolyn. CNNXORGoLConclusions

GA: Crossover IntroCNN & GAPolyn. CNNXORGoLConclusions

I=0 Individual 1 Individual 2 GA: Crossover IntroCNN & GAPolyn. CNNXORGoLConclusions a1a1 b1b1 c1c1 a2a2 b2b2 c2c2

GA: Mutation IntroCNN & GAPolyn. CNNXORGoLConclusions where r  U(0,1) is a random variable with uniform distribution defined on a probability space ( , ,P),   

GA: Mutation (resolution) IntroCNN & GAPolyn. CNNXORGoLConclusions I=0 Individual 1 Individual 2 a1a1 b1b1 c1c1 a2a2 b2b2 c2c2

GA: Selecting Parents IntroCNN & GAPolyn. CNNXORGoLConclusions

GA: Adding Random Parent IntroCNN & GAPolyn. CNNXORGoLConclusions

THEOREM 1: (Weierstrass’s Approximation Theorem) Let g be a continuous real valued function defined on a closed interval [a,b]. Then, given any  positive, there exists a polynomial y (which may depend on  ) with real coefficients such that: For every x  [a,b]. Polynomial Discrete Time Cellular Neural Network IntroCNN & GAPolyn. CNNXORGoLConclusions

THEOREM 2 *: Any Boolean Function of n-variables can be realized using a Polynomial Threshold gates of order s  n. The quadratic threshold gate can be defined: And s is the number of inputs and T is the threshold constant. Polynomial Discrete Time Cellular Neural Network IntroCNN & GAPolyn. CNNXORGoLConclusions * N. J. Nilsson. The Mathematical Foundations of Learning Machines. McGraw Hill, New York, 1990.

PDTCNN: the model (I) IntroCNN & GAPolyn. CNNXORGoLConclusions

PDTCNN: the model (II) IntroCNN & GAPolyn. CNNXORGoLConclusions

PDTCNN: Solving XOR problem Some papers:  Z. Yang, Y. Nishio, A. Ushida, Templates and algorithms for two-layer cellular neural networks. IJCNN’02,  F. Chen, G. He, G. Chen & X. Xu, Implementation of Arbitrary Boolean Functions via CNN. CNNA’06, IntroCNN & GAPolyn. CNNXORGoLConclusions

PDTCNN:Solving XOR problem  M. Balsi, Generalized CNN: Potentials of a CNN with Non-Uniform Weights. CNNA-92,  E. Bilgili, I. C. Göknar and O. N. Ucan, Cellular neural network with trapezoidal activation function. Int. J. Circ. Theor. Appl., 2005 IntroCNN & GAPolyn. CNNXORGoLConclusions

Learning parameters Initialpop=20000 Number of fathers=7 Maximum number of random parents to be add = 3 Kpro=0.8 Increment=1 Mutation Probability=0.15 IntroCNN & GAPolyn. CNNXORGoLConclusions

PDTCNN:First Scheme U:u ij =x ij  x ij+1 IntroCNN & GAPolyn. CNNXORGoLConclusions

PDTCNN:Second Scheme U:u ij =x ij  y ij b) c) IntroCNN & GAPolyn. CNNXORGoLConclusions

The Game of Life (I) The Game of Life (GoL) is a totalistic cellular automaton consisting in a two-dimensional grid cells, that may be either alive (black) or dead (white). IntroCNN & GAPolyn. CNNXORGoLConclusions

The Game of Life (II) The state of each cell varies according to the following rules:  Birth: a cell that is dead at time t becomes alive at time t + 1 only if exactly 3 of its neighbors were alive at time t;  Survival: a cell that was living at time t will remain alive at t + 1 if and only if it had exactly 2 or 3 alive neighbors at time t. IntroCNN & GAPolyn. CNNXORGoLConclusions

The Game of Life (III)  Every sufficient well-stated mathematical problem can be reduced to a question about Life;  It is possible to make a life computer (logic gates, storage etc.);  Life is universal: it can be programmed to perform any desired calculation;  Given a large enough Life space and enough time, self-reproducing animals will emerge...  The whole universe is a CA! (E.Fredkin, MIT). IntroCNN & GAPolyn. CNNXORGoLConclusions

The Game of Life – NOT gate IntroCNN & GAPolyn. CNNXORGoLConclusions A

CNN & GoL Multilayer CNN (Chua, Roska) – 1990 Activation function (Chua, Roska) – 1990 CNN-UM (Roska,Chua) CNN Universal Cells (Dogaru, Chua) – 1999 Simplicity vs. Computational power IntroCNN & GAPolyn. CNNXORGoLConclusions

Polynomial CNN (I) What’s g(u d,y d )? IntroCNN & GAPolyn. CNNXORGoLConclusions

Polynomial CNN (II) In the simplest case g(u d, y d ) is a second degree polynomial, whose general form is IntroCNN & GAPolyn. CNNXORGoLConclusions

Polynomial CNN (III) Thanks to some considerations we find that IntroCNN & GAPolyn. CNNXORGoLConclusions

Polynomial CNN (IV) u c and appear in the state equation direct link with totalistic Cellular Automata IntroCNN & GAPolyn. CNNXORGoLConclusions

GoL: Rules (I)

GoL: Rules (II)  Rule 1: a cell will be alive at least 3 of the 9 cells in its 3 × 3 neigh. are alive Black pixel= +1 White pixel= -1 pixel centr. = 1 (black) Σ neigh. = -2 (5 w, 2 b) next state = -1 (white)

GoL: Rules (III)  Rule 2: a cell will be alive if at most 3 of its 8 neighbors are alive Black pixel= +1 White pixel= -1 pixel centr. = 1 (black) Σ neigh. = -2 (5 w, 2 b) next state = 1 (black)

Design algorithm (I) First iteration: we try to perform the first rule (a cell will be alive at least 3 of the 9 cells in its 3 × 3 neigh. are alive) If Y(0)=0 b c =1 b p =1 i=3 IntroCNN & GAPolyn. CNNXORGoLConclusions

Design algorithm (II) Second iteration: we try to accomplish the second rule (a cell will be alive if at most 3 of its 8 neighbors are alive) IntroCNN & GAPolyn. CNNXORGoLConclusions

Design algorithm (III) Hyp: p c =0

Templates found using learning Coming soon... IntroCNN & GAPolyn. CNNXORGoLConclusions

Conclusions (I) In general: In some cases it is possible to reduce a multilayer DTCNN to a single layer PDTCNN Thanks to the GoL we can explore the capacity of PDTCNNs for Universal Machine IntroCNN & GAPolyn. CNNXORGoLConclusions

Conclusions (II) About learning:  The resolution used reduces the search space  The step “Add random parent” improves the behavior to avoid local minimas About design We give a simple algorithm to design templates for the Polynomial CNN

Future Work Implementations of mathematical morphology functions with PDTCNNs IntroCNN & GAPolyn. CNNXORGoLConclusions

Polynomial Discrete Time Cellular Neural Networks Eduardo Gomez-Ramirez Giovanni Egidio Pazienza