Presentation is loading. Please wait.

Presentation is loading. Please wait.

Growing Curvilinear Component Analysis (GCCA)

Published byHerta Böhler Modified over 5 years ago

Similar presentations

Presentation on theme: "Growing Curvilinear Component Analysis (GCCA)"— Presentation transcript:

1 Growing Curvilinear Component Analysis (GCCA)
G. Cirrincione University of Picardie Jules Verne Lab. LTI, Amiens, France University of Nottingham Dept. Electrical Eng., Nottingham, UK V. Randazzo Politecnico di Torino DET, Torino, Italy

2 Introduction CCA GCCA algorithm Analysis of bridges Simulations Application Conclusions

3 continuous learning online data stream
introduction CCA GCCA algorithm analysis of bridges simulations application conclusions

4 connected graph online ISOMAP (2004) GNLG-NG (2006) TRNMap (2008)
RBF-NDR (2011) linear PCA (Oja, 1992) neural PCA : GHA (1989) APEX (1996) nonlinear VQ + P fusion VQP (1993) OVI-NG (2006) incremental SOM : GNG-U (1997) DASH (2005) DSOM (2010) autoencoders (2006) stationary input data stream nonstationary input linear nonlinear PCA (Oja, 1992) DSOM (2010) real time PR fault diagnosis novelty detection intrusion detection speech recognition text recognition computer vision scene analysis face recognition forgetting network introduction CCA GCCA algorithm analysis of bridges simulations application conclusions

5 What is the goal ?

6 What is the goal ?

7 What is the goal ? memoryless SOM with low constant learningrate
DSOM with high constant learning rate SOM with high constant learning rate memoryless

8 What is the goal ? bridge memory
introduction CCA GCCA algorithm analysis of bridges simulations application conclusions

9 What is the goal ? bridge memory t = t nonstationary (jump) x bridge
nonstationary (jump) x bridge neuron local threshold edge Competitive Hebbian Learning T first winner edge n second winner t < t introduction CCA GCCA algorithm analysis of bridges simulations application conclusions

10 Curvilinear Component Analysis
projection local vector quantization dynamic introduction CCA GCCA algorithm analysis of bridges simulations application conclusions

11 Network initialization with two neurons (initial seed)
start Network initialization with two neurons (initial seed) data space seed latent space GCCA algorithm introduction CCA GCCA algorithm analysis of bridges simulations application conclusions

12 x t = t neuron threshold T nonstationary (jump) w edge n t < t
start Create a new neuron in the X space whose weight vector is x0 Network initialization with two neurons (initial seed) New data x0 Compute distances from existing neurons (candidate neurons) Sort distances from min to max, say d1,..,dn Tw1 > d1? t = t w n T W edge t < t nonstationary (jump) no x neuron threshold 1 introduction CCA GCCA algorithm analysis of bridges simulations application conclusions

13 x bridge t = t T w edge n t < t
start Create a new neuron in the X space whose weight vector is x0 Network initialization with two neurons (initial seed) Link x0 to w1 with a bridge New data x0 Compute distances from existing neurons (candidate neurons) t = t Sort distances from min to max, say d1,..,dn no Tw1 > d1? x bridge w n T W edge t < t 1 introduction CCA GCCA algorithm analysis of bridges simulations application conclusions

14 x bridge t = t T w edge n t < t
start Create a new neuron in the X space whose weight vector is x0 Network initialization with two neurons (initial seed) Link x0 to w1 with a bridge New data x0 ComputeTx0 =x0-w1  Compute distances from existing neurons (candidate neurons) t = t Sort distances from min to max, say d1,..,dn no Tw1 > d1? x bridge w n T W edge t < t 1 introduction CCA GCCA algorithm analysis of bridges simulations application conclusions

15 projection with extrapolation (latent space)
start Create a new neuron in the X space whose weight vector is x0 Network initialization with two neurons (initial seed) Link x0 to w1 with a bridge New data x0 ComputeTx0 =x0-w1  Compute distances from existing neurons (candidate neurons) Triangulate initial y0 t = t Sort distances from min to max, say d1,..,dn no Tw1 > d1? x bridge w n T W edge t < t 1 introduction CCA GCCA algorithm analysis of bridges simulations application conclusions

16 projection with extrapolation (latent space)
Create a new neuron in the X space whose weight vector is x0 Link x0 to w1 with a bridge ComputeTx0 =x0-w1  Triangulate initial y0 farthest from the third winner y d 1 d 2 t >= t w 1 w 2 w 3 t < t introduction CCA GCCA algorithm analysis of bridges simulations application conclusions

17 projection with extrapolation (latent space)
Create a new neuron in the X space whose weight vector is x0 Link x0 to w1 with a bridge ComputeTx0 =x0-w1  Triangulate initial y0 Fix the first and the second winner and adjust y0 according to CCA gradient flow for one iteration extrapolation y d 1 d 2 t >= t FIXED w 1 w 2 w 3 FIXED t < t introduction CCA GCCA algorithm analysis of bridges simulations application conclusions

18 x bridge t = t T w edge n t < t
start Create a new neuron in the X space whose weight vector is x0 Network initialization with two neurons (initial seed) Link x0 to w1 with a bridge New data x0 ComputeTx0 =x0-w1  Compute distances from existing neurons (candidate neurons) Triangulate initial y0 t = t Sort distances from min to max, say d1,..,dn Fix the first and the second winner and adjust y0 according to CCA gradient flow for one iteration no Tw1 > d1? x bridge w n T W edge t < t 1 introduction CCA GCCA algorithm analysis of bridges simulations application conclusions

19 Competitive Hebbian Learning (input space)
start CHL: first and second winner linking (if not already linked) Network initialization with two neurons (initial seed) New data x0 Compute distances from existing neurons (candidate neurons) Sort distances from min to max, say d1,..,dn Tw1 > d1? yes Is the connection w1 w2 a bridge? no Competitive Hebbian Learning (input space) introduction CCA GCCA algorithm analysis of bridges simulations application conclusions

20 Competitive Hebbian Learning (input space)
start CHL: first and second winner linking (if not already linked) Network initialization with two neurons (initial seed) second winner New data x0 Compute distances from existing neurons (candidate neurons) x winner Sort distances from min to max, say d1,..,dn Tw1 > d1? yes Is the connection w1 w2 a bridge? Competitive Hebbian Learning (input space) introduction CCA GCCA algorithm analysis of bridges simulations application conclusions

21 Competitive Hebbian Learning (input space)
start CHL: first and second winner linking (if not already linked) Network initialization with two neurons (initial seed) Increment first winner neighbor edge ages by one (neuron pruning) second winner New data x0 Set the edge age between first and second winner to 0 Compute distances from existing neurons (candidate neurons) x winner +1 Sort distances from min to max, say d1,..,dn Tw1 > d1? yes Is the connection w1 w2 a bridge? Competitive Hebbian Learning (input space) introduction CCA GCCA algorithm analysis of bridges simulations application conclusions

22 Soft Competitive Learning (input space)
start CHL: first and second winner linking (if not already linked) Network initialization with two neurons (initial seed) Increment first winner neighbor edge ages by one (neuron pruning) New data x0 Set the edge age between first and second winner to 0 Compute distances from existing neurons (candidate neurons) x winner SCL on w1 and its linked neighbors Sort distances from min to max, say d1,..,dn Tw1 > d1? yes Is the connection w1 w2 a bridge? no Soft Competitive Learning (input space) introduction CCA GCCA algorithm analysis of bridges simulations application conclusions

23 Soft Competitive Learning (input space)
start CHL: first and second winner linking (if not already linked) Network initialization with two neurons (initial seed) Increment first winner neighbor edge ages by one (neuron pruning) New data x0 Set the edge age between first and second winner to 0 Compute distances from existing neurons (candidate neurons) x SCL on w1 and its linked neighbors Sort distances from min to max, say d1,..,dn Tw1 > d1? yes Is the connection w1 w2 a bridge? no Soft Competitive Learning (input space) introduction CCA GCCA algorithm analysis of bridges simulations application conclusions

24 neuron thresholding (input space)
start CHL: first and second winner linking (if not already linked) Network initialization with two neurons (initial seed) Tw2 Increment first winner neighbor edge ages by one (neuron pruning) New data x0 Set the edge age between first and second winner to 0 Compute distances from existing neurons (candidate neurons) Tw1 SCL on w1 and its linked neighbors Sort distances from min to max, say d1,..,dn Compute Tw1 and Tw2 Tw1 > d1? yes Is the connection w1 w2 a bridge? no neuron thresholding (input space) introduction CCA GCCA algorithm analysis of bridges simulations application conclusions

25 projection with interpolation (latent space)
start CHL: first and second winner linking (if not already linked) Network initialization with two neurons (initial seed) Increment first winner neighbor edge ages by one (neuron pruning) New data x0 Set the edge age between first and second winner to 0 Compute distances from existing neurons (candidate neurons) SCL on w1 and its linked neighbors Sort distances from min to max, say d1,..,dn Compute Tw1 and Tw2 Detect neurons within the sphere in the Y space centered in the first winner and with radius λ (λ-neurons) Tw1 > d1? yes Is the connection w1 w2 a bridge? λ no projection with interpolation (latent space) introduction CCA GCCA algorithm analysis of bridges simulations application conclusions

26 projection with interpolation (latent space)
start CHL: first and second winner linking (if not already linked) Network initialization with two neurons (initial seed) Increment first winner neighbor edge ages by one (neuron pruning) New data x0 Set the edge age between first and second winner to 0 Compute distances from existing neurons (candidate neurons) SCL on w1 and its linked neighbors Sort distances from min to max, say d1,..,dn Compute Tw1 and Tw2 Detect neurons within the sphere in the Y space centered in the first winner and with radius λ (λ-neurons) Tw1 > d1? yes Is the connection w1 w2 a bridge? no Fix w1 projection and adjust the λ-neurons according to CCA gradient flow for one iteration FIXED λ projection with interpolation (latent space) introduction CCA GCCA algorithm analysis of bridges simulations application conclusions

27 start CHL: first and second winner linking (if not already linked) Network initialization with two neurons (initial seed) Increment first winner neighbor edge ages by one (neuron pruning) New data x0 Set the edge age between first and second winner to 0 Compute distances from existing neurons (candidate neurons) SCL on w1 and its linked neighbors Sort distances from min to max, say d1,..,dn Compute Tw1 and Tw2 Detect neurons within the sphere in the Y space centered in the first winner and with radius λ (λ-neurons) Tw1 > d1? yes Is the connection w1 w2 a bridge? no Fix w1 projection and adjust the λ-neurons according to CCA gradient flow for one iteration FIXED λ yes Is w1 the bridge tail? yes Turn the bridge into a link introduction CCA GCCA algorithm analysis of bridges simulations application conclusions

28 start CHL: first and second winner linking (if not already linked) Network initialization with two neurons (initial seed) Increment first winner neighbor edge ages by one (neuron pruning) New data x0 Compute distances from existing neurons (candidate neurons) Set the edge age between first and second winner to 0 SCL on w1 and its linked neighbors Sort distances from min to max, say d1,..,dn Compute Tw1 and Tw2 no Tw1 > d1? Detect neurons within the sphere in the Y space centered in the first winner and with radius λ (λ-neurons) yes Is the connection w1 w2 a bridge? no w1 doubling (w1new ) using HCL Fix w1 projection and adjust the λ-neurons according to CCA gradient flow for one iteration yes Is w1 the bridge tail? no yes Turn the bridge into a link introduction CCA GCCA algorithm analysis of bridges simulations application conclusions

29 neuron doubling seed T edge bridge x t >= t w edge n t < t
1new neuron doubling seed T W1 W 1 edge x 1 bridge t >= t w1 doubling (w1new ) using HCL Link w1 and w1new Compute Tw1= Tw1new = w1-w1new  w edge n t < t introduction CCA GCCA algorithm analysis of bridges simulations application conclusions

30 projection with extrapolation (latent space)
1 W 1new edge x 1 bridge t >= t w1 doubling (w1new ) using HCL Link w1 and w1new Compute Tw1= Tw1new = w1-w1new  w Triangulate yw1new edge n t < t introduction CCA GCCA algorithm analysis of bridges simulations application conclusions

31 projection with extrapolation (latent space)
farthest from the third winner FIXED W 1 W 1new edge bridge t >= t w1 doubling (w1new ) using HCL Link w1 and w1new Compute Tw1= Tw1new = w1-w1new  FIXED w Triangulate yw1new edge Fix yw1 and yw2 adjust yw1new according to CCA gradient flow for one iteration n t < t introduction CCA GCCA algorithm analysis of bridges simulations application conclusions

32 Compute distances from existing neurons (candidate neurons)
Sort distances from min to max, say d1,..,dn start Tw1 > d1? yes no Is the connection w1 w2 a bridge? Turn the bridge into a link CHL: first and second winner linking (if not already linked) Increment first winner neighbor edge ages by one (neuron pruning) Set the edge age between first and second winner to 0 SCL on w1 and its linked neighbors Compute Tw1 and Tw2 Detect neurons within the sphere in the Y space centered in the first winner and with radius λ (λ-neurons) Fix w1 projection and adjust the λ-neurons according to CCA gradient flow for one iteration Is w1 the bridge tail? Network initialization with two neurons (initial seed) New data x0 start Tw1 > d1? yes no Create a new neuron in the X space whose weight vector is x0 Link x0 to w1 with a bridge ComputeTx0 =x0-w1  Triangulate initial y0 Fix the first and the second winner and adjust y0 according to CCA gradient flow for one iteration New data x0 w1 doubling (w1new ) using HCL Link w1 and w1new Compute Tw1= Tw1new = w1-w1new  Triangulate yw1new Fix yw1 and yw2 adjust yw1new according to CCA gradient flow for one iteration t < t introduction CCA GCCA algorithm analysis of bridges simulations application conclusions

33 Analysis of bridges nonstationary: jump t >= t
t >= t The jump is detected by the bridges introduction CCA GCCA algorithm analysis of bridges simulations application conclusions

34 Analysis of bridges nonstationary: jump
introduction CCA GCCA algorithm analysis of bridges simulations application conclusions

35 Analysis of bridges nonstationary: jump
introduction CCA GCCA algorithm analysis of bridges simulations application conclusions

36 Analysis of bridges nonstationary: smooth displacement
possible bridges in the upper area no bridges in the overlap area t 2 t 1 The density of bridges is proportional to the displacement velocity of the distribution introduction CCA GCCA algorithm analysis of bridges simulations application conclusions

37 Analysis of the bridge: length
If the length of the bridge is great, there are two interpretations : bridge link old neuron new neuron double neuron bridge no link old neuron new neuron Novelty detection Outlier (bridge pruning) introduction CCA GCCA algorithm analysis of bridges simulations application conclusions

38 Simulations data space latent space
introduction CCA GCCA algorithm analysis of bridges simulations application conclusions

39 Simulations simplified onCCA no seeds no bridges no double neurons
introduction CCA GCCA algorithm analysis of bridges simulations application conclusions

40 Simulations simplified onCCA good initial conditions
bad initial conditions introduction CCA GCCA algorithm analysis of bridges simulations application conclusions

41 Simulations simplified onCCA noisy data
introduction CCA GCCA algorithm analysis of bridges simulations application conclusions

42 Application NASA data repository : experiments on bearings' accelerated life tests provided by FEMTO-ST Institute, Besancon, France CCA dimensional vectors whose components correspond to statistical features extracted by measurements drawn from four vibration transducers installed in a kinematic chain of an electrical motor nonstationary framework evolving from the healthy state to a double fault occurring in the inner-race of a bearing and in the ball of another bearing Intrinsic dim = 3 introduction CCA GCCA algorithm analysis of bridges simulations application conclusions

43 Application simplified onCCA
introduction CCA GCCA algorithm analysis of bridges simulations application conclusions

44 Application simplified onCCA
novelty detection by means of the outlier frequency fault onset introduction CCA GCCA algorithm analysis of bridges simulations application conclusions

45 Application GCCA introduction CCA GCCA algorithm analysis of bridges simulations application conclusions

46 Application latent space
introduction CCA GCCA algorithm analysis of bridges simulations application conclusions

47 Application latent space
introduction CCA onCCA algorithm analysis of bridges simulations application conclusions

48 euclidean geodesic equivalence
Conclusions  coordination euclidean geodesic equivalence local seed bridge topology preserving distance preserving colonization nonstationary outlier introduction CCA GCCA algorithm analysis of bridges simulations application conclusions

49 Aaron Nimzowitsch ( )

Download ppt "Growing Curvilinear Component Analysis (GCCA)"

Similar presentations

Memristor in Learning Neural Networks

Memristor in Learning Neural Networks
2806 Neural Computation Self-Organizing Maps Lecture 9 2005 Ari Visa.

2806 Neural Computation Self-Organizing Maps Lecture Ari Visa.
Introduction to Training and Learning in Neural Networks n CS/PY 399 Lab Presentation # 4 n February 1, 2001 n Mount Union College.

Introduction to Training and Learning in Neural Networks n CS/PY 399 Lab Presentation # 4 n February 1, 2001 n Mount Union College.
Vehicles Detection From Aerial Sequences 4 th European Micro-UAV Meeting University of Picardie Jules Verne 1 VEHICLES DETECTION FROM AERIAL SEQUENCES.

Vehicles Detection From Aerial Sequences 4 th European Micro-UAV Meeting University of Picardie Jules Verne 1 VEHICLES DETECTION FROM AERIAL SEQUENCES.
Self Organization: Competitive Learning

Self Organization: Competitive Learning
5/16/2015Intelligent Systems and Soft Computing1 Introduction Introduction Hebbian learning Hebbian learning Generalised Hebbian learning algorithm Generalised.

5/16/2015Intelligent Systems and Soft Computing1 Introduction Introduction Hebbian learning Hebbian learning Generalised Hebbian learning algorithm Generalised.
Artificial neural networks:

Artificial neural networks:
Competitive learning College voor cursus Connectionistische modellen M Meeter.

Competitive learning College voor cursus Connectionistische modellen M Meeter.
Non-linear Dimensionality Reduction CMPUT 466/551 Nilanjan Ray Prepared on materials from the book Non-linear dimensionality reduction By Lee and Verleysen,

Non-linear Dimensionality Reduction CMPUT 466/551 Nilanjan Ray Prepared on materials from the book Non-linear dimensionality reduction By Lee and Verleysen,
Machine Learning Neural Networks

Machine Learning Neural Networks
A New Block Based Motion Estimation with True Region Motion Field Jozef Huska & Peter Kulla EUROCON 2007 The International Conference on “Computer as a.

A New Block Based Motion Estimation with True Region Motion Field Jozef Huska & Peter Kulla EUROCON 2007 The International Conference on “Computer as a.
ETHEM ALPAYDIN © The MIT Press, 2010 Lecture Slides for.

ETHEM ALPAYDIN © The MIT Press, Lecture Slides for.
Slides are based on Negnevitsky, Pearson Education, 2005 1 Lecture 8 Artificial neural networks: Unsupervised learning n Introduction n Hebbian learning.

Slides are based on Negnevitsky, Pearson Education, Lecture 8 Artificial neural networks: Unsupervised learning n Introduction n Hebbian learning.
1 Abstract This paper presents a novel modification to the classical Competitive Learning (CL) by adding a dynamic branching mechanism to neural networks.

1 Abstract This paper presents a novel modification to the classical Competitive Learning (CL) by adding a dynamic branching mechanism to neural networks.
November 9, 2010Neural Networks Lecture 16: Counterpropagation 1 Unsupervised Learning So far, we have only looked at supervised learning, in which an.

November 9, 2010Neural Networks Lecture 16: Counterpropagation 1 Unsupervised Learning So far, we have only looked at supervised learning, in which an.
MSU CSE 803 Stockman Linear Operations Using Masks Masks are patterns used to define the weights used in averaging the neighbors of a pixel to compute.

MSU CSE 803 Stockman Linear Operations Using Masks Masks are patterns used to define the weights used in averaging the neighbors of a pixel to compute.
Manifold Learning: ISOMAP Alan O'Connor April 29, 2008.

Manifold Learning: ISOMAP Alan O'Connor April 29, 2008.
Pattern Classification All materials in these slides were taken from Pattern Classification (2nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John Wiley.

Pattern Classification All materials in these slides were taken from Pattern Classification (2nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John Wiley.
Implementing a reliable neuro-classifier

Implementing a reliable neuro-classifier
Three Algorithms for Nonlinear Dimensionality Reduction Haixuan Yang Group Meeting Jan. 011, 2005.

Three Algorithms for Nonlinear Dimensionality Reduction Haixuan Yang Group Meeting Jan. 011, 2005.

Similar presentations

Ads by Google