Curvilinear Component Analysis and Bregman divergences

Slides:

Advertisements

Similar presentations

Similarity Search on Bregman Divergence, Towards Non- Metric Indexing Zhenjie Zhang, Beng Chi Ooi, Srinivasan Parthasarathy, Anthony K. H. Tung.

Advertisements

Non-linear Dimensionality Reduction CMPUT 466/551 Nilanjan Ray Prepared on materials from the book Non-linear dimensionality reduction By Lee and Verleysen,

University of Joensuu Dept. of Computer Science P.O. Box 111 FIN Joensuu Tel fax Isomap Algorithm.

“Random Projections on Smooth Manifolds” -A short summary

Spectral embedding Lecture 6 1 © Alexander & Michael Bronstein

6. One-Dimensional Continuous Groups 6.1 The Rotation Group SO(2) 6.2 The Generator of SO(2) 6.3 Irreducible Representations of SO(2) 6.4 Invariant Integration.

Dimensional reduction, PCA

Extending metric multidimensional scaling with Bregman divergences

Spectral Embedding Alexander Bronstein, Michael Bronstein

1/ 30. Problems for classical IR models Introduction & Background(LSI,SVD,..etc) Example Standard query method Analysis standard query method Seeking.

Bregman Divergences in Clustering and Dimensionality Reduction COMS : Learning and Empirical Inference Irina Rish IBM T.J. Watson Research Center.

Proximity matrices and scaling Purpose of scaling Similarities and dissimilarities Classical Euclidean scaling Non-Euclidean scaling Horseshoe effect Non-Metric.

The Implicit Mapping into Feature Space. In order to learn non-linear relations with a linear machine, we need to select a set of nonlinear features.

Gradient Methods Yaron Lipman May Preview Background Steepest Descent Conjugate Gradient.

Particle Filtering for Non- Linear/Non-Gaussian System Bohyung Han

A Global Geometric Framework for Nonlinear Dimensionality Reduction Joshua B. Tenenbaum, Vin de Silva, John C. Langford Presented by Napat Triroj.

ISM 206 Lecture 6 Nonlinear Unconstrained Optimization.

1 Numerical geometry of non-rigid shapes Non-Euclidean Embedding Non-Euclidean Embedding Lecture 6 © Alexander & Michael Bronstein tosca.cs.technion.ac.il/book.

1 M. Bronstein Multigrid multidimensional scaling Multigrid Multidimensional Scaling Michael M. Bronstein Department of Computer Science Technion – Israel.

Summarized by Soo-Jin Kim

Cao et al. ICML 2010 Presented by Danushka Bollegala.

Biointelligence Laboratory, Seoul National University

Non Negative Matrix Factorization

Intelligent Database Systems Lab 國立雲林科技大學 National Yunlin University of Science and Technology Adaptive nonlinear manifolds and their applications to pattern.

Dimensionality reduction: Some Assumptions High-dimensional data often lies on or near a much lower dimensional, curved manifold. A good way to represent.

Discriminant Functions

GAUSSIAN PROCESS FACTORIZATION MACHINES FOR CONTEXT-AWARE RECOMMENDATIONS Trung V. Nguyen, Alexandros Karatzoglou, Linas Baltrunas SIGIR 2014 Presentation:

Neighbourhood relation preservation (NRP) A rank-based data visualisation quality assessment criterion Jigang Sun PhD studies finished in July 2011 PhD.

From: McCune, B. & J. B. Grace Analysis of Ecological Communities. MjM Software Design, Gleneden Beach, Oregon

Multidimensional Scaling Vuokko Vuori Based on: Data Exploration Using Self-Organizing Maps, Samuel Kaski, Ph.D. Thesis, 1997 Multivariate Statistical.

Low-Rank Kernel Learning with Bregman Matrix Divergences Brian Kulis, Matyas A. Sustik and Inderjit S. Dhillon Journal of Machine Learning Research 10.

Manifold learning: MDS and Isomap

Jan Kamenický.  Many features ⇒ many dimensions  Dimensionality reduction ◦ Feature extraction (useful representation) ◦ Classification ◦ Visualization.

Non-Linear Dimensionality Reduction

Tony Jebara, Columbia University Advanced Machine Learning & Perception Instructor: Tony Jebara.

CpSc 881: Machine Learning PCA and MDS. 2 Copy Right Notice Most slides in this presentation are adopted from slides of text book and various sources.

Multidimensional Scaling By Marc Sobel. The Goal  We observe (possibly non-euclidean) proximity data. For each pair of objects number ‘i’ and ‘j’ we.

Extended Kalman Filter

Extending metric multidimensional scaling with Bregman divergences Mr. Jigang Sun Supervisor: Prof. Colin Fyfe Nov 2009.

ViSOM － A Novel Method for Multivariate Data Projection and Structure Visualization Advisor : Dr. Hsu Graduate : Sheng-Hsuan Wang Author : Hujun Yin.

Linear machines márc Decison surfaces We focus now on the decision surfaces Linear machines = linear decision surface Non-optimal solution but.

High Dimensional Probabilistic Modelling through Manifolds

Section 6.1 Inner Products.

Neil Lawrence Machine Learning Group Department of Computer Science

Module 5 Optional Component

Deep Feedforward Networks

Chapter 9: Differential Analysis of Fluid Flow

Date of download: 11/9/2017 Copyright © ASME. All rights reserved.

Even More on Dimensions

Dimensionality Reduction

Linear machines 28/02/2017.

Clustering and Multidimensional Scaling

Today (2/23/16) Learning objectives:

Dimensionality Reduction

COSC 4335: Other Classification Techniques

Multidimensional Scaling

Support Vector Machines

Extended Kalman Filter

Video Analysis via Nonlinear Dimensionality Reduction Technique

Non-negative Matrix Factorization (NMF)

Physics 451/551 Theoretical Mechanics

NonLinear Dimensionality Reduction or Unfolding Manifolds

Gauge invariant computable quantities in Timelike Liouville theory

Notes are from D. R. Wilton, Dept. of ECE

Simplify (or Evaluate)

Non-Negative Matrix Factorization

Extended Kalman Filter

Can Genetic Programming Do Manifold Learning Too?

Linear regression with one variable

Presentation transcript:

Curvilinear Component Analysis and Bregman divergences Jigang Sun Colin Fyfe Malcolm Crowe 28 April 2010 University of the West of Scotland

Multidimensional Scaling(MDS) A group of information visualisation methods that projects data from high dimensional space, to a low dimensional space, often two or three dimensions, keeping inter-point dissimilarities (e.g. distances) in low dimensional space as close as possible to the original dissimilarities in high dimensional space. When Euclidean distances are used, it is Metric MDS.

Visualising 18 dimensional data

Basic MDS Improve the Sammon mapping with Bregman divergence The basic MDS, the stress function to be minimised Sammon Mapping (1969) Improve the Sammon mapping with Bregman divergence

Bregman divergence Intuitively, it is the difference between the value of F at point p and the value of the first-order Taylor expansion of F around point q evaluated at point p. q p

2 representations When F is in one variable, the Bregman Divergence is truncated Taylor series Two useful properties for MDS 1. Non-negativity 2. Non-symmetry Except in special cases such as F(x)=x^2

Improving Sammon Mapping with Bregman divergences Recall the classical Sammon Mapping (1969) Choose a base convex function Important to say the first term in last line is Sammon mapping common term: the first term of ExtendedSammon is Sammon, not considering constant coefficients

An Experiment on Swiss roll data set

Two groups of Convex functions No 1 is for the Extended Sammon mapping.

OpenBox, Sammon and FirstGroup Important to spend time saying this is a smooth deformation of box

SecondGroup on OpenBox Take time to show this slide

Curvilinear Component Analysis (CCA) and Bregman Divergences W( .) has argument the inter-point distance in latent space Good at unfolding strongly nonlinear structures Stochastic gradient descent updating rule

A version of CCA One weight function can be Updating rule

Rewriting stress function for CCA using right Bregman divergences Given convex function Emphasise that latent distances in right position. Updating rule is the same

The common term between BasicCCA and Real CCA = The first term is common with

Real CCA vs Basic CCA Note that basic cca takes latent further away if they are to the right of critical point

Conclusions We introduced The Extended Sammon mapping vs the Sammon mapping We create two groups of left Bregman divergences and experiment on artificial data sets. A right Bregman divergence redefines the stress function for Curvilinear Component Analysis Any questions?