Non-Linear Dimensionality Reduction

Slides:

Advertisements

Similar presentations

M. Belkin and P. Niyogi, Neural Computation, pp. 1373–1396, 2003.

Advertisements

Principal Component Analysis Based on L1-Norm Maximization Nojun Kwak IEEE Transactions on Pattern Analysis and Machine Intelligence, 2008.

Input Space versus Feature Space in Kernel- Based Methods Scholkopf, Mika, Burges, Knirsch, Muller, Ratsch, Smola presented by: Joe Drish Department of.

Dimensionality reduction. Outline From distances to points : – MultiDimensional Scaling (MDS) Dimensionality Reductions or data projections Random projections.

Dimensionality Reduction PCA -- SVD

Manifold Learning Dimensionality Reduction. Outline Introduction Dim. Reduction Manifold Isomap Overall procedure Approximating geodesic dist. Dijkstra’s.

Presented by: Mingyuan Zhou Duke University, ECE April 3, 2009

Non-linear Dimensionality Reduction CMPUT 466/551 Nilanjan Ray Prepared on materials from the book Non-linear dimensionality reduction By Lee and Verleysen,

University of Joensuu Dept. of Computer Science P.O. Box 111 FIN Joensuu Tel fax Isomap Algorithm.

Principal Component Analysis CMPUT 466/551 Nilanjan Ray.

One-Shot Multi-Set Non-rigid Feature-Spatial Matching

“Random Projections on Smooth Manifolds” -A short summary

ETHEM ALPAYDIN © The MIT Press, Lecture Slides for.

Dimensionality Reduction and Embeddings

LLE and ISOMAP Analysis of Robot Images Rong Xu. Background Intuition of Dimensionality Reduction Linear Approach –PCA(Principal Component Analysis) Nonlinear.

Correspondence & Symmetry

Principal Component Analysis IML Outline Max the variance of the output coordinates Optimal reconstruction Generating data Limitations of PCA.

1 Numerical geometry of non-rigid shapes Spectral Methods Tutorial. Spectral Methods Tutorial 6 © Maks Ovsjanikov tosca.cs.technion.ac.il/book Numerical.

Manifold Learning: ISOMAP Alan O'Connor April 29, 2008.

Lecture 21 SVD and Latent Semantic Indexing and Dimensional Reduction

The Terms that You Have to Know! Basis, Linear independent, Orthogonal Column space, Row space, Rank Linear combination Linear transformation Inner product.

Three Algorithms for Nonlinear Dimensionality Reduction Haixuan Yang Group Meeting Jan. 011, 2005.

Dimension reduction : PCA and Clustering Christopher Workman Center for Biological Sequence Analysis DTU.

3D Geometry for Computer Graphics

INTRODUCTION TO Machine Learning ETHEM ALPAYDIN © The MIT Press, Lecture Slides for.

A Global Geometric Framework for Nonlinear Dimensionality Reduction Joshua B. Tenenbaum, Vin de Silva, John C. Langford Presented by Napat Triroj.

Atul Singh Junior Undergraduate CSE, IIT Kanpur.  Dimension reduction is a technique which is used to represent a high dimensional data in a more compact.

Dimensionality Reduction

NonLinear Dimensionality Reduction or Unfolding Manifolds Tennenbaum|Silva|Langford [Isomap] Roweis|Saul [Locally Linear Embedding] Presented by Vikas.

Lightseminar: Learned Representation in AI An Introduction to Locally Linear Embedding Lawrence K. Saul Sam T. Roweis presented by Chan-Su Lee.

Dimensionality Reduction. Multimedia DBs Many multimedia applications require efficient indexing in high-dimensions (time-series, images and videos, etc)

Random Projections of Signal Manifolds Michael Wakin and Richard Baraniuk Random Projections for Manifold Learning Chinmay Hegde, Michael Wakin and Richard.

Nonlinear Dimensionality Reduction by Locally Linear Embedding Sam T. Roweis and Lawrence K. Saul Reference: "Nonlinear dimensionality reduction by locally.

Nonlinear Dimensionality Reduction Approaches. Dimensionality Reduction The goal: The meaningful low-dimensional structures hidden in their high-dimensional.

SVD(Singular Value Decomposition) and Its Applications

Manifold learning: Locally Linear Embedding Jieping Ye Department of Computer Science and Engineering Arizona State University

Graph Embedding: A General Framework for Dimensionality Reduction Dong XU School of Computer Engineering Nanyang Technological University

IEEE TRANSSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE

Data Reduction. 1.Overview 2.The Curse of Dimensionality 3.Data Sampling 4.Binning and Reduction of Cardinality.

Learning a Kernel Matrix for Nonlinear Dimensionality Reduction By K. Weinberger, F. Sha, and L. Saul Presented by Michael Barnathan.

Neighbourhood relation preservation (NRP) A rank-based data visualisation quality assessment criterion Jigang Sun PhD studies finished in July 2011 PhD.

Computer Vision Lab. SNU Young Ki Baik Nonlinear Dimensionality Reduction Approach (ISOMAP, LLE)

Local Fisher Discriminant Analysis for Supervised Dimensionality Reduction Presented by Xianwang Wang Masashi Sugiyama.

ISOMAP TRACKING WITH PARTICLE FILTER Presented by Nikhil Rane.

GRASP Learning a Kernel Matrix for Nonlinear Dimensionality Reduction Kilian Q. Weinberger, Fei Sha and Lawrence K. Saul ICML’04 Department of Computer.

Chapter 7 Multivariate techniques with text Parallel embedded system design lab 이청용.

Manifold learning: MDS and Isomap

1 LING 696B: MDS and non-linear methods of dimension reduction.

Nonlinear Dimensionality Reduction Approach (ISOMAP)

Jan Kamenický.  Many features ⇒ many dimensions  Dimensionality reduction ◦ Feature extraction (useful representation) ◦ Classification ◦ Visualization.

Project 11: Determining the Intrinsic Dimensionality of a Distribution Okke Formsma, Nicolas Roussis and Per Løwenborg.

Principal Component Analysis Machine Learning. Last Time Expectation Maximization in Graphical Models – Baum Welch.

Tony Jebara, Columbia University Advanced Machine Learning & Perception Instructor: Tony Jebara.

Data Projections & Visualization Rajmonda Caceres MIT Lincoln Laboratory.

Data Mining Course 2007 Eric Postma Clustering. Overview Three approaches to clustering 1.Minimization of reconstruction error PCA, nlPCA, k-means clustering.

Math 285 Project Diffusion Maps Xiaoyan Chong Department of Mathematics and Statistics San Jose State University.

CSC321: Lecture 25: Non-linear dimensionality reduction Geoffrey Hinton.

Nonlinear Dimension Reduction: Semi-Definite Embedding vs. Local Linear Embedding Li Zhang and Lin Liao.

CSC321: Extra Lecture (not on the exam) Non-linear dimensionality reduction Geoffrey Hinton.

Out of sample extension of PCA, Kernel PCA, and MDS WILSON A. FLORERO-SALINAS DAN LI MATH 285, FALL

Eric Xing © Eric CMU, Machine Learning Data visualization and dimensionality reduction Eric Xing Lecture 7, August 13, 2010.

Spectral Methods for Dimensionality

Dipartimento di Ingegneria «Enzo Ferrari»,

Spectral Methods Tutorial 6 1 © Maks Ovsjanikov

Machine Learning Dimensionality Reduction

Outline Nonlinear Dimension Reduction Brief introduction Isomap LLE

Principal Component Analysis

Lecture 13: Singular Value Decomposition (SVD)

Nonlinear Dimension Reduction:

NonLinear Dimensionality Reduction or Unfolding Manifolds

Presentation transcript:

Non-Linear Dimensionality Reduction Presented by: Johnathan Franck Mentor: Alex Cloninger

Outline Different Representations 5 Techniques Principal component analysis (PCA)/Multi-dimensional scaling (MDS) Sammons non-linear mapping Isomap Kernel PCA Locally linear embedding Less details

Different viewpoints Pull out specific information using mathematical techniques

Principal Component Analysis/Multidimensional Scaling Singular Value Decomposition of A:

PCA/MDS Formulation Distance Preserved Rotation X – reduced dimension Y – higher dimension W - Orthogonal Singular Value decomposition of Y equivalent to Eigenvalue decomposition of gram matrix Preserve Gram Matrix: S Optimization: Linear Algebra Distance Preserved Rotation

Swiss Roll Open Box

Sammons Nonlinear mapping Weight distances differently

Swiss Roll Open Box

Isomap Approximate geodesic Build graph with K-neighbors/ε-ball Weight graph with Euclidean distance Compute pairwise distances with Dijkstra’s algorithm

Swiss Roll Open Box

Kernel PCA What other distance functions? Mercer’s theorem

Swiss Roll Open Box

Local Linear Embedding Preserve relation to local points

Swiss Roll Open Box

Isomap Original 3 dimensions PCA Kernel PCA Sammon’s nonlinear mapping Locally Linear Embedding

Thanks Mentor: Alex Cloninger DRP program Contact information: Jafranck2@gmail.com

Sources Images Nonlinear Dimensionality reduction By John Lee and Michael Verleyson http://blog.pluskid.org/wp- content/uploads/2010/05/isomap-graph.png http://imagineatness.files.wordpress.com/2011/12/ gaussian.png http://scikit- learn.org/0.11/_images/plot_kernel_pca_1.png http://www.cs.nyu.edu/~roweis/lle/images/llef2me d.gif http://upload.wikimedia.org/wikipedia/commons/b /bb/SOMsPCA.PNG