Data Projections & Visualization Rajmonda Caceres MIT Lincoln Laboratory.

Slides:

Advertisements

Similar presentations

Data Analysis Lecture 8 Tijl De Bie. Dimensionality reduction How to deal with high-dimensional data? How to visualize it? How to explore it? Dimensionality.

Advertisements

Text mining Gergely Kótyuk Laboratory of Cryptography and System Security (CrySyS) Budapest University of Technology and Economics

Manifold Learning Dimensionality Reduction. Outline Introduction Dim. Reduction Manifold Isomap Overall procedure Approximating geodesic dist. Dijkstra’s.

Presented by: Mingyuan Zhou Duke University, ECE April 3, 2009

Computer Vision Spring ,-685 Instructor: S. Narasimhan Wean 5403 T-R 3:00pm – 4:20pm Lecture #20.

ETHEM ALPAYDIN © The MIT Press, Lecture Slides for.

Dimensional reduction, PCA

Principal Component Analysis IML Outline Max the variance of the output coordinates Optimal reconstruction Generating data Limitations of PCA.

1 Efficient Clustering of High-Dimensional Data Sets Andrew McCallum WhizBang! Labs & CMU Kamal Nigam WhizBang! Labs Lyle Ungar UPenn.

SVD and PCA COS 323. Dimensionality Reduction Map points in high-dimensional space to lower number of dimensionsMap points in high-dimensional space to.

The Terms that You Have to Know! Basis, Linear independent, Orthogonal Column space, Row space, Rank Linear combination Linear transformation Inner product.

Three Algorithms for Nonlinear Dimensionality Reduction Haixuan Yang Group Meeting Jan. 011, 2005.

INTRODUCTION TO Machine Learning ETHEM ALPAYDIN © The MIT Press, Lecture Slides for.

A Global Geometric Framework for Nonlinear Dimensionality Reduction Joshua B. Tenenbaum, Vin de Silva, John C. Langford Presented by Napat Triroj.

Atul Singh Junior Undergraduate CSE, IIT Kanpur.  Dimension reduction is a technique which is used to represent a high dimensional data in a more compact.

Principal Component Analysis. Consider a collection of points.

NonLinear Dimensionality Reduction or Unfolding Manifolds Tennenbaum|Silva|Langford [Isomap] Roweis|Saul [Locally Linear Embedding] Presented by Vikas.

Lightseminar: Learned Representation in AI An Introduction to Locally Linear Embedding Lawrence K. Saul Sam T. Roweis presented by Chan-Su Lee.

CS 485/685 Computer Vision Face Recognition Using Principal Components Analysis (PCA) M. Turk, A. Pentland, "Eigenfaces for Recognition", Journal of Cognitive.

Nonlinear Dimensionality Reduction Approaches. Dimensionality Reduction The goal: The meaningful low-dimensional structures hidden in their high-dimensional.

Manifold learning: Locally Linear Embedding Jieping Ye Department of Computer Science and Engineering Arizona State University

The Tutorial of Principal Component Analysis, Hierarchical Clustering, and Multidimensional Scaling Wenshan Wang.

Summarized by Soo-Jin Kim

Dimensionality Reduction: Principal Components Analysis Optional Reading: Smith, A Tutorial on Principal Components Analysis (linked to class webpage)

Principal Components Analysis BMTRY 726 3/27/14. Uses Goal: Explain the variability of a set of variables using a “small” set of linear combinations of.

CSE 185 Introduction to Computer Vision Pattern Recognition.

Feature extraction 1.Introduction 2.T-test 3.Signal Noise Ratio (SNR) 4.Linear Correlation Coefficient (LCC) 5.Principle component analysis (PCA) 6.Linear.

Principal Coordinate Analysis, Correspondence Analysis and Multidimensional Scaling: Multivariate Analysis of Association Matrices BIOL4062/5062 Hal Whitehead.

1 Recognition by Appearance Appearance-based recognition is a competing paradigm to features and alignment. No features are extracted! Images are represented.

Principal Component Analysis Bamshad Mobasher DePaul University Bamshad Mobasher DePaul University.

Computer Vision Lab. SNU Young Ki Baik Nonlinear Dimensionality Reduction Approach (ISOMAP, LLE)

ISOMAP TRACKING WITH PARTICLE FILTER Presented by Nikhil Rane.

GRASP Learning a Kernel Matrix for Nonlinear Dimensionality Reduction Kilian Q. Weinberger, Fei Sha and Lawrence K. Saul ICML’04 Department of Computer.

Principal Component Analysis (PCA). Data Reduction summarization of data with many (p) variables by a smaller set of (k) derived (synthetic, composite)

CSE 185 Introduction to Computer Vision Face Recognition.

Dimensionality Reduction

Manifold learning: MDS and Isomap

Nonlinear Dimensionality Reduction Approach (ISOMAP)

Jan Kamenický.  Many features ⇒ many dimensions  Dimensionality reduction ◦ Feature extraction (useful representation) ◦ Classification ◦ Visualization.

Non-Linear Dimensionality Reduction

Project 11: Determining the Intrinsic Dimensionality of a Distribution Okke Formsma, Nicolas Roussis and Per Løwenborg.

Tony Jebara, Columbia University Advanced Machine Learning & Perception Instructor: Tony Jebara.

Supervisor: Nakhmani Arie Semester: Winter 2007 Target Recognition Harmatz Isca.

EIGENSYSTEMS, SVD, PCA Big Data Seminar, Dedi Gadot, December 14 th, 2014.

Math 285 Project Diffusion Maps Xiaoyan Chong Department of Mathematics and Statistics San Jose State University.

Feature Selection and Dimensionality Reduction. “Curse of dimensionality” – The higher the dimensionality of the data, the more data is needed to learn.

ECE 8443 – Pattern Recognition ECE 8527 – Introduction to Machine Learning and Pattern Recognition LECTURE 10: PRINCIPAL COMPONENTS ANALYSIS Objectives:

CS685 : Special Topics in Data Mining, UKY The UNIVERSITY of KENTUCKY Dimensionality Reduction CS 685: Special Topics in Data Mining Spring 2008 Jinze.

Multivariate statistical methods. Multivariate methods multivariate dataset – group of n objects, m variables (as a rule n>m, if possible). confirmation.

Out of sample extension of PCA, Kernel PCA, and MDS WILSON A. FLORERO-SALINAS DAN LI MATH 285, FALL

The UNIVERSITY of NORTH CAROLINA at CHAPEL HILL Dimensionality Reduction Part 1: Linear Methods Comp Spring 2007.

Machine Learning Supervised Learning Classification and Regression K-Nearest Neighbor Classification Fisher’s Criteria & Linear Discriminant Analysis Perceptron:

1 C.A.L. Bailer-Jones. Machine Learning. Data exploration and dimensionality reduction Machine learning, pattern recognition and statistical data modelling.

Spectral Methods for Dimensionality

Principal Component Analysis (PCA)

Dimensionality Reduction

INTRODUCTION TO Machine Learning 3rd Edition

LECTURE 10: DISCRIMINANT ANALYSIS

Dipartimento di Ingegneria «Enzo Ferrari»,

Principal Component Analysis (PCA)

Dimensionality Reduction

Machine Learning Dimensionality Reduction

Principal Component Analysis

PCA is “an orthogonal linear transformation that transfers the data to a new coordinate system such that the greatest variance by any projection of the.

Outline H. Murase, and S. K. Nayar, “Visual learning and recognition of 3-D objects from appearance,” International Journal of Computer Vision, vol. 14,

Dimensionality Reduction

Pattern Classification All materials in these slides were taken from Pattern Classification (2nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John.

CS4670: Intro to Computer Vision

LECTURE 09: DISCRIMINANT ANALYSIS

NonLinear Dimensionality Reduction or Unfolding Manifolds

Presentation transcript:

Data Projections & Visualization Rajmonda Caceres MIT Lincoln Laboratory

Reduce complexity Visual Computational Identify the intrinsic dimensionality of data Identify the most relevant aspects of data given a task

Lower Dimension Higher Dimension

a) b) Not all projections are equal

Desired properties Reduced, compressed representation Preserved useful/intrinsic properties of the data Applify patterns of interest (e.g. outliers) Simple, interpretable Trade-off between simplicity and preservation of structure

Helps us organize the data Helps us discriminate patterns

Manhattan distance (1 norm, taxicab distance) Euclidean distance (2 norm)

L-p Distance As p grows the largest coordinate distances tends to dominate the global distance

Projective methods: preserve a property of data Principal Component Analysis (PCA) Many others: ICA, Factor Analysis, Manifold Learning Multidimensional Dimension Reduction (MDS) LLE, Isomap

Goal: Find a linear projection that captures most of variance 1 st Principal Component 2 nd Principal Component 1 st Principal Component

PCA pseudo code: Centralize the data by subtracting the mean Calculate the covariance matrix: Calculate the eigenvectors(principal components) of the covariance matrix Select top few(2-3) eigenvectors (highest eigenvalues) Project the data using these eigenvectors as axis

Screeplot Biplot

Goal: Find a lower embedding of the data that preserves pairwise distances Formally: : Input distance values : Output distances values

Shepard Diagram MDS Distances Data Distances

More features are not necessarily better Understand the assumptions of different modeling choices When choosing distance functions, projection methods Consider the characteristics of the data Consider the learning objective Explore multiple choices simultaneously to gain better insight

clustering-in-r/ Multidimensional Scaling, Leland Wilkinson Dimension Reduction: A Guided Tour, Christopher J.C. Burgesti When is “nearest neighbor” meaningful?, Beyer, K.S., GoldStein, J. Ramakrishnan, R. & Shaft g, by

The effect of concentration of distances Lower DimensionHigher Dimension