1 UNC, Stat & OR Nonnegative Matrix Factorization.

Slides:

Advertisements

Similar presentations

Object Orie’d Data Analysis, Last Time •Clustering –Quantify with Cluster Index –Simple 1-d examples –Local mininizers –Impact of outliers •SigClust –When.

Advertisements

SHAPE THEORY USING GEOMETRY OF QUOTIENT SPACES: STORY STORY SHAPE THEORY USING GEOMETRY OF QUOTIENT SPACES: STORY STORY ANUJ SRIVASTAVA Dept of Statistics.

Part II – TIME SERIES ANALYSIS C3 Exponential Smoothing Methods © Angel A. Juan & Carles Serrat - UPC 2007/2008.

Independent Component Analysis Personal Viewpoint: Directions that maximize independence Motivating Context: Signal Processing “Blind Source Separation”

Computer vision: models, learning and inference Chapter 13 Image preprocessing and feature extraction.

Bet you’ve never seen a graph like this one before. . .

Object Orie’d Data Analysis, Last Time Finished NCI 60 Data Started detailed look at PCA Reviewed linear algebra Today: More linear algebra Multivariate.

A 4-WEEK PROJECT IN Active Shape and Appearance Models

Principal Component Analysis CMPUT 466/551 Nilanjan Ray.

Tracking Intersection Curves of Two Deforming Parametric Surfaces Xianming Chen¹, Richard Riesenfeld¹ Elaine Cohen¹, James Damon² ¹School of Computing,

Tools for Reduction of Mechanical Systems Ravi Balasubramanian, Klaus Schmidt, and Elie Shammas Carnegie Mellon University.

The Table Lens: Merging Graphical and Symbolic Representations in an Interactive Focus + Context Visualization for Tabular Information R. Rao and S. K.

PATTERN RECOGNITION : PRINCIPAL COMPONENTS ANALYSIS Prof.Dr.Cevdet Demir

Business Math, Eighth Edition Cleaves/Hobbs © 2009 Pearson Education, Inc. Upper Saddle River, NJ All Rights Reserved 7.1 Measures of Central Tendency.

Atul Singh Junior Undergraduate CSE, IIT Kanpur.  Dimension reduction is a technique which is used to represent a high dimensional data in a more compact.

Principal component analysis (PCA) Purpose of PCA Covariance and correlation matrices PCA using eigenvalues PCA using singular value decompositions Selection.

12.3 – Measures of Dispersion

Statistics – O. R. 892 Object Oriented Data Analysis J. S. Marron Dept. of Statistics and Operations Research University of North Carolina.

Object Orie’d Data Analysis, Last Time Finished Algebra Review Multivariate Probability Review PCA as an Optimization Problem (Eigen-decomp. gives rotation,

Manifold learning: Locally Linear Embedding Jieping Ye Department of Computer Science and Engineering Arizona State University

Exploratory Data Analysis. Computing Science, University of Aberdeen2 Introduction Applying data mining (InfoVis as well) techniques requires gaining.

1 Business Math Chapter 7: Business Statistics. Cleaves/Hobbs: Business Math, 7e Copyright 2005 by Pearson Education, Inc. Upper Saddle River, NJ

Statistics – O. R. 892 Object Oriented Data Analysis J. S. Marron Dept. of Statistics and Operations Research University of North Carolina.

Object Orie’d Data Analysis, Last Time

Image Processing1 Morphological and Other Area Operations Introduction –Morphology is the science of form and structure –It is about regions or.

CSE554AlignmentSlide 1 CSE 554 Lecture 5: Alignment Fall 2011.

Chapter 11 Descriptive Statistics Gay, Mills, and Airasian

Descriptive Statistics

A Challenging Example Male Pelvis –Bladder – Prostate – Rectum.

Statistics – O. R. 891 Object Oriented Data Analysis J. S. Marron Dept. of Statistics and Operations Research University of North Carolina.

Statistics – O. R. 891 Object Oriented Data Analysis J. S. Marron Dept. of Statistics and Operations Research University of North Carolina.

Object Orie’d Data Analysis, Last Time Discrimination for manifold data (Sen) –Simple Tangent plane SVM –Iterated TANgent plane SVM –Manifold SVM Interesting.

SINGULAR VALUE DECOMPOSITION (SVD)

1 UNC, Stat & OR Metrics in Curve Space. 2 UNC, Stat & OR Metrics in Curve Quotient Space Above was Invariance for Individual Curves Now extend to: 

CSE554AlignmentSlide 1 CSE 554 Lecture 8: Alignment Fall 2013.

Geometry of Shape Manifolds

Maximal Data Piling Visual similarity of & ? Can show (Ahn & Marron 2009), for d < n: I.e. directions are the same! How can this be? Note lengths are different.

Common Property of Shape Data Objects: Natural Feature Space is Curved I.e. a Manifold (from Differential Geometry) Shapes As Data Objects.

1 UNC, Stat & OR PCA Extensions for Data on Manifolds Fletcher (Principal Geodesic Anal.) Best fit of geodesic to data Constrained to go through geodesic.

Return to Big Picture Main statistical goals of OODA: Understanding population structure –Low dim ’ al Projections, PCA … Classification (i. e. Discrimination)

EIGENSYSTEMS, SVD, PCA Big Data Seminar, Dedi Gadot, December 14 th, 2014.

PATTERN RECOGNITION : PRINCIPAL COMPONENTS ANALYSIS Richard Brereton

Participant Presentations Please Sign Up: Name (Onyen is fine, or …) Are You ENRolled? Tentative Title (???? Is OK) When: Thurs., Early, Oct., Nov.,

Stat 31, Section 1, Last Time Course Organization & Website What is Statistics? Data types.

Math 285 Project Diffusion Maps Xiaoyan Chong Department of Mathematics and Statistics San Jose State University.

Statistics – O. R. 893 Object Oriented Data Analysis Steve Marron Dept. of Statistics and Operations Research University of North Carolina.

New Curriculum Changes in Maths Kat Portou Maths Co-ordinator.

Object Orie’d Data Analysis, Last Time PCA Redistribution of Energy - ANOVA PCA Data Representation PCA Simulation Alternate PCA Computation Primal – Dual.

1 UNC, Stat & OR Hailuoto Workshop Object Oriented Data Analysis, I J. S. Marron Dept. of Statistics and Operations Research, University of North Carolina.

Object Orie’d Data Analysis, Last Time PCA Redistribution of Energy - ANOVA PCA Data Representation PCA Simulation Alternate PCA Computation Primal – Dual.

GWAS Data Analysis. L1 PCA Challenge: L1 Projections Hard to Interpret (i.e. Little Data Insight) Solution: 1)Compute PC Directions Using L1 2)Compute.

1 UNC, Stat & OR Hailuoto Workshop Object Oriented Data Analysis, III J. S. Marron Dept. of Statistics and Operations Research, University of North Carolina.

Unsupervised Learning II Feature Extraction

Educational Research Descriptive Statistics Chapter th edition Chapter th edition Gay and Airasian.

Cornea Data Main Point: OODA Beyond FDA Recall Interplay: Object Space  Descriptor Space.

Recall Flexibility From Kernel Embedding Idea HDLSS Asymptotics & Kernel Methods.

SigClust Statistical Significance of Clusters in HDLSS Data When is a cluster “really there”? Liu et al (2007), Huang et al (2014)

Object Orie’d Data Analysis, Last Time DiProPerm Test –Direction – Projection – Permutation –HDLSS hypothesis testing –NCI 60 Data –Particulate Matter.

Unsupervised Learning

School of Computer Science & Engineering

Unsupervised Riemannian Clustering of Probability Density Functions

Functional Data Analysis

Statistics – O. R. 881 Object Oriented Data Analysis

An Interesting Question

The Functional Space of an Activity Ashok Veeraraghavan , Rama Chellappa, Amit Roy-Chowdhury Avinash Ravichandran.

Principal Nested Spheres Analysis

Today is Last Class Meeting

Brief Review of Recognition + Context

Principal Component Analysis

Statistics – O. R. 891 Object Oriented Data Analysis

Presentation transcript:

1 UNC, Stat & OR Nonnegative Matrix Factorization

2 UNC, Stat & OR Standard NMF But Note Not Nested No “Multi-scale” Analysis Possible (Scores Plot?!?) Nonnegative Matrix Factorization

3 UNC, Stat & OR Same Toy Data Set Rank 1 Approx. Properly Nested Nonnegative Nested Cone Analysis

4 UNC, Stat & OR 5-d Toy Example Rank 3 NNCA Approx. Current Research: How Many Nonneg. Basis El’ts Needed? Nonnegative Nested Cone Analysis

5 UNC, Stat & OR How generally applicable is Backwards approach to PCA? Potential Application: Principal Curves Hastie & Stuetzle, (1989) (Foundation of Manifold Learning) An Interesting Question

6 UNC, Stat & OR How generally applicable is Backwards approach to PCA? An Attractive Answer: James Damon, UNC Mathematics Key Idea: Express Backwards PCA as Nested Series of Constraints An Interesting Question

7 UNC, Stat & OR New Topic Curve Registration

8 UNC, Stat & OR Context Functional Data Analysis Curves as Data Objects Toy Example: How Can We Understand Variation?

9 UNC, Stat & OR Context Functional Data Analysis Curves as Data Objects Toy Example: How Can We Understand Variation?

10 UNC, Stat & OR Functional Data Analysis Insightful Decomposition Vertical Variation Horiz’l Var’n

11 UNC, Stat & OR Challenge  Fairly Large Literature  Many (Diverse) Past Attempts  Limited Success (in General)  Surprisingly Slippery (even mathematical formulation)

12 UNC, Stat & OR Challenge (Illustrated) Thanks to Wei Wu

13 UNC, Stat & OR Challenge (Illustrated) Thanks to Wei Wu

14 UNC, Stat & OR Functional Data Analysis Appropriate Mathematical Framework? Vertical Variation Horiz’l Var’n

15 UNC, Stat & OR Landmark Based Shape Analysis Approach: Identify objects that are: Translations Rotations Scalings of each other Mathematics: Equivalence Relation Results in: Equivalence Classes Which become the Data Objects

16 UNC, Stat & OR Curve Registration

17 UNC, Stat & OR Time Warping Intuition

18 UNC, Stat & OR Curve Registration

19 UNC, Stat & OR Data Objects I

20 UNC, Stat & OR Data Objects I Equivalence Classes of Curves (Set of All Warps of Given Curve) Next Task: Find Metric on Equivalence Classes

21 UNC, Stat & OR Metrics in Curve Space Find Metric on Equivalence Classes Start with Warp Invariance on Curves & Extend

22 UNC, Stat & OR Metrics in Curve Space

23 UNC, Stat & OR Metrics in Curve Space

24 UNC, Stat & OR Metrics in Curve Space

25 UNC, Stat & OR Metrics in Curve Space

26 UNC, Stat & OR Metrics in Curve Space

27 UNC, Stat & OR Metrics in Curve Space

28 UNC, Stat & OR Metrics in Curve Space

29 UNC, Stat & OR Metrics in Curve Space

30 UNC, Stat & OR Metrics in Curve Space

31 UNC, Stat & OR Metrics in Curve Space

32 UNC, Stat & OR Metrics in Curve Space Why square roots?

33 UNC, Stat & OR Metrics in Curve Space Why square roots? Thanks to Xiaosun Lu

34 UNC, Stat & OR Metrics in Curve Space Why square roots?

35 UNC, Stat & OR Metrics in Curve Space Why square roots?

36 UNC, Stat & OR Metrics in Curve Space Why square roots?

37 UNC, Stat & OR Metrics in Curve Space Why square roots?

38 UNC, Stat & OR Metrics in Curve Space Why square roots?

39 UNC, Stat & OR Metrics in Curve Space Why square roots?

40 UNC, Stat & OR Metrics in Curve Space Why square roots?

41 UNC, Stat & OR Metrics in Curve Space Why square roots?

42 UNC, Stat & OR Metrics in Curve Space Why square roots? Dislikes Pinching Focusses Well On Peaks of Unequal Height

43 UNC, Stat & OR Metrics in Curve Space

44 UNC, Stat & OR Metrics in Curve Quotient Space Above was Invariance for Individual Curves Now extend to:  Equivalence Classes of Curves  I.e. Orbits as Data Objects  I.e. Quotient Space

45 UNC, Stat & OR Metrics in Curve Quotient Space

46 UNC, Stat & OR Mean in Curve Quotient Space Benefit of a Metric: Allows Definition of a “Mean” Fréchet Mean Geodesic Mean Barycenter Karcher Mean

47 UNC, Stat & OR Mean in Curve Quotient Space

48 UNC, Stat & OR Mean in Curve Quotient Space

49 UNC, Stat & OR Mean in Curve Quotient Space

50 UNC, Stat & OR Mean in Curve Quotient Space

51 UNC, Stat & OR Mean in Curve Quotient Space Thanks to Anuj Srivastava

52 UNC, Stat & OR Toy Example – (Details Later) Estimated Warps (Note: Represented With Karcher Mean At Identity)

53 UNC, Stat & OR Mean in Curve Quotient Space

54 UNC, Stat & OR More Data Objects

55 UNC, Stat & OR More Data Objects Data Objects II

56 UNC, Stat & OR More Data Objects Data Objects II ~ Kendall’s Shapes

57 UNC, Stat & OR More Data Objects Data Objects III

58 UNC, Stat & OR More Data Objects Data Objects III ~ Chang’s Transfo’s

59 UNC, Stat & OR Computation Several Variations of Dynamic Programming Done by Eric Klassen, Wei Wu

60 UNC, Stat & OR Toy Example Raw Data

61 UNC, Stat & OR Toy Example Raw Data Both Horizontal And Vertical Variation

62 UNC, Stat & OR Toy Example Conventional PCA Projections

63 UNC, Stat & OR Toy Example Conventional PCA Projections Power Spread Across Spectrum

64 UNC, Stat & OR Toy Example Conventional PCA Projections Power Spread Across Spectrum

65 UNC, Stat & OR Toy Example Conventional PCA Scores

66 UNC, Stat & OR Toy Example Conventional PCA Scores Views of 1-d Curve Bending Through 4 Dim’ns’

67 UNC, Stat & OR Toy Example Conventional PCA Scores Patterns Are “Harmonics” In Scores

68 UNC, Stat & OR Toy Example Scores Plot Shows Data Are “1” Dimensional So Need Improved PCA Decomp.

69 UNC, Stat & OR Visualization

70 Toy Example Aligned Curves (Clear 1-d Vertical Var’n)

71 UNC, Stat & OR Toy Example Aligned Curve PCA Projections All Var’n In 1 st Component

72 UNC, Stat & OR Visualization

73 Toy Example Estimated Warps

74 UNC, Stat & OR Toy Example Warps, PC Projections

75 UNC, Stat & OR Toy Example Warps, PC Projections Mostly 1 st PC

76 UNC, Stat & OR Toy Example Warps, PC Projections Mostly 1 st PC, But 2 nd Helps Some

77 UNC, Stat & OR Toy Example Warps, PC Projections Rest is Not Important

78 UNC, Stat & OR Toy Example Horizontal Var’n Visualization Challenge: (Complicated) Warps Hard to Interpret Approach: Apply Warps to Template Mean (PCA components)

79 UNC, Stat & OR Toy Example Warp Compon’ts (+ Mean) Applied to Template Mean

80 UNC, Stat & OR Refined Calculations

81 UNC, Stat & OR PNS on SRVF Sphere Toy Example Tangent Space PCA (on Horiz. Var’n) Thanks to Xiaosun Lu

82 UNC, Stat & OR PNS on SRVF Sphere Toy Example PNS Projections (Fewer Modes)

83 UNC, Stat & OR PNS on SRVF Sphere Toy Example Tangent Space PCA Note: 3 Comp’s Needed for This

84 UNC, Stat & OR PNS on SRVF Sphere Toy Example PNS Projections Only 2 for This

85 UNC, Stat & OR TIC testbed Serious Data Challenge: TIC (Total Ion Count) Chromatograms Modern type of “chemical spectra” Thanks to Peter Hoffmann

86 UNC, Stat & OR TIC testbed Raw Data: 15 TIC Curves (5 Colors)

87 UNC, Stat & OR TIC testbed Special Feature: Answer Key of Known Peaks Found by Major Time & Labor Investment

88 UNC, Stat & OR TIC testbed Special Feature: Answer Key of Known Peaks Goal: Find Warps To Align These

89 UNC, Stat & OR TIC testbed Fisher – Rao Alignment

90 UNC, Stat & OR TIC testbed Fisher – Rao Alignment Spike-In Peaks

91 UNC, Stat & OR TIC testbed Next Zoom in on This Region

92 UNC, Stat & OR TIC testbed Zoomed Fisher – Rao Alignment

93 UNC, Stat & OR TIC testbed Before Alignment

94 UNC, Stat & OR TIC testbed Next Zoom in on This Region

95 UNC, Stat & OR TIC testbed Zoomed Fisher – Rao Alignment

96 UNC, Stat & OR TIC testbed Before Alignment

97 UNC, Stat & OR TIC testbed Next Zoom in on This Region

98 UNC, Stat & OR TIC testbed Zoomed Fisher – Rao Alignment

99 UNC, Stat & OR TIC testbed Before Fisher-Rao Alignment

100 UNC, Stat & OR TIC testbed Next Zoom in on This Region

101 UNC, Stat & OR TIC testbed Zoomed Fisher – Rao Alignment

102 UNC, Stat & OR TIC testbed Zoomed Fisher – Rao Alignment Note: Very Challenging

103 UNC, Stat & OR TIC testbed Before Alignment

104 UNC, Stat & OR TIC testbed Next Zoom in on This Region

105 UNC, Stat & OR TIC testbed Zoomed Fisher – Rao Alignment

106 UNC, Stat & OR TIC testbed Before Alignment

107 UNC, Stat & OR TIC testbed Warping Functions

108 UNC, Stat & OR References for Much More Big Picture Survey: Marron, Ramsay, Sangalli & Srivastava (2014) TIC Proteomics Example: Koch, Hoffman & Marron (2014)

109 UNC, Stat & OR Take Away Message Curve Registration is Slippery Thus, Careful Mathematics is Useful Fisher-Rao Approach: Gets the Math Right Intuitively Sensible Computable Generalizable Worth the Complication