1 UNC, Stat & OR PCA Extensions for Data on Manifolds Fletcher (Principal Geodesic Anal.) Best fit of geodesic to data Constrained to go through geodesic mean Huckemann, Hotz & Munk (Geod. PCA) Best fit of any geodesic to data Jung, Foskey & Marron (Princ. Arc Anal.) Best fit of any circle to data (motivated by conformal maps)

2 UNC, Stat & OR PCA Extensions for Data on Manifolds

3 UNC, Stat & OR Landmark Based Shape Analysis Key Step: mod out Translation Scaling Rotation Result: Data Objects   points on Manifold ( ~ S 2k-4 )

4 UNC, Stat & OR Principal Nested Spheres Analysis Main Goal: Extend Principal Arc Analysis (S 2 to S k ) Jung, Dryden & Marron (2012)

5 UNC, Stat & OR Principal Nested Spheres Analysis Top Down Nested (small) spheres

6 UNC, Stat & OR Principal Nested Spheres Analysis Main Goal: Extend Principal Arc Analysis (S 2 to S k ) Jung, Dryden & Marron (2012) Important Landmark: This Motivated Backwards PCA

7 UNC, Stat & OR Principal Nested Spheres Analysis Replace usual forwards view of PCA Data  PC1 (1-d approx)  PC2 (1-d approx of Data-PC1)  PC1 U PC2 (2-d approx)  PC1 U … U PCr (r-d approx)

8 UNC, Stat & OR Principal Nested Spheres Analysis With a backwards approach to PCA Data  PC1 U … U PCr (r-d approx)  PC1 U … U PC(r-1)  PC1 U PC2 (2-d approx)  PC1 (1-d approx)

9 UNC, Stat & OR Principal Component Analysis Euclidean Settings: Forwards PCA = Backwards PCA (Pythagorean Theorem, ANOVA Decomposition) So Not Interesting But Very Different in Non-Euclidean Settings (Backwards is Better !?!)

10 UNC, Stat & OR Principal Component Analysis

11 UNC, Stat & OR How generally applicable is Backwards approach to PCA? Where is this already being done??? An Interesting Question

12 UNC, Stat & OR An Interesting Question

13 UNC, Stat & OR Nonnegative Matrix Factorization

14 UNC, Stat & OR Standard NMF (Projections All Inside Orthant) Nonnegative Matrix Factorization

15 UNC, Stat & OR Standard NMF But Note Not Nested No “Multi-scale” Analysis Possible (Scores Plot?!?) Nonnegative Matrix Factorization

16 UNC, Stat & OR Improved Version:  Use Backwards PCA Idea  “Nonnegative Nested Cone Analysis” Collaborator: Lingsong Zhang (Purdue) Zhang, Marron, Lu (2013) Nonnegative Matrix Factorization

17 UNC, Stat & OR Same Toy Data Set All Projections In Orthant Nonnegative Nested Cone Analysis

18 UNC, Stat & OR Same Toy Data Set Rank 1 Approx. Properly Nested Nonnegative Nested Cone Analysis

19 UNC, Stat & OR Chemical Spectral Data Gives Clearer View Nonnegative Nested Cone Analysis

20 UNC, Stat & OR Chemical Spectral Data Rank 3 Approximation Highlights Lab Early Error Nonnegative Nested Cone Analysis

21 UNC, Stat & OR 5-d Toy Example (Rainbow Colored by Peak Order) Nonnegative Nested Cone Analysis

22 UNC, Stat & OR 5-d Toy Example Rank 1 NNCA Approx. Nonnegative Nested Cone Analysis

23 UNC, Stat & OR 5-d Toy Example Rank 2 NNCA Approx. Nonnegative Nested Cone Analysis

24 UNC, Stat & OR 5-d Toy Example Rank 2 NNCA Approx. Nonneg. Basis Elements (Not Trivial) Nonnegative Nested Cone Analysis

25 UNC, Stat & OR 5-d Toy Example Rank 3 NNCA Approx. Current Research: How Many Nonneg. Basis El’ts Needed? Nonnegative Nested Cone Analysis

26 UNC, Stat & OR How generally applicable is Backwards approach to PCA? Potential Application: Principal Curves Hastie & Stuetzle, (1989) (Foundation of Manifold Learning) An Interesting Question

27 UNC, Stat & OR Goal: Find lower dimensional manifold that well approximates data  ISOmap Tennenbaum (2000)  Local Linear Embedding Roweis & Saul (2000) Manifold Learning

28 UNC, Stat & OR 1 st Principal Curve Linear Reg’n Usual Smooth

29 UNC, Stat & OR 1 st Principal Curve Linear Reg’n Proj’s Reg’n Usual Smooth

30 UNC, Stat & OR 1 st Principal Curve Linear Reg’n Proj’s Reg’n Usual Smooth Princ’l Curve

31 UNC, Stat & OR How generally applicable is Backwards approach to PCA? Potential Application: Principal Curves Perceived Major Challenge: How to find 2 nd Principal Curve? Backwards approach??? An Interesting Question

32 UNC, Stat & OR Key Component: Principal Surfaces LeBlanc & Tibshirani (1996) An Interesting Question

33 UNC, Stat & OR Key Component: Principal Surfaces LeBlanc & Tibshirani (1996) Challenge: Can have any dimensional surface, But how to nest??? An Interesting Question

34 UNC, Stat & OR How generally applicable is Backwards approach to PCA? Another Potential Application: Trees as Data (early days) An Interesting Question

35 UNC, Stat & OR How generally applicable is Backwards approach to PCA? An Attractive Answer An Interesting Question

36 UNC, Stat & OR How generally applicable is Backwards approach to PCA? An Attractive Answer: James Damon, UNC Mathematics Geometry Singularity Theory An Interesting Question

37 UNC, Stat & OR How generally applicable is Backwards approach to PCA? An Attractive Answer: James Damon, UNC Mathematics Damon and Marron (2013) An Interesting Question

38 UNC, Stat & OR How generally applicable is Backwards approach to PCA? An Attractive Answer: James Damon, UNC Mathematics Key Idea: Express Backwards PCA as Nested Series of Constraints An Interesting Question

39 UNC, Stat & OR General View of Backwards PCA

40 UNC, Stat & OR Define Nested Spaces via Constraints E.g. SVD (Singular Value Decomposition = = Not Mean Centered PCA) (notationally very clean) General View of Backwards PCA

41 UNC, Stat & OR General View of Backwards PCA

42 UNC, Stat & OR General View of Backwards PCA

43 UNC, Stat & OR General View of Backwards PCA

44 UNC, Stat & OR General View of Backwards PCA

45 UNC, Stat & OR Define Nested Spaces via Constraints Backwards PCA Reduce Using Affine Constraints General View of Backwards PCA

46 UNC, Stat & OR Define Nested Spaces via Constraints Backwards PCA Principal Nested Spheres Use Affine Constraints (Planar Slices) In Ambient Space General View of Backwards PCA

47 UNC, Stat & OR Define Nested Spaces via Constraints Backwards PCA Principal Nested Spheres Principal Surfaces Spline Constraint Within Previous? General View of Backwards PCA

48 UNC, Stat & OR Define Nested Spaces via Constraints Backwards PCA Principal Nested Spheres Principal Surfaces Spline Constraint Within Previous? {Been Done Already???} General View of Backwards PCA

49 UNC, Stat & OR Define Nested Spaces via Constraints Backwards PCA Principal Nested Spheres Principal Surfaces Other Manifold Data Spaces Sub-Manifold Constraints?? (Algebraic Geometry) General View of Backwards PCA

50 UNC, Stat & OR Define Nested Spaces via Constraints Backwards PCA Principal Nested Spheres Principal Surfaces Other Manifold Data Spaces Tree Spaces Suitable Constraints??? General View of Backwards PCA

51 UNC, Stat & OR New Topic Curve Registration

52 UNC, Stat & OR Collaborators Anuj Srivastava (Florida State U.) Wei Wu (Florida State U.) Derek Tucker (Florida State U.) Xiaosun Lu (U. N. C.) Inge Koch (U. Adelaide) Peter Hoffmann (U. Adelaide)

53 UNC, Stat & OR Context Functional Data Analysis Curves as Data Objects Toy Example:

54 UNC, Stat & OR Context Functional Data Analysis Curves as Data Objects Toy Example: How Can We Understand Variation?

55 UNC, Stat & OR Context Functional Data Analysis Curves as Data Objects Toy Example: How Can We Understand Variation?

56 UNC, Stat & OR Context Functional Data Analysis Curves as Data Objects Toy Example: How Can We Understand Variation?

57 UNC, Stat & OR Functional Data Analysis Insightful Decomposition

58 UNC, Stat & OR Functional Data Analysis Insightful Decomposition Horiz’l Var’n

59 UNC, Stat & OR Functional Data Analysis Insightful Decomposition Vertical Variation Horiz’l Var’n

60 UNC, Stat & OR Challenge  Fairly Large Literature  Many (Diverse) Past Attempts  Limited Success (in General)  Surprisingly Slippery (even mathematical formulation)

61 UNC, Stat & OR Challenge (Illustrated) Thanks to Wei Wu

62 UNC, Stat & OR Challenge (Illustrated) Thanks to Wei Wu

63 UNC, Stat & OR Functional Data Analysis Appropriate Mathematical Framework? Vertical Variation Horiz’l Var’n

64 UNC, Stat & OR Landmark Based Shape Analysis Approach: Identify objects that are: Translations Rotations Scalings of each other Mathematics: Equivalence Relation Results in: Equivalence Classes Which become the Data Objects

65 UNC, Stat & OR Landmark Based Shape Analysis Equivalence Classes become Data Objects a.k.a. “Orbits” Mathematics: Called “Quotient Space”,,,,,,

66 UNC, Stat & OR Curve Registration What are the Data Objects? Vertical Variation  Horiz’l  Var’n

67 UNC, Stat & OR Curve Registration

68 UNC, Stat & OR Curve Registration

69 UNC, Stat & OR Time Warping Intuition Elastically Stretch & Compress Axis

70 UNC, Stat & OR Time Warping Intuition

71 UNC, Stat & OR Time Warping Intuition

72 UNC, Stat & OR Time Warping Intuition

73 UNC, Stat & OR Time Warping Intuition

74 UNC, Stat & OR Curve Registration

75 UNC, Stat & OR Curve Registration

76 UNC, Stat & OR Curve Registration

77 UNC, Stat & OR Curve Registration Toy Example: Warping Functions

78 UNC, Stat & OR Curve Registration Toy Example: Non-Equivalent Curves Cannot Warp Into Each Other

79 UNC, Stat & OR Data Objects I Equivalence Classes of Curves (parallel to Kendall shape analysis)

80 UNC, Stat & OR Data Objects I

81 UNC, Stat & OR Data Objects I Equivalence Classes of Curves (Set of All Warps of Given Curve) Next Task: Find Metric on Equivalence Classes

82 UNC, Stat & OR Metrics in Curve Space Find Metric on Equivalence Classes Start with Warp Invariance on Curves & Extend

83 UNC, Stat & OR Metrics in Curve Space

84 UNC, Stat & OR Metrics in Curve Space

85 UNC, Stat & OR Metrics in Curve Space

86 UNC, Stat & OR Metrics in Curve Space