1. 1 UNC, Stat & OR PCA Extensions for Data on Manifolds Fletcher (Principal Geodesic Anal.) Best fit of geodesic to data Constrained to go through geodesic mean Huckemann, Hotz & Munk (Geod. PCA) Best fit of any geodesic to data Jung, Foskey & Marron (Princ. Arc Anal.) Best fit of any circle to data (motivated by conformal maps)

2. 2 UNC, Stat & OR PCA Extensions for Data on Manifolds

3. 3 UNC, Stat & OR Landmark Based Shape Analysis Key Step: mod out Translation Scaling Rotation Result: Data Objects   points on Manifold ( ~ S 2k-4 )

4. 4 UNC, Stat & OR Principal Nested Spheres Analysis Main Goal: Extend Principal Arc Analysis (S 2 to S k ) Jung, Dryden & Marron (2012)

5. 5 UNC, Stat & OR Principal Nested Spheres Analysis Top Down Nested (small) spheres

6. 6 UNC, Stat & OR Principal Nested Spheres Analysis Main Goal: Extend Principal Arc Analysis (S 2 to S k ) Jung, Dryden & Marron (2012) Important Landmark: This Motivated Backwards PCA

7. 7 UNC, Stat & OR Principal Nested Spheres Analysis Replace usual forwards view of PCA Data  PC1 (1-d approx)  PC2 (1-d approx of Data-PC1)  PC1 U PC2 (2-d approx)  PC1 U … U PCr (r-d approx)

8. 8 UNC, Stat & OR Principal Nested Spheres Analysis With a backwards approach to PCA Data  PC1 U … U PCr (r-d approx)  PC1 U … U PC(r-1)  PC1 U PC2 (2-d approx)  PC1 (1-d approx)

9. 9 UNC, Stat & OR Principal Component Analysis Euclidean Settings: Forwards PCA = Backwards PCA (Pythagorean Theorem, ANOVA Decomposition) So Not Interesting But Very Different in Non-Euclidean Settings (Backwards is Better !?!)

10. 10 UNC, Stat & OR Principal Component Analysis

11. 11 UNC, Stat & OR How generally applicable is Backwards approach to PCA? Where is this already being done??? An Interesting Question

12. 12 UNC, Stat & OR An Interesting Question

13. 13 UNC, Stat & OR Nonnegative Matrix Factorization

14. 14 UNC, Stat & OR Standard NMF (Projections All Inside Orthant) Nonnegative Matrix Factorization

15. 15 UNC, Stat & OR Standard NMF But Note Not Nested No “Multi-scale” Analysis Possible (Scores Plot?!?) Nonnegative Matrix Factorization

16. 16 UNC, Stat & OR Improved Version:  Use Backwards PCA Idea  “Nonnegative Nested Cone Analysis” Collaborator: Lingsong Zhang (Purdue) Zhang, Marron, Lu (2013) Nonnegative Matrix Factorization

17. 17 UNC, Stat & OR Same Toy Data Set All Projections In Orthant Nonnegative Nested Cone Analysis

18. 18 UNC, Stat & OR Same Toy Data Set Rank 1 Approx. Properly Nested Nonnegative Nested Cone Analysis

19. 19 UNC, Stat & OR Chemical Spectral Data Gives Clearer View Nonnegative Nested Cone Analysis

20. 20 UNC, Stat & OR Chemical Spectral Data Rank 3 Approximation Highlights Lab Early Error Nonnegative Nested Cone Analysis

21. 21 UNC, Stat & OR 5-d Toy Example (Rainbow Colored by Peak Order) Nonnegative Nested Cone Analysis

22. 22 UNC, Stat & OR 5-d Toy Example Rank 1 NNCA Approx. Nonnegative Nested Cone Analysis

23. 23 UNC, Stat & OR 5-d Toy Example Rank 2 NNCA Approx. Nonnegative Nested Cone Analysis

24. 24 UNC, Stat & OR 5-d Toy Example Rank 2 NNCA Approx. Nonneg. Basis Elements (Not Trivial) Nonnegative Nested Cone Analysis

25. 25 UNC, Stat & OR 5-d Toy Example Rank 3 NNCA Approx. Current Research: How Many Nonneg. Basis El’ts Needed? Nonnegative Nested Cone Analysis

26. 26 UNC, Stat & OR How generally applicable is Backwards approach to PCA? Potential Application: Principal Curves Hastie & Stuetzle, (1989) (Foundation of Manifold Learning) An Interesting Question

27. 27 UNC, Stat & OR Goal: Find lower dimensional manifold that well approximates data  ISOmap Tennenbaum (2000)  Local Linear Embedding Roweis & Saul (2000) Manifold Learning

28. 28 UNC, Stat & OR 1 st Principal Curve Linear Reg’n Usual Smooth

29. 29 UNC, Stat & OR 1 st Principal Curve Linear Reg’n Proj’s Reg’n Usual Smooth

30. 30 UNC, Stat & OR 1 st Principal Curve Linear Reg’n Proj’s Reg’n Usual Smooth Princ’l Curve

31. 31 UNC, Stat & OR How generally applicable is Backwards approach to PCA? Potential Application: Principal Curves Perceived Major Challenge: How to find 2 nd Principal Curve? Backwards approach??? An Interesting Question

32. 32 UNC, Stat & OR Key Component: Principal Surfaces LeBlanc & Tibshirani (1996) An Interesting Question

33. 33 UNC, Stat & OR Key Component: Principal Surfaces LeBlanc & Tibshirani (1996) Challenge: Can have any dimensional surface, But how to nest??? An Interesting Question

34. 34 UNC, Stat & OR How generally applicable is Backwards approach to PCA? Another Potential Application: Trees as Data (early days) An Interesting Question

35. 35 UNC, Stat & OR How generally applicable is Backwards approach to PCA? An Attractive Answer An Interesting Question

36. 36 UNC, Stat & OR How generally applicable is Backwards approach to PCA? An Attractive Answer: James Damon, UNC Mathematics Geometry Singularity Theory An Interesting Question

37. 37 UNC, Stat & OR How generally applicable is Backwards approach to PCA? An Attractive Answer: James Damon, UNC Mathematics Damon and Marron (2013) An Interesting Question

38. 38 UNC, Stat & OR How generally applicable is Backwards approach to PCA? An Attractive Answer: James Damon, UNC Mathematics Key Idea: Express Backwards PCA as Nested Series of Constraints An Interesting Question

39. 39 UNC, Stat & OR General View of Backwards PCA

40. 40 UNC, Stat & OR Define Nested Spaces via Constraints E.g. SVD (Singular Value Decomposition = = Not Mean Centered PCA) (notationally very clean) General View of Backwards PCA

41. 41 UNC, Stat & OR General View of Backwards PCA

42. 42 UNC, Stat & OR General View of Backwards PCA

43. 43 UNC, Stat & OR General View of Backwards PCA

44. 44 UNC, Stat & OR General View of Backwards PCA

45. 45 UNC, Stat & OR Define Nested Spaces via Constraints Backwards PCA Reduce Using Affine Constraints General View of Backwards PCA

46. 46 UNC, Stat & OR Define Nested Spaces via Constraints Backwards PCA Principal Nested Spheres Use Affine Constraints (Planar Slices) In Ambient Space General View of Backwards PCA

47. 47 UNC, Stat & OR Define Nested Spaces via Constraints Backwards PCA Principal Nested Spheres Principal Surfaces Spline Constraint Within Previous? General View of Backwards PCA

48. 48 UNC, Stat & OR Define Nested Spaces via Constraints Backwards PCA Principal Nested Spheres Principal Surfaces Spline Constraint Within Previous? {Been Done Already???} General View of Backwards PCA

49. 49 UNC, Stat & OR Define Nested Spaces via Constraints Backwards PCA Principal Nested Spheres Principal Surfaces Other Manifold Data Spaces Sub-Manifold Constraints?? (Algebraic Geometry) General View of Backwards PCA

50. 50 UNC, Stat & OR Define Nested Spaces via Constraints Backwards PCA Principal Nested Spheres Principal Surfaces Other Manifold Data Spaces Tree Spaces Suitable Constraints??? General View of Backwards PCA

51. 51 UNC, Stat & OR New Topic Curve Registration

52. 52 UNC, Stat & OR Collaborators Anuj Srivastava (Florida State U.) Wei Wu (Florida State U.) Derek Tucker (Florida State U.) Xiaosun Lu (U. N. C.) Inge Koch (U. Adelaide) Peter Hoffmann (U. Adelaide)

53. 53 UNC, Stat & OR Context Functional Data Analysis Curves as Data Objects Toy Example:

54. 54 UNC, Stat & OR Context Functional Data Analysis Curves as Data Objects Toy Example: How Can We Understand Variation?

55. 55 UNC, Stat & OR Context Functional Data Analysis Curves as Data Objects Toy Example: How Can We Understand Variation?

56. 56 UNC, Stat & OR Context Functional Data Analysis Curves as Data Objects Toy Example: How Can We Understand Variation?

57. 57 UNC, Stat & OR Functional Data Analysis Insightful Decomposition

58. 58 UNC, Stat & OR Functional Data Analysis Insightful Decomposition Horiz’l Var’n

59. 59 UNC, Stat & OR Functional Data Analysis Insightful Decomposition Vertical Variation Horiz’l Var’n

60. 60 UNC, Stat & OR Challenge  Fairly Large Literature  Many (Diverse) Past Attempts  Limited Success (in General)  Surprisingly Slippery (even mathematical formulation)

61. 61 UNC, Stat & OR Challenge (Illustrated) Thanks to Wei Wu

62. 62 UNC, Stat & OR Challenge (Illustrated) Thanks to Wei Wu

63. 63 UNC, Stat & OR Functional Data Analysis Appropriate Mathematical Framework? Vertical Variation Horiz’l Var’n

64. 64 UNC, Stat & OR Landmark Based Shape Analysis Approach: Identify objects that are: Translations Rotations Scalings of each other Mathematics: Equivalence Relation Results in: Equivalence Classes Which become the Data Objects

65. 65 UNC, Stat & OR Landmark Based Shape Analysis Equivalence Classes become Data Objects a.k.a. “Orbits” Mathematics: Called “Quotient Space”,,,,,,

66. 66 UNC, Stat & OR Curve Registration What are the Data Objects? Vertical Variation  Horiz’l  Var’n

67. 67 UNC, Stat & OR Curve Registration

68. 68 UNC, Stat & OR Curve Registration

69. 69 UNC, Stat & OR Time Warping Intuition Elastically Stretch & Compress Axis

70. 70 UNC, Stat & OR Time Warping Intuition

71. 71 UNC, Stat & OR Time Warping Intuition

72. 72 UNC, Stat & OR Time Warping Intuition

73. 73 UNC, Stat & OR Time Warping Intuition

74. 74 UNC, Stat & OR Curve Registration

75. 75 UNC, Stat & OR Curve Registration

76. 76 UNC, Stat & OR Curve Registration

77. 77 UNC, Stat & OR Curve Registration Toy Example: Warping Functions

78. 78 UNC, Stat & OR Curve Registration Toy Example: Non-Equivalent Curves Cannot Warp Into Each Other

79. 79 UNC, Stat & OR Data Objects I Equivalence Classes of Curves (parallel to Kendall shape analysis)

80. 80 UNC, Stat & OR Data Objects I

81. 81 UNC, Stat & OR Data Objects I Equivalence Classes of Curves (Set of All Warps of Given Curve) Next Task: Find Metric on Equivalence Classes

82. 82 UNC, Stat & OR Metrics in Curve Space Find Metric on Equivalence Classes Start with Warp Invariance on Curves & Extend

83. 83 UNC, Stat & OR Metrics in Curve Space

84. 84 UNC, Stat & OR Metrics in Curve Space

85. 85 UNC, Stat & OR Metrics in Curve Space

86. 86 UNC, Stat & OR Metrics in Curve Space