Download presentation
Presentation is loading. Please wait.
Published byBrandon Harper Modified over 9 years ago
1
IEEE TRANSSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
Graph Embedding and Extensions: A General Framework for Dimensionality Reduction IEEE TRANSSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE Shuicheng Yan, Dong Xu, Benyu Zhang, Hong-Jiang Zhang, Qiang Yang, Stephen Lin Presented by meconin
2
Outline Introduction Graph Embedding (GE)
Marginal Fisher Analysis (MFA) Experiments Conclusion and Future Work
3
Introduction Dimensionality Reduction Linear
PCA, LDA, are the two most popular due to simplicity and effectiveness LPP, preserves local relationships in the data set, and uncovers its essential manifold structure
4
Introduction Dimensionality Reduction
For nonlinear methods, ISOMAP, LLE, Laplacian Eigenmap are three algorithms have been developed recently Kernel trick: linear methods → nonlinear ones performing linear operations on higher or even infinite dimensional by kernel mapping function
5
Introduction Dimensionality Reduction Tensor based algorithms
2DPCA, 2DLDA, DATER
6
Introduction Graph Embedding is a general framework for dimensionality reduction With it’s linearization, kernelization, and tensorization, we have a unified view for understanding DR algorithms The above-mentioned algorithms can all be reformulated with in it
7
Introduction This paper show that GE can be used as a platform for developing new DR algorithms Marginal Fisher Analysis (MFA) Overcome the limitations of LDA
8
Introduction LDA (Linear Discriminant Analysis)
Find the linear combination of features best separate classes of objects Number of available projection directions is lower than class number Based upon interclass and intraclass scatters, optimal only when the data of each class is approximately Gaussian distributed
9
Introduction MFA advantage: (compare with LDA)
The number of available projection directions is much larger No assumption on the data distribution, more general for discriminant analysis The interclass margin can better characterize the separability of different classes
10
Graph Embedding For classification problem, the sample set is represented as a matrix X = [x1, x2, …, xN], xi Rm In practice, the feature dimension m is often very high, thus it’s necessary to transform the data to a low-dimensional one yi = F(xi), for all i
11
Graph Embedding
12
Graph Embedding Different motivations of DR algorithms, their objectives are similar – to derive lower dimensional representation Can we reformulate them within a unifying framework? Whether the framework assists design new algorithms?
13
Graph Embedding Give a possible answer
Represent each vertex of a graph as a low-dimensional vector that preserves similarities between the vertex pairs The similarity matrix of the graph characterizes certain statistical or geometric properties of the data set
14
Graph Embedding G = { X, W } be an undirected weighted graph with vertex set X and similarity matrix W RNN The diagonal matrix D and the Laplacian matrix L of a graph G are defined as L = D W, Dii = , i
15
Graph Embedding Graph embedding of G is an algorithm to find low-dimensional vector representations relationships among the vertices of G B is the constraint matrix, and d is a constant, for avoid trivial solution
16
Graph Embedding For larger similarity between samples xi and xj, the distance between yi and yj should be smaller to minimize the objective function To offer mappings for data points throughout the entire feature space Linearization, Kernelization, Tensorization
17
Graph Embedding Linearization Assuming y = XTw
Kernelization : x F, assuming
18
Graph Embedding The solutions are obtained by solving the generalized eigenvalue decomposition problem F. Chung, “Spectral Graph Theory,” Regional Conf. Series in Math.,no. 92, 1997
19
Graph Embedding Tensor
the extracted feature from an object may contain higher-order structure Ex: an image is a second-order tensor sequential data such as video sequences is a third-order tensor
20
Graph Embedding Tensor
In n dimensional space, nr directions, r is the rank(order) of a tensor For tensor A, B Rm1m2…mn the inner product
21
Graph Embedding Tensor For a matrix U Rmkm’k, B = A k U
22
Graph Embedding The objective funtion:
In many case, there is no closed-form solution, but we can obtain the local optimum by fixing the projection vector
23
General Framework for DR
The differences of DR algorithms: the computation of the similarity matrix of the graph the selection of the constraint matrix
24
General Framework for DR
25
General Framework for DR
PCA seeks projection directions with maximal variances it finds and removes the projection direction with minimal variance
26
General Framework for DR
KPCA applies the kernel trick on PCA, hence it is a kernelization of graph embedding 2DPCA is a simplified second-order tensorization of PCA and only optimizes one projection direction
27
General Framework for DR
LDA searches for the directions that are most effective for discrimination by minimizing the ratio between the intraclass and interclass scatters
28
General Framework for DR
LDA
29
General Framework for DR
LDA follows the linearization of graph embedding the intrinsic graph connects all the pairs with same class labels the weights are in inverse proportion to the sample size of the corresponding class
30
General Framework for DR
The intrinsic graph of PCA is used as the penalty graph of LDA PCA LDA
31
General Framework for DR
KDA is the kernel extension of LDA 2DLDA is the second-order tensorization of LDA DATER is the tensorization of LDA in arbitrary order
32
General Framework for DR
LLP ISOMAP LLE Laplacian Eigenmap (LE)
33
Related Works Kernel Interpretation Ham et al.
KPCA, ISOMAP, LLE, LE share a common KPCA formulation with different kernel definitions Kernel matrix v.s Laplacian matrix from similarity matrix Only unsupervised v.s more general
34
Related Works Out-of-Sample Extension Brand
Mentioned the concept of graph embedding Brand’s work can be considered as a special case of our graph embedding
35
Related Works Laplacian Eigenmap
Work with only a single graph, i.e., the intrinsic graph, and cannot be used to explain algorithms such as ISOMAP, LLE, and LDA Some works use a Gaussian function to compute the nonnegative similarity matrix
36
Marginal Fisher Analysis
37
Marginal Fisher Analysis
Intraclass compactness (intrinsic graph)
38
Marginal Fisher Analysis
Interclass separability (penalty graph)
39
The first step of MFA
40
The second step of MFA
41
Marginal Fisher Analysis
Intraclass compactness (intrinsic graph)
42
Marginal Fisher Analysis
Interclass separability (penalty graph)
43
The third step of MFA
44
First of Four steps of MFA
45
LDA v.s MFA The available projection directions are much greater than that of LDA There is no assumption on the data distribution of each class The interclass margin in MFA can better characterize the separability of different classes than the interclass variance in LDA
46
Kernel MFA The distance between two samples
For a new data point x, its projection to the derived optimal direction
47
Tensor MFA
48
Experiments Face Recognition XM2VTS, CMU PIE, ORL A Non-Gaussian Case
49
Experiments XM2VTS, PIE-1, PIE-2, ORL
50
Experiments
51
Experiments
52
Experiments
53
Experiments
54
Experiments
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.