Sparse Learning Based on L2,1-norm Xiaohong Chen 04-13-2012
Outline Review of sparse learning Efficient and robust feature selection via joint l2,1-norm minimzation Exploiting the entire feature space with sparsity for automatic image annotation Further works
Outline Review of sparse learning Efficient and robust feature selection via joint l2,1-norm minimzation Exploiting the entire feature space with sparsity for automatic image annotation Further works
Review of Sparse Learning
Some examples LeastR LeastC GlLeastR
Shortcoming of Sparse Learning The projection matrix W is optimized one by one, and their sparsity patterns are independent, so it can’t reflect the sparsity of the original features, e.g.,
Matrix norm
Outline Review of sparse learning Efficient and robust feature selection via joint l2,1-norm minimzation Exploiting the entire feature space with sparsity for automatic image annotation Further works
Efficient and robust feature selection via joint l2,1-norm minimzation
Robust Feature Selection Based on l21-norm Given training data {x1, x2,…, xn} and the associated class labels {y1,y2,…, yn} Least square regression solves the following optimizaiton problem to obtain the projection matrix W Add a regularization R(W) to the robust version of LS,
Robust Feature Selection Based on l21-norm Possible regularizations Ridge regularization Lasso regularization Penalize all c regression coefficients corresponding to a single feature as a whole
Robust Feature Selection Based on l21-norm
Robust Feature Selection Based on l21-norm Denote (14)
Robust Feature Selection Based on l21-norm Then we have (19)
The iterative algorithm to solve problem (14) Theorem1: The algorithm will monotonically decrease the objective of the problem in Eq.(14) in each iteration, and converge to the global optimum of the problem.
Proof of theorem1 u
Proof of theorem1
(1) (2) (1)+(2)
Experimental results-1
Experimental results-2
Experimental results-3
Outline Review of sparse learning Efficient and robust feature selection via joint l2,1-norm minimzation Exploiting the entire feature space with sparsity for automatic image annotation Further works
Exploiting the entire feature space with sparsity for automatic image annotation
The illustration of image annotation
The illustration of image annotation
The illustration of image annotation
Formulation The algorithm can be generalized as the following problem Applying manifold learning and semi-supervised learning to define the loss function, then obtain the optimization problem
Formulation The definition of A and B are shown in the paper. With the Lagrange technique, we have
The SFSS Algorithm
Because Wt+1 is the minimum of
Owing to The fact that And above inequality Incorporating (19) to (18), we can get: The objective of the framework is convex, so the proposed approach converges to the global optimum.
Experimental results-1 MAP: Mean Average precision
Experimental results-2
Experimental results-3
Experimental results-4
Outline Review of sparse learning Efficient and robust feature selection via joint l2,1-norm minimzation Exploiting the entire feature space with sparsity for automatic image annotation Further works
Future works-1 Incorporate sparse learning based on L21-norm into multi-view dimensionality reduction, e.g., A risk: a degenerate solution! How to avoid?
Future works-2 (2) Incorporate the space structural information of the features to preserving the continuity of the features
Reference [1]F.Nie, D.Xu, X.Cai, and C.Ding. Efficient and robust feature selection via joint l2,1-norm minimzation. NIPS 2010. [2]Z.Ma, Y.Yang, F.Nie, J.Uijlings, and N.Sebe. Exploiting the entire feature space with sparsity for automatic image annotation. Proceedings of the 19th ACM international conference on Multimedia:283-292 [3]Y.Yang,H.Shen, Z.Ma, Z.Huang, and X.Zhou. L2,1-norm regularization discriminative feature selection for unsupervised learning. [4] DBLP:Feiping Nie http://www.informatik.uni-trier.de/~ley/db/indices/a-tree/n/Nie:Feiping.html
Thanks! Q&A