1. Rank-Sparsity Incoherence for Matrix Decomposition
Reading Group
Rank-Sparsity Incoherence for Matrix Decomposition
(MIT EECS: Venkat Chandrasekaran, Sujay Sanghavi, Pablo A. Parrilo, and Alan S. Willsky, in 2009)
Presenter: Zhe Chen
ECE / CMR
Tennessee Technological University
February 18, 2011

2. Outline Overview Introduction Rank-Sparsity Incoherence
Exact Decomposition Using Semidefinite Programming
Simulation Results
One-Sentence Summary
2/25/2019

3. Overview
Given a matrix formed by adding an unknown sparse matrix to an unknown low-rank matrix. The problem is to decompose the given matrix into its sparse and low-rank components.
A notion of rank-sparsity incoherence is developed.
Sufficient conditions for exact recovery are given.
When the sparse and low-rank matrices are drawn from certain natural random ensembles, the sufficient conditions for exact recovery are satisfied with high probability.
2/25/2019

4. Outline Overview Introduction Rank-Sparsity Incoherence
Exact Decomposition Using Semidefinite Programming
Simulation Results
One-Sentence Summary
2/25/2019

5. The Problem
Indeed, there are a number of scenarios in which a unique splitting of C into “low-rank” and “sparse” parts may not exist.
Conditions should be met.
2/25/2019

6. Identifiability Problems (1)
1. If the low-rank matrix is very sparse, impose certain conditions on the row/column spaces of the low-rank matrix:
For a matrix M let T(M) be the tangent space at M with respect to the variety of all matrices with rank less than or equal to rank(M)
Here is the spectral norm
being small implies that M cannot be very sparse
2/25/2019

7. Identifiability Problems (2)
2. If the sparse matrix has all its support concentrated in one row/column, impose conditions on the sparsity pattern of the sparse matrix:
For a matrix M let be the tangent space at M with respect to the variety of all matrices with number of non-zero entries less than or equal to |support(M)|
being small implies singular values are not too large
2/25/2019

8. Rank-sparsity incoherence
However, for a given matrix M, it is impossible for both quantities and to be small simultaneously.
The authors develop a notion of rank-sparsity incoherence, an uncertainty principle between the sparsity pattern of a matrix and its row/column spaces.
2/25/2019

9. Optimization Formulation
In general solving the decomposition problem is NP-hard
Convex relaxation is employed here
The following optimization formulation is proposed to recover A* and B* given C = A* + B*:
is a trade-off parameter
is the decomposed
2/25/2019

10. Conditions
This paper provides a simple deterministic condition for exact recovery.
The conditions only depend on the row/column spaces of the low-rank matrix B* and the support of the sparse matrix A*.
2/25/2019

11. Outline Overview Introduction Rank-Sparsity Incoherence
Exact Decomposition Using Semidefinite Programming
Simulation Results
One-Sentence Summary
2/25/2019

12. Tangent-Space Identifiability (1)
The algebraic variety of rank-constrained matrices is defined as:
For any matrix , the tangent space T(M) with respect to P(rank(M)) at M is the span of all matrices with either the same row-space as M or the same column-space as M.
Let
Then:
2/25/2019

13. Tangent-Space Identifiability (2)
The variety of support-constrained matrix is defined as
For any matrix , the tangent space with respect to S(|support(M)|) at M is given by
2/25/2019

14. Tangent-Space Identifiability (3)
A necessary and sufficient condition for unique recovery is:
That is, the subspaces and have a trivial intersection.
(Proof is omitted here.)
2/25/2019

15. Rank-Sparsity Uncertainty Principle
For any matrix both and cannot be simultaneously small.
Note that Proposition 1 is for different matrices, while Theorem 1 is for the same matrix.
(Proof is omitted here.)
2/25/2019

16. Outline Overview Introduction Rank-Sparsity Incoherence
Exact Decomposition Using Semidefinite Programming
Simulation Results
One-Sentence Summary
2/25/2019

17. Optimality Condition (1)
Notations
The orthogonal projection onto the space is denoted , which simply sets to zero those entries with support not inside support(A*).
The subspace orthogonal to is denoted The projection onto is denoted
The orthogonal projection onto the space is denoted
The space orthogonal to is denoted , and the corresponding projection is denoted
2/25/2019

18. Optimality Condition (2)
(Proof is omitted here.)
2/25/2019

19. Sufficient Conditions Based on and
Given matrices A* and B* with , and from Proposition 1, condition (1) of Proposition 2 is satisfied.
If a slightly stronger condition holds, there exists a dual Q that satisfies the requirements of condition (2) of Proposition 2.
(Proof is omitted here.)
2/25/2019

20. Sparse and Low-Rank Matrices with
(Proof is omitted here.)
(Proof is omitted here.)
2/25/2019

21. Sparse and Low-Rank Matrices with
(Proof is omitted here.)
This is a result with deterministic sufficient conditions on exact decomposability.
2/25/2019

22. Decomposing Random Sparse and Low-Rank Matrices (1)
Sparse and low-rank matrices drawn from certain natural random ensembles satisfy the sufficient conditions of Corollary 3 with high probability.
Random sparsity model
(Proof is omitted here.)
2/25/2019

23. Decomposing Random Sparse and Low-Rank Matrices (2)
Random orthogonal model
Consider low-rank matrices in which the singular vectors are chosen uniformly at random from the set of all partial isometries
Low-rank matrices drawn from such a model have incoherent row/column spaces.
(Proof is omitted here.)
2/25/2019

24. Decomposing Random Sparse and Low-Rank Matrices (3)
Applying these two results in conjunction with Corollary 3, we have that sparse and low-rank matrices drawn from the random sparsity model and the random orthogonal model can be uniquely decomposed with high probability.
(Proof is omitted here.)
2/25/2019

25. Outline Overview Introduction Rank-Sparsity Incoherence
Exact Decomposition Using Semidefinite Programming
Simulation Results
One-Sentence Summary
2/25/2019

26. Simulation Results (1)
2/25/2019

27. Simulation Results (2) Define:
Declare success in recovering (A*, B*) if
Exact recovery is possible for a range of
Modify (1.3) with
2/25/2019

28. Simulation Results (3)
Compute the difference between solutions for some t and as follows:
Generate a random that is 25-sparse and a random with rank = 2
If a reasonable guess for t (or ) is not available, one could solve (5.2) for a range of t and choose a solution corresponding to the “middle” range in which is stable and near zero.
2/25/2019

29. Simulation Results (4)
2/25/2019

30. Outline Overview Introduction Rank-Sparsity Incoherence
Exact Decomposition Using Semidefinite Programming
Simulation Results
One-Sentence Summary
2/25/2019

31. One-Sentence Summary
Sufficient conditions on sparse and low-rank matrices are provided so that the SDP exactly recovers such matrices.
2/25/2019

32. Thank you!
2/25/2019