E XACT MATRIX C OMPLETION VIA CONVEX OPTIMIZATION E MMANUEL J. C ANDES AND B ENJAMIN R ECHT M AY 2008 Presenter: Shujie Hou January, 28 th,2011 Department of Electrical and Computer Engineering Cognitive Radio Institute Tennessee Technological university

S OME A VAILABLE C ODES rank/sample_code.html#RPCA e.html

O UTLINE The problem statement Examples of impossible recovery Algorithms Main theorems Proof Experimental results Discussion

PROBLEM C ONSIDERED The problem of low-rank matrix completion: Recovery of a data matrix from a sampling of its entries. A matrix with rows and columns Only observing a number of of its entries which is much smaller than. Can the partially observed matrix be recovered and under what kind of conditions such a matrix can be exactly recovered?

E XAMPLES OF IMPOSSIBLE RECOVERY This matrix can not be recovered unless all of the entries are given. Reason: for most sampling sets, only observing all of zeros. Not all of the matrices can be completed from a sample of their entries.

E XAMPLES OF IMPOSSIBLE R ECOVERY The observation does not include samples from first row, the first component could never be guessed out. Not all the sampling set can be used to complete the matrix.

S HORT CONCLUSION One can not recover all low-rank matrices from any set of sampled entries. Can one recover most matrices from almost all sampling sets of cardinality ? The two theorems given later will tell that this is possible for most low-rank matrices under some specific conditions.

A LGORITHM Intuitively, (NP-hard problem) Alternatively, considering a heuristic optimization in which Is (1.5) reasonable or to what extent it is equivalent to rank minimization? Locations of observed entries Sum of singular values

T HE FIRST T HEOREM There is unique low-rank matrix consistent with the observed entries. The heuristic model (1.5) is equivalent to the above NP-hard formulation. Talk later

R ANDOM O RTHOGONAL M ODEL Generic low-rank matrix SVD (singular value decomposition ) of a matrix

A DEFINITION The subspace with low coherence is the special interest of this paper. Singular vectors with low coherence is “ spread out.”(not sparse) It can guarantee that the sampling set cannot really be a zero set.

T WO ASSUMPTIONS

M AIN RESULTS Theorem1.1 is a special case of theorem 1.3. If only a few matrices satisfy the conditions, it will also make the theorem 1.3 of little practical use.

T HE CONDITIONS OF THEOREM 1.3 The random orthogonal model obeys the two assumptions A0 and A1 with large probability.

T HE PROOF OF THE LEMMAS

R EVIEW OF T HEOREM 1.3

T HE PROOF The author employs the tools of subgradient (in distributions (generalized functional) and duality ( in optimization theory) and tools in asymptotic geometric analysis to prove the existence and uniqueness of the theorem (1.3). The proof is from page The details won’t be discussed here.

D UALITY (1) In optimization theory, the duality principle states that optimization problems may be viewed from either of two perspectives, the primal problem or the dual problem. (Wikipedia) Concept in constraint optimization

D UALITY (2) The largest singular value

D UALITY (3) in which Orthogonal complement

D UALITY (4) Ensure Y is the subgradient of nuclear norm at X0 Equivalent to the operator Property of injectivity

A RCHITECTURE OF THE PROOF (1) The candidate Y which vanishes on the complement of the will be the solution to the optimization model of

A RCHITECTURE OF THE PROOF (2) The candidate Y which vanishes on the complement of the will be the solution to The first part of the statement 1. Hopefully, the small Frobenius norm will indicate the small spectral norm as well. Prove the first statement

A RCHITECTURE OF THE PROOF (3) Ready to prove the second property: injectivity of Property of the orthogonal projection If is a one-to-one linear mapping, then The solution of the model:

A RCHITECTURE OF THE PROOF (4) Bernoulli model of the sampling set

A RCHITECTURE OF THE PROOF (5)

A RCHITECTURE OF THE P ROOF (6)

C ONNECTIONS TO TRACE HEURISTIC When the matrix variable is symmetric and positive semidefinite: Which is equivalent to

E XTENSIONS The matrix completion can be extended to multitask and multiclass learning problems in machine learning.

N UMERICAL EXPERIMENTS

N UMERICAL RESULTS

D ISCUSSIONS Under suitable conditions, the matrix can be completed for a small number of the sampled entries. The required number of the sample entries is on the order of.

Questions? Thank you!