Nuclear Norm Heuristic for Rank Minimization Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization Necessary and Sufficient Conditions for Success of the Nuclear Norm Heuristic for Rank Minimization Presenter: Zhen Hu

Outline Introduction From Compressed Sensing to Rank Minimization Restricted Isometry and Recovery of Low-Rank Matrices Algorithms for Nuclear Norm Minimization Necessary and Sufficient Conditions for Success of the Nuclear Norm Heuristic for Rank Minimization Discussion and Future Developments 11/21/2018

Introduction An affine rank minimization problem Minimization of the l1 norm is a well known heuristic for the cardinality minimization problem. L1 heuristic can be a priori guaranteed to yield the optimal solution. The results from the compressed sensing literature might be extended to provide guarantees about the nuclear norm heuristic for the more general rank minimization problem. 11/21/2018

From Compressed Sensing 11/21/2018

Matrix Norm Frobenius Norm Operator Norm Nuclear Norm 11/21/2018

Convex Envelopes of Rank and Cardinality Functions 11/21/2018

Additive of Rank and Nuclear Norm 11/21/2018

Nuclear Norm Minimization This problem admits the primal-dual convex formulation The primal-dual pair of semidefinite programs 11/21/2018

Restricted Isometry and Recovery of Low Rank Matrices 11/21/2018

Nearly Isometric Families 11/21/2018

11/21/2018

Main Results 11/21/2018

Main Results 11/21/2018

Algorithms for Nuclear Norm Minimization The trade-offs between computational speed and guarantees on the accuracy of the resulting solution. Algorithms for Nuclear Norm Minimization Interior Point Methods for Semidefinite Programming For small problems where a high-degree of numerical precision is required, interior point methods for semidefinite programming can be directly applied to solve affine nuclear minimization problems. Projected Subgradient Methods Low-rank Parametrization SDPLR and the Method of Multipliers 11/21/2018

Numerical Experiments 11/21/2018

11/21/2018

Necessary and Sufficient Conditions The authors present a necessary and sufficient condition for the solution of the nuclear norm heuristic to coincide with the minimum rank solution in an affine space. The condition characterizes a particular property of the null space of the linear map which defines the affine space. The authors also present a reduction of the standard Linear Matrix Inequality (LMI) constrained rank minimization problem to a rank minimization problem with only equality constraints. 11/21/2018

Moreover, the authors show that when the linear map defining the constraint set is generated by sampling its entries independently from a Gaussian distribution, the null-space characterization holds with overwhelming probability provided the dimensions of the equality constraints are of appropriate size. The authors provide numerical experiments demonstrating that even when matrix dimensions are small, the nuclear norm heuristic does indeed always recover the minimum rank solution when the number of constraints is sufficiently large. 11/21/2018

Necessary and Sufficient Conditions The main optimization problem under study is The main concern is when the optimal solution coincides with the optimal solution of 11/21/2018

11/21/2018

Theorem 3.1 The following theorem generalizes this null-space criterion to a critical property that guarantees when the nuclear norm heuristic finds the minimum rank solution of A(X)=b for all values of the vector b. A particular property of the null space of the linear map 11/21/2018

Lemma 4.1 For the proof of Theorem 3.1 11/21/2018

Theorem 3.2 Reduction to the affine case! 11/21/2018

Weak Bound Probabilistic generation of constraints satisfying null space characterization 11/21/2018

Strong Bound 11/21/2018

Performance 11/21/2018

Discussion Having illustrated the natural connections between affine rank minimization and affine cardinality minimization, we were able to draw on these parallels to determine scenarios where the nuclear norm heuristic was able to exactly solve the rank minimization problem. Future developments Factored measurements and alternative ensembles Noisy measurements and low rank approximation Incoherent ensembles and partially observed transforms Alternative numerical methods Geometric interpretations Parsimonious models and optimization 11/21/2018

Question? 11/21/2018