1. Non-negative Matrix Factorization
Recent algorithms, extensions and available software
Atina Dunlap Brooks
North Carolina State University

2. Recent Algorithms
Lee & Seung’s multiplicative updates are easy to understand and to implement
Can be very slow to converge
ALS can speed things up
Convergence theory is not particularly strong
Most NMF methods do not have robust convergence, but work well in practice

3. Projected Gradient Descent Method
Chih-Jen Lin (2007)
Bound-constrained optimization
Projected Gradient

4. Projected Gradient Descent Method
Can be applied to both the multiplicative updates and the ALS solution
Generally, greatest speed was achieved with the projected gradient combined with ALS

5. Fast Non-Negative Matrix Approximation
Kim, Sra & Dhillon (2007)
Employs Newton-type methods to solve NMF
Uses curvature information vs. gradient descent approach
Provide an exact method (good accuracy, but still slow) and a very fast inexact method

6. References for Algorithm Comparisons
Algorithms and Applications for Approximate Nonnegative Matrix Factorization by Berry, Browne, Langville, Pauca & Plemmons (2006)
Optimality, Computation, and Interpretations of Nonnegative Matrix Factorizations by Chu, Diele, Plemmons & Ragni (2004)

7. Extensions Tri-Factorization Semi-NMF Convex-NMF
Non-negative Tensor Factorization
Inferential Robust Matrix Factorization

8. Orthogonal Tri-factorization
Ding, Li, Peng & Park (2006)
Requiring orthogonality introduces uniqueness and improves clustering interpretations
A = WSH, where WTW=I and HTH=I
W gives row clusters while H gives column clusters

9. Semi-NMF Ding, Li & Jordan (2006)
Allows A and W to contain negative values, but H is restricted to non-negative
Provides more flexibility (negative entries) and a clustering which is usually better than k-means

10. Convex-NMF Ding, Li & Jordan (2006)
Restricts W to be convex combinations of the columns of A
Ensures meaningful cluster centroids
W and H tend to be sparse

11. Non-Negative Tensor Factorization
Uses n-way arrays instead of the 2-dimensional arrays used by NMF
Presentations during the workshop by Michael Berry and Bob Plemmons

12. Inferential Robust Matrix Factorization
Fogel, Young, Hawkins & Ledirac (2007)
Uses the same method for robustness as Liu et al. (2003) for robust SVD
Paul Fogel will be presenting on an application

13. Software - Matlab Matlab Code Patrik Hoyer Chih-Jen Lin
Includes Lee & Seung’s multiplicative updates and Hoyer’s sparseness
Chih-Jen Lin
Includes projected gradient descent applied to multiplicative updates and ALS

14. Software C code – nnmf() JMP script - irMF Simon Sheperd Paul Fogel
Very fast algorithm (as of 2004)
JMP script - irMF
Paul Fogel
Inferential Robust Matrix Factorization

15. Thank You