1. Nonnegative Matrix Factorization via Rank-one Downdate
Ali Ghodsi
Department of Statistics and Actuarial Science
David R. Cheriton School of Computer Science
University of Waterloo
Joint work with
Stephen Vavasis and Michael Biggs

2. Nonnegative Matrix Factorization

4. 2 by 1965
560 by 1965
560 by 2
-2.19
-0.02
-3.19
1.02
20 by 28
20 by 28
2 by 1
2 by 1

5. Singular Value Decomposition (SVD)

6. History

7. History

8. History

9. History

10. History (Algorithms)

11. History (Algorithms)

12. First observation

13. Power method
Computes the leading singular vectors/value (or eigenvector/value) of a matrix 1
2 while not converged 3 4 5
6 end

14. Naive approach to NMF using this observation1 2 3 4
5 for all set
6 end for
Without step 5, this will simply compute the SVD (Jordan's algorithm, Camille Jordan )

15. Rank-one Downdata (R1D)

16. Objective function

17. ApproxRankOneSubmatrix(A)

18. Modified power iteration: Demo
Rank-1
submatrix
A =
Rank-1
submatrix

19. Modified power iteration: Demov:
Rank-1
submatrix
Rank-1
submatrix

20. Modified power iteration: Demov:
Rank-1
submatrix
Rank-1
submatrix

21. Modified power iteration: Demov:
Rank-1
submatrix
Rank-1
submatrix

22. Modified power iteration: Demov:
0.16
0.21
0.22
0.44
0.74
0.20
Rank-1
submatrix u:
Rank-1
submatrix

23. Modified power iteration: Demov:
0.0
0.44
0.74
0.20
Rank-1
submatrix u:
Rank-1
submatrix

24. Modified power iteration: Demov:
0.0
0.44
0.74
0.20
Rank-1
submatrix u:
Rank-1
submatrix

25. Modified power iteration: Demov:
0.0
0.44
0.74
0.20
Rank-1
submatrix u:
Rank-1
submatrix

26. Modified power iteration: Demov:
0.0
0.44
0.74
0.20
Rank-1
submatrix u:
Rank-1
submatrix

27. Modified power iteration: Demov:
0.0
0.44
0.74
0.20
Rank-1
submatrix u:
Rank-1
submatrix
Zero-out!

28. Modified power iteration: Demo
Rank-1
submatrix
Anew =

29. Rank-one Downdata (R1D)

30. A simple model for text

31. Generating a corpus in the model

32. Theorem about text

33. LSI

34. R1D

35. Theorem about images

36. Experimental results

37. LSI

38. NMF-DIV

39. R1D

40. LSI

41. NMF_DIV

42. R1D

43. Thank you!