1. Nonnegative Matrix Factorization via Rank-one Downdate

2. Nonnegative Matrix Factorization

4. 560 by 1965 560 by 2 2 by 1965 20 by 28 -2.19 -0.02 -3.19 1.02 2 by 1

5. Singular Value Decomposition (SVD)

6. History

7. History

8. History

9. History

10. History (Algorithms)

12. First observation

13. Power method Computes the leading singular vectors/value (or eigenvector/value) of a matrix 1 2while not converged 3 4 5 6end

14. Example Computes the leading singular vectors/value (or eigenvector/value) of a matrix 2while not converged 3 4 5 6end

15. 1 2 3 4 5for all set 6end for Naive approach to NMF using this observation Without step 5, this will simply compute the SVD ( Jordan's algorithm, Camille Jordan 1874. )

16. Second observation

17. Modified power iteration 1 2while not converged 3 4 5 6 7 8end

18. Modified power iteration: Demo Rank-1 submatrix Rank-1 submatrix A =

19. Modified power iteration: Demo Rank-1 submatrix Rank-1 submatrix 0.14 0.07 0.64 0.41 0.55 v:

20. Modified power iteration: Demo Rank-1 submatrix Rank-1 submatrix 0.0 0.0 0.64 0.41 0.55 v:

21. Modified power iteration: Demo Rank-1 submatrix Rank-1 submatrix v: 0.0 0.0 0.64 0.41 0.55

22. Modified power iteration: Demo Rank-1 submatrix Rank-1 submatrix u: 0.16 0.21 0.22 0.44 0.74 0.20 v: 0.0 0.0 0.64 0.41 0.55

23. Modified power iteration: Demo Rank-1 submatrix Rank-1 submatrix u: v: 0.0 0.0 0.64 0.41 0.55 0.0 0.44 0.74 0.20

24. Modified power iteration: Demo Rank-1 submatrix Rank-1 submatrix u: v: 0.0 0.0 0.64 0.41 0.55 0.0 0.44 0.74 0.20

25. Modified power iteration: Demo Rank-1 submatrix Rank-1 submatrix u: v: 0.0 0.0 0.60 0.28 0.59 0.0 0.44 0.74 0.20

26. Modified power iteration: Demo Rank-1 submatrix Rank-1 submatrix u: v: 0.0 0.44 0.74 0.20 0.0 0.0 0.60 0.28 0.59

27. Modified power iteration: Demo Rank-1 submatrix Rank-1 submatrix u: v: 0.0 0.44 0.74 0.20 0.0 0.0 0.60 0.28 0.59 Zero-out!

28. Modified power iteration: Demo A new = Rank-1 submatrix

29. Rank-one Downdata (R1D) 1 2 3 4 5end for

30. Objective function

31. ApproxRankOneSubmatrix(A)

33. Rank-one Downdata (R1D)

35. A simple model for text

36. Generating a corpus in the model

37. Theorem about text

38. LSI

39. R1D

40. Experimental results Information retrieval task –TDT corpus (pilot study, v. 1.3, 1997): news articles Identified topics (first few columns of ): Topic: president, clinton, house, white Topic: bosnian, serbs, bosnia, serb, nato, sarajevo, air, bihac Topic: haiti, military, aristide, haitian, troops, port, invasion, … Topic: simpson, defense, judge, case, jury, trial, angeles, los, court, … Topic: bill, today, senate, republicans, house, congress, republican, …

41. Experimental results For topic: “OJ Simpson trial” (simpson, defense, judge, case, jury, trial) Document 1 Allen Simpson Judge Defence Marcus Court Witness Testify Nicole gloves Document 2 Simpson Judge Defense Statements Opening Jury Legal Yesterday Prosecution Marc los Document 3 Simpson Gloves Case Prosecution Defense Roger Put Fit Today Problems jury Document 4 Simpson Kaelin Police Defense Chicago Testimony Night Knapsack Cnn Blood Witness

42. Theorem about images

43. Experimental results

44. LSI

45. NMF-DIV

46. R1D

47. LSI

48. NMF_DIV

49. NMF_SC

50. R1D