1. 1 High-Performance Implementation of Positive Matrix Completion for SDPs Makoto Yamashita (Tokyo Institute of Technology) Kazuhide Nakata (Tokyo Institute of Technology) 2013/10/6 INFORMS Annual Meeting 2013 The research was financially supported by the Sasakawa Sientific Research Grant from The Japan Science Society. 2013/10/6-9 Minneapolis Convention Center, Minneapolis, USA

2. Sparsity in SDPs 2013/10/6 INFORMS Annual Meeting 2013 2 Notation Only blue elements are involved in inner-product. However, we also have SDP condition.

3. Structural Sparsity in Spin-Glass SDPs Each node interacts with only its 6 neighbors Only the blue elements are involved in inner-product ⇒ Exploit this structural sparsity SDP condition ⇒ Positive Matrix Completion 2013/10/6 INFORMS Annual Meeting 2013 3

4. Idea of Matrix-Completion type Interior-Point Method 2013/10/6 INFORMS Annual Meeting 2013 4 Without Blacks Complement Blacks

5. Outline of this talk Introduction of Matrix-Completion IPM Speed-up by new factorization formula Multiple threads computation Numerical results This talk corresponds to the new version of SDPA-C (SDPA with the Completion) Available at http://sdpa.sf.net/ 2013/10/6 INFORMS Annual Meeting 2013 5

6. Standard form of SDP Primal-Dual form. 2013/10/6 INFORMS Annual Meeting 2013 6

7. Framework of IPM 2013/10/6 INFORMS Annual Meeting 2013 7

8. Keywords in Matrix-Completion How many elements are necessary? ⇒ Aggregate Sparsity Pattern How to convert into smaller matrices? ⇒ Chordal Graph & Maximal Cliques How to complete ? ⇒ The form of 2013/10/6 INFORMS Annual Meeting 2013 8

9. Aggregate Sparsity Pattern Non-zero pattern in the dual side 2013/10/6 INFORMS Annual Meeting 2013 9 Example 1 5 3 2 4 6 7 graph

10. Chordal Graph Chodal, if every cycle longer than 3 has a chord The variable matrix is decomposed by the maximal cliques. Maximal Cliques (Clique, if there is an edge between any pair of the verticies.) 2013/10/6 INFORMS Annual Meeting 2013 10 1 5 3 2 4 6 7 length4 length5 1 5 3 2 4 6 7 Chordal

11. Decomopostion of 2013/10/6 INFORMS Annual Meeting 2013 11 1 5 3 2 4 6 7 Blue: aggregate, Red: Chordal Grone et al. 1984 The entire matrix can be positive definite.

12. An example of Matrix Completion 2013/10/6 INFORMS Annual Meeting 2013 12 1 2 3 Positive Definite Non-singular & Transpose Non-singular

13. A remarkable property of the matrix completion The matrix is fully-dense, but its inverse is sparse.. 2013/10/6 INFORMS Annual Meeting 2013 13

14. The factorization of the variable matrix 2013/10/6 INFORMS Annual Meeting 2013 14 Point::

15. Schur complement matrix with the sparse matrices 2013/10/6 INFORMS Annual Meeting 2013 15

16. Outline of this talk Introduction of Matrix-Completion IPM Speed-up by new factorization formula Mutliple threads computation Numerical results 2013/10/6 INFORMS Annual Meeting 2013 16

17. Review of Matrix Factorization.. Point 2013/10/6 INFORMS Annual Meeting 2013 17

18. 2013/10/6 INFORMS Annual Meeting 2013 18

19. 2013/10/6 INFORMS Annual Meeting 2013 19

20. Speed-up by the new factorization 2013/10/6 INFORMS Annual Meeting 2013 20 SDPA-C6.2.0SDPA-C7.3.8Speed-up Schur Complement4205.70sec2094.03sec2.00x Total4729.28sec2515.50sec1.88x The computation time is shrunk, but there is still room to improve. Parallel computation by multiple threads Max-clique SDP

21. Multiple threaded Schur complement matrix 2013/10/6 INFORMS Annual Meeting 2013 21 Each column is independent from others. The thread that becomes idle computes the next column. 1 1 2 2 3 3 4 4

22. The effect of multiple threads 2013/10/6 INFORMS Annual Meeting 2013 22 SDPA-C 6.2.0(1) SDPA-C 7.3.8(1) SDPA-C 7.3.8(2) SDPA-C 7.3.8(4) Schur4205.70 sec2094.03 sec1170.37 sec 1.78 x 731.98 sec 2.86 x Total4729.28sec2515.50 sec1410.52 sec 1.78 x 889.15 sec 2.82 x Max-clique The number in （ ） is threads 5.31 times speed-up

23. New version of SDPA-C 2013/10/6 INFORMS Annual Meeting 2013 23 SDPA-C 6.2.1SDPA 7.3.8SDPA-C 7.3.8 Interior-Point MethodMatrix CompletionStandardMatrix Compeltion Sparse CholeskyOur own codeMUMPSCHOLMOD &MUMPS Multiple Threads× △ ○ Callable Library× ○○ Matlab Interface× ○○

24. Test Environments and Test Problems CPU Xeon X5365(3.0GHz), Memory 48GB, Red Hat Linux Test Problem 1 SDP relaxation of Max-Clique Problem on lattice (p,q) Test Problem 2 Spin-glass computation in quantum chemistry 2013/10/6 INFORMS Annual Meeting 2013 24

25. Max clique SDPs (p=400,q=10) 2013/10/6 INFORMS Annual Meeting 2013 25 SDPA-C 6.2.0 SDPA-C 7.3.8(4) SDPA 7.3.8(4) SeDuMi 1.3 Schur9314.341431.69262.04- ΔX734.41175.2413150.10- Total12681.161903.1326159.4061861.30 New SDPA-C is the fastest. #Clique 438, Average Size 29.89, Max Size 59

26. Spin-glass SDPs 2013/10/6 INFORMS Annual Meeting 2013 26 pn#CliquesAve SizeMax SizeSDPA7.3.8SDPA-C7.3.8 10100015525.6929411.8520.77 15337519129.97773336.40560.40 185832111828.139131522.602136.88 208000173725.7510803726.034552.10 2515625317329.50179826023.6724913.20 The unit of computation time is second. The computation cost grows up mildly in SDPA-C. The clique size is almost constant. For larger SDPs, SDPA-C is more efficient.

27. Conclusion and future works Speed-up of Matrix-Completion IPM by the new factorization formula  More effective for larger problems Speed-up by multiple threads. We should automatically select the standard IPM or Matrix-Completion IPM 2013/10/6 INFORMS Annual Meeting 2013 27