2. Arora: SDP + Approx Survey Semidefinite Programming and Approximation Algorithms for NP-hard Problems: A Survey Sanjeev Arora Princeton University

3. Arora: SDP + Approx Survey NP-completeness Pragmatic Researcher “Why the fuss? I am perfectly content with approximately optimal solutions.” (e.g., cost within 10% of optimum) Bad News: NP-hard for many problems. (“PCPs”) Good news: Possible for a few problems. (“Approximation Algorithms”) Thousands of problems are NP-complete (TSP, Scheduling, Circuit layout, Machine Learning,..)

4. Arora: SDP + Approx Survey Talk Outline Defn of approximation and example SDP and its use in approximation Understanding SDPs high dimensional geometry Faster algorithms (multiplicative update rule) Limitations of SDPs: local vs global issues Connections (a) metric spaces (b) avg case complexity (c) unique games conjecture Open problems

5. Arora: SDP + Approx Survey Approximation Algorithms MAX-3SAT: Given 3-CNF formula , find assignment maximizing the number of satisfied clauses. An  -approximation algorithm is one that for every formula, produces in polynomial time an assignment that satisfies at least OPT/  clauses. (  >= 1). Good News: [KZ’97] An 8/7-approximation algorithm exists. Bad News: [Hastad’97] If P  NP then for every  > 0, an (8/7 -  )-approximation algorithm does not exist. (Similar results for many other problems…)

6. Arora: SDP + Approx Survey Good news (for me) Status of many basic problems is still unresolved: Vertex Cover Sparsest Cut and most graph partitioning problems Graph coloring Random instances of 3SAT My feeling: Interesting algorithms remain undiscovered; semidefinite programming (SDP) may be helpful. SDP = Generalization of linear programming Graph Vector Representation

7. Arora: SDP + Approx Survey Example: 2-approximation for Min Vertex Cover G= (V, E) Vertex Cover = Set of vertices that touches every edge “LP Relaxation” Claim: Value at least OPT/2 Proof: “Rounding” most Proof: On Complete Graph K n, OPT = n-1 but setting all x i = 1/2 gives feasible LP soln

8. Arora: SDP + Approx Survey General Philosophy… Interested in: NP-hard Minimization Problem Value = OPT Write tractable relaxation value= Round to get a solution of cost = Approximation ratio = Integrality gap

9. Arora: SDP + Approx Survey Main Idea in SDP: “Simulate” nonlinear programming Nonlinear program for Vertex Cover Homogenized SDP relaxation: New variable intended to stand for

10. Arora: SDP + Approx Survey How do you understand these vector programs? Ans. Interesting geometric analysis

11. Arora: SDP + Approx Survey Understanding SDPs Understanding phenomena in high-dimensional geometry computes c-approximation for c < 2 iff following is true Vertex Cover SDP Every graph in this family has an independent set of size Thm [Frankl-Rodl’87] False. Vertices: n unit vectors Edges: almost-antipodal pairs RnRn

12. Arora: SDP + Approx Survey SDP rounding: The two generations First generation: *Uses random hyperplane as in [GW]; * Edge-by-edge analysis Max-2SAT and Max-CUT [GW’94] ;Graph coloring [KMS’95]; MAX-3SAT [KZ’97]; Algorithms for Unique Games;.. Second generation: Global rounding and analysis Graph partitioning problems [ARV’04], Graph deletion and directed partitioning problems [ACMM’05], New analysis of graph coloring [ACC’06] Disproof of UGC for expanding constraints [AKKSTV’08] (Similarly, two generations of results showing limits on performance of SDPs)

13. Arora: SDP + Approx Survey 1 st Generation Rounding: Ratio 1.13.. for MAX-CUT [Goemans- Williamson’93] G = (V,E) Find that maximizes capacity. Quadratic Programming Formulation Semidefinite Relaxation [DP ’91, GW ’93]

14. Arora: SDP + Approx Survey Randomized Rounding [GW ’93] v6v6 v2v2 v3v3 v5v5 RnRn v1v1 Form a cut by partitioning v 1,v 2,...,v n around a random hyperplane. SDP OPT vjvj vivi  ij Old math rides to the rescue...

15. Arora: SDP + Approx Survey Fact 1: No rounding algorithm can produce a better solution out of this SDP [Feige-Schectman] Fact 2: If P NP then impossible to get 1.09-approximation by any efficient algorithm [Hastad’97] Fact 3: If “unique games conjecture” is true, it is impossible to get a better than 1.13-approximation.[KKMO’05] (i.e., algorithm on prev. slide is optimal) “Edges between all pairs of vectors making an angle 138 degrees.”

16. Arora: SDP + Approx Survey 2 nd Generation: for c-balanced separator G= (V, E); constant c >0 Goal: Find cut s.t. each side contains at least c fraction of nodes and minimized 1 SDP: “Triangle inequality” Angle subtended by the line joining two of them on the third is non-obtuse; “ “ condition.

17. Arora: SDP + Approx Survey Rounding algorithm for -approximation [ARV’04] 1. Pick random hyperplane 2. Remove points in “slab” of width 3. Remove any pair (i, j) that lie on opp. sides of slab but 4. Call remaining sets S, T. Do BFS from S to T according to distance 5. Output level of BFS tree with least # of edges. S T ST

18. Arora: SDP + Approx Survey Geometric fact underlying the analysis ( restatement of [ARV04] “Structure Theorem” by [AL06]) (“expander” : |  (S)|¸  (|S|) ) Vertices: unit vectors satisfying “triangle inequality” Edges: If then no graph in this family is an “expander.” Proof is delicate and difficult

19. Arora: SDP + Approx Survey Solving SDPs with m constraints takes time. Issue of Running Time m =n 3 in some of these SDPs! Next few slides: Often, can reduce running time: O(n 2 ) or O(n 3 ). [AHK’05], [AK’07] Main idea: “Primal-dual schema.”Solve to approximate optimality; using insights from the rounding algorithms. “Multiplicative Weight-Update Rule for psd matrices”

20. Arora: SDP + Approx Survey Classical MW update rule (Example: predicting the market) N “experts” on TV Can we perform as good as the best expert ? 1$ for correct prediction 0$ for incorrect Thm[Going back to Hannan, 1950s] Yes.

21. Arora: SDP + Approx Survey Weighted Majority Algorithm [LW’94] “Predict according to the weighted majority” Maintain a weight for each expert. Initially At step t, if expert i’s prediction was incorrect, Claim: Expected Payoff of our algorithm Similar algorithms discovered in a variety of areas: decision theory, learning theory (“boosting”), cryptography (“hardcore sets”), approx soln of LPs,.. (see survey [A, Hazan, Kale])

22. Arora: SDP + Approx Survey Primal-dual approach for SDP relaxations [A., Kale’07] At step t: Primal player: PSD matrix P t ; candidate primal Dual player: Let me run the rounding algorithm on P t, get a primal integer candidate and point out how pitiful it is. “Feedback” matrix M t Primal player: P t+1 = exp(-   t M t ) (Analysis uses formal analogy between real #s and symmetric matrics: [Other ingredients: flow computations, eigenvalues, dimension reduction tricks, etc.]

23. Arora: SDP + Approx Survey Implications for geometric embeddings of metric spaces x (X, d): metric space y d(x, y) f f(x) f(y) C = distortion Thm (Bourgain’85) For every X, there is f s.t. C= O(log n). Open qs since then: is it possible to achieve smaller C for concrete X, say X = ? [CGR’05,ALN05]: Yes, C possible for X = [KV06]: Cannot reduce C below

24. Arora: SDP + Approx Survey Unique Games Given: Number p, and m equations in n vars of the form: Promise: Either there is a solution that satisfies fraction of constraints or no solution satisfies even fraction. UGC [Khot’02]: Deciding which case holds is intractable. Seems to capture our current limitations of thinking about SDPs; basis of many recent “hardness” results.

25. Arora: SDP + Approx Survey Anatomy of a UGC-based hardness result Interpret as a graph Equations Variables Replace edges/vertices with hypercube-like gadgets Equations Variables Prove using harmonic analysis that near- -optimum solns correspond to good Solution to the unique game

26. Arora: SDP + Approx Survey Limitations of SDPs For many problems, we know neither an NP-hardness result (via PCPs) nor a good SDP-based approach. 2nd Generation results: Large families of LPs or SDPs don’t work [ABL’02], [ABLT’06]: “Proving integrality gaps without knowing the LP.” Much subsequent work, especially on families obtained from “lift and project” ideas) 1st generation results: Specific SDPs don’t work Can we show that known SDPs don’t work??

27. Arora: SDP + Approx Survey “Lifted” SDP relaxations Recall: SDP tries to “simulate” nonlinear programming; Variable for Why not take it to the next level? Variables for products of up to k variables. This is the main idea of Lovasz-Schrijver’91, Sherali-Adams, Lasserre etc. “SDP as a proof system”: Integrality gaps proved in 2 nd generation results.

28. Arora: SDP + Approx Survey Main issue: Local versus Global Example: [Erdos] There are graphs on n vertices that cannot be colored with 100 colors yet every subgraph on 0.01 n vertices is 3-colorable. LP relaxations or SDP relaxations concern local conditions. How well do such local conditions capture global property in question? Results for MAX-k-SAT [], [AAT’05], Vertex Cover[ABLT’06], [STT’07a+b] MAX-CUT, Vertex Cover etc.[CMM’08] “Lifted SDPs.” Connections to Proof Complexity.

29. Arora: SDP + Approx Survey Connections to Avg. Case Complexity (SDP used in reductions) Recent development: Interreducibility among some “average case” problems of interest. [Feige’01][Alekh.03] Problems like 3SAT seem difficult not only in the worst case but also “on average.” (Needs careful definition!) Theory of Avg Case complexity exists, but doesn’t usually apply to problems of practical interest. SDP is used in the reduction!

30. Arora: SDP + Approx Survey Open problems Techniques for proving lowerbounds on lifted SDPs. (difficult local-global results) New rounding algorithms Clarify nature of connection to average case complexity. Resolve UGC (recently, disproof of UGC when the constraint graph is an expander.[AKKSTV08] SDP as a proof technique---apply to open problems of circuit complexity, communication complexity etc. Looking forward to many developments THANK YOU!

31. Arora: SDP + Approx Survey SDPs and MW Updates: Primal-dual algorithm Known: MW Update rule --> Approx. solutions to LPs [PST’91, Y’95, GK’97,..etc.] [AK’07] Matrix MW update rule that uses formal analogy between psd matrices and nonnegative real #s. (Spl. Case: LPs= SDPs with 0’s on offdiagonals) [Other ingredients: flow computations, eigenvalues, dimension reduction tricks, etc.] “experts” constraints “payoffs” “slack in constraint” [Golden-Thompson]

32. Arora: SDP + Approx Survey Embeddings and Cuts Thm[LLR94, AR94]: Integrality gap for SDP for Nonuniform Sparsest Cut = Min distortion of any embedding of into Rounding algorithm of [ARV04] gives insight into structure of ; basis of new embeddings Hardness results for sparsest cut yielded insights at the heart of the embedding impossibility results.