1. Recent Progress in Approximability

2. Administrivia Most agreeable times: Monday 2:30-4:00 Wednesday 4:00-5:30 Thursday 4:00-5:30 Friday 1:00-2:30 Please Fill Up Survey: http://www.surveymonkey.com/s/9TSVQM7 http://www.surveymonkey.com/s/9TSVQM7 Evaluation: 6-8 short homeworks and class participation.

3. Max Cut 10 15 3 7 1 1 Max CUT Input: A weighted graph G Find: A Cut with maximum number/weight of crossing edges Fraction of crossing edges MaxCut is NP-complete (Karp’s original list of 21 NP- complete problems (1971)

4. An algorithm A is an α -approximation for a problem if for every instance I, A (I) ≥ α ∙ OPT(I) --Vast Literature-- Approximation Algorithms

5. Max Cut 10 15 3 7 1 1 Max CUT Input: A weighted graph G Find: A Cut with maximum number/weight of crossing edges Trivial ½ Approximation Assign each vertex randomly to left or right side of the cut Analysis For every edge e, Probability[edge is cut] = ½ Fraction of edges cut = ½ Optimum MaxCut < 1 So, Solution returned = ½ > ½ *Optimum MaxCut Till 1994, this was the state of the art. Many linear programming techniques were known to NOT get any better approximation.

6. The Tools Till 1994, A majority of approximation algorithms directly or indirectly relied on Linear Programming. In 1994, Semidefinite Programming based algorithm for Max Cut [Goemans-Williamson] Semidefinite Programming - A generalization of Linear Programming. Semidefinite Programming is the one of the most powerful tools in approximation algorithms.

7. Semidefinite Program Variables : v 1, v 2 … v n | v i | 2 = 1 Maximize Max Cut SDP Quadratic Program Variables : x 1, x 2 … x n x i = 1 or -1 Maximize 10 15 3 7 1 1 1 1 1 Relax all the x i to be unit vectors instead of {1,-1}. All products are replaced by inner products of vectors 1

8. Semidefinite Program: [Goemans-Williamson 94] Embedd the graph on the N - dimensional unit ball, Maximizing ¼ ( Average Squared Length of the edges ) Semidefinite Program [Goemans-Williamson 94] Variables : v 1, v 2 … v n |v i | 2 = 1 Maximize MaxCut 10 15 3 7 1 1 1 1 1 1 Max Cut Problem Given a graph G, Find a cut that maximizes the number of crossing edges v1v1 v2v2 v3v3 v4v4 v5v5

9. MaxCut Rounding v1v1 v2v2 v3v3 v4v4 v5v5 Cut the sphere by a random hyperplane, and output the induced graph cut. -A 0.878 approximation for the problem. [Goemans-Williamson]

10. Analysis v1v1 v2v2 v3v3 v4v4 v5v5 SDP Optiumum 10 15 3 7 1 1 Optimal MaxCut v1v1 v2v2 v3v3 v4v4 v5v5 Algorithm’s Output 01 Rounding Ratio > 0.878 Integrality Gap Algorithm Output > 0.878 X SDP Optimum > 0.878 X Optimum MaxCut

11. minimum over all instances = value of rounded solution value of SDP solution rounding – ratio A (approximation ratio) ≤ integrality gap = value of optimal solution value of SDP solution minimum over all instances For any rounding algorithm A, and a SDP relaxation ¦ v1v1 v2v2 v3v3 v4v4 v5v5 SDP Optiumum 10 15 3 7 1 1 Optimal MaxCut v1v1 v2v2 v3v3 v4v4 v5v5 Algorithm’s Output 01 Rounding Ratio > 0.878 Integrality Gap = “algorithm achieves the gap’’

12. Inapproximability Is 0.878 the best possible approximation ratio for MaxCut? Satisfiable Unsatisfiable MaxCut value = K MaxCut value < K 10 15 3 7 1 1 1 1 1 3-SAT Instance Polynomial time reduction

13. What we need.. (Completeness) Satisfiable (Soundness) Unsatisfiable MaxCut value = K MaxCut value < 0.9K 10 15 3 7 1 1 1 1 1 3-SAT Instance Polynomial time reduction If we had a polytime 0.95 approximation algorithm for MaxCut  A polytime algorithm for 3-SAT

14. A probabilistically checkable proof (PCP) Goal: Alice wants to prove to Bob that 3-SAT instance A is satisfiable 3-SAT Instance A Alex Bob (polytime machine) Satisfying assignment

15. A probabilistically checkable proof (PCP) Goal: Alice wants to prove to Bob that 3-SAT instance A is satisfiable 3-SAT Instance A Alex Bob (polytime machine) 10 15 3 7 1 1 1 1 1 Polynomial time reduction 3-SAT Instance A 10 15 3 7 1 1 1 1 1 Polynomial time reduction Probabilistically Checkable Proof A cut of value > 0.9 Verifier (Bob): Sample a random edge in graph, Accept if edge is cut. Prob[Bob Accepts] = Value of the Cut

16. Suppose, (Completeness) Satisfiable (Soundness) Unsatisfiable MaxCut value = 0.99 MaxCut value < 0.9 10 15 3 7 1 1 1 1 1 3-SAT Instance Polynomial time reduction Completeness: There exists a ``proof” that Bob accepts with probability 0.99 Soundness: No matter what Alex does, Bob accepts with probability < 0.9 Bob reads only 2 bits of the proof!!

17. Analogy to Math Proofs Could you check the proof of a theorem with any reasonable confidence by reading only 3 bits of the proof??? Guess: Probably Not.. Max-SNP complexity class was defined, because it was believable that we will never be able to get a Gap Reduction aka Probabilistically Checkable Proof for NP.

18. PCP Theorem: [Arora-Lund-Motwani-Sudan-Szegedy 1991] Max-3-SAT is NP-hard to approximate better than 1- 10^{-100}. Corollary: Max-Cut is NP-hard to approximate better than 1- 10^{-200}. Long and very difficult proof, simplified over the years.. (*Check out History of PCP Theorem: http://www.cs.washington.edu/education/courses/cse533/05au/pcp-history.pdf) Completely new proof by Irit Dinur in 2005.

19. Hastad’s 3-Query PCP [ Håstad STOC97] For any ε > 0, NP has a 3-query probabilistically checkable proof system such that: Completeness = (1 – ε) Soundness = 1/2 + ε Verifier reads only 3-bits, and checks a linear equation on them! X i + X j = X k + c (mod p) Alternately,

20. Hastad’s 3-Query PCP [1997] For any ε > 0, given a set of linear equations modulo 2, it is NP-hard to distinguish between: (1 – ε) – fraction of the equations can be satisfied. 1/2 + ε – fraction of the equations can be satisfied. All equations are of the form X i + X j = X k + c (mod p) By Very Clever Gadget reductions, [Sudan-Sorkin-Trevisan-Williamson] MaxCut is NP-hard to approximate beyond 0.94.

21. ALGORITHMS [Charikar-Makarychev-Makarychev 06] [Goemans-Williamson] [Charikar-Wirth] [Lewin-Livnat-Zwick] [Charikar-Makarychev-Makarychev 07] [Hast] [Charikar-Makarychev-Makarychev 07] [Frieze-Jerrum] [Karloff-Zwick] [Zwick SODA 98] [Zwick STOC 98] [Zwick 99] [Halperin-Zwick 01] [Goemans-Williamson 01] [Goemans 01] [Feige-Goemans] [Matuura-Matsui] [Trevisan-Sudan-Sorkin-Williamson] Approximability of CSPs Gap for MaxCUT Algorithm = 0.878 Hardness = 0.941 MAX CUT MAX 2-SAT MAX 3-SAT MAX 4-SAT MAX DI CUT MAX k-CUT Unique Games MAX k-CSP MAX Horn SAT MAX 3 DI-CUT MAX E2 LIN3 MAX 3-MAJ MAX 3-CSP MAX 3-AND 01 NP HARD

22. Given linear equations of the form: X i – X k = c ik mod p Satisfy maximum number of equations. x-y = 11 (mod 17) x-z = 13 (mod 17) … …. z-w = 15(mod 17) Unique Games Conjecture [Khot 02] [KKMO] For every ε> 0, for large enough p, Given : 1-ε (99%) satisfiable system, NP-hard to satisfy ε (1%) fraction of equations. Towards bridging this gap, In 2002, Subhash Khot introduced the Unique Games Conjecture

23. A notorious open problem. Hardness Results: No constant factor approximation for unique games. [Feige- Reichman] Algorithm On (1-Є) satisfiable instances [Khot 02] [Trevisan] [Gupta-Talwar] 1 – O(ε logn) [Charikar-Makarychev-Makarychev] [Chlamtac-Makarychev-Makarychev] [Arora-Khot-Kolla-Steurer-Tulsiani-Vishnoi]

24. Assuming UGC UGC Hardness Results [Khot-Kindler-Mossel-O’donnell] [Austrin 06] [Austrin 07] [Khot-Odonnell] [Odonnell-Wu] [Samorodnitsky-Trevisan] NP HARDUGC HARD 01 MAX CUT MAX 2-SAT MAX 3-SAT MAX 4-SAT MAX DI CUT MAX k-CUT Unique Games MAX k-CSP MAX Horn SAT MAX 3 DI-CUT MAX E2 LIN3 MAX 3-MAJ MAX 3-CSP MAX 3-AND For MaxCut, Max-2-SAT, Unique Games based hardness = approximation obtained by Semidefinite programming!

25. The Connection MAX CUT MAX 2-SAT MAX 3-SAT MAX 4-SAT MAX DI CUT MAX k-CUT Unique Games MAX k-CSP MAX Horn SAT MAX 3 DI-CUT MAX E2 LIN3 MAX 3-MAJ MAX 3-CSP MAX 3-AND 01 UGC Hard GENERIC ALGORITHM Theorem: Assuming Unique Games Conjecture, For every CSP, “the simplest semidefinite programs give the best approximation computable efficiently.”

26. Constraint Satisfaction Problems [Raghavendra`08][Austrin-Mossel] M AX C UT [Khot-Kindler-Mossel-ODonnell][Odonnell-Wu] M AX 2S AT [Austrin07][Austrin08] Ordering CSPs [Charikar-Guruswami-Manokaran-Raghavendra-Hastad`08] M AX A CYCLIC S UBGRAPH, B ETWEENESS Grothendieck Problems [Khot-Naor, Raghavendra-Steurer] Metric Labeling Problems [Manokaran-Naor-Raghavendra-Schwartz`08] M ULTIWAY C UT, 0- EXTENSION Kernel Clustering Problems [Khot-Naor`08,10] Strict Monotone CSPs [Kumar-Manokaran-Tulsiani-Vishnoi`10] V ERTEX C OVER [Khot-Regev], H YPERGRAPH V ERTEX C OVER Assuming the Unique Games Conjecture, A simple semidefinite program (Basic-SDP) yields the optimal approximation ratio for Is the conjecture true? Many many ways to disprove the conjecture! Find a better algorithm for any one of these problems.

27. The UG Barrier Constraint Satisfaction Problems Graph Labelling Problems Ordering CSPs Kernel Clustering Problems Monotone Min-One CSPs UGC HARD If UGC is true, Then Simplest SDPs give the best approximation possible. If UGC is false, Hopefully, a new algorithmic technique will arise.

28. What if UGC is false? Could existing techniques ( LPs/SDPs) disprove the UGC?

29. What if UGC is false?

30. UGC is false  New algorithms? Unique Games Constraint Satisfaction Problems [Raghavendra`08] M AX C UT, M AX 2S AT Ordering CSPs [GMR`08] M AX A CYCLIC S UBGRAPH, B ETWEENESS Grothendieck Problems [KNS`08, RS`09] Metric Labeling Problems [MNRS`08] M ULTIWAY C UT, 0- EXTENSION Kernel Clustering Problems [KN`08,10] Strict Monotone CSPs [KMTV`10] V ERTEX C OVER, H YPERGRAPH V ERTEX C OVER … Problem X UGC is false  New algorithm for Problem X Despite considerable efforts, No such reverse reduction known for any of the above problems [Feige-Kindler-Odonnell,Raz’08, BHHRRS’08]

31. Graph Expansion d-regular graph G d expansion(S) = # edges leaving S d |S| vertex set S A random neighbor of a random vertex in S is outside of S with probability expansion(S) Ф G = expansion(S) minimum |S| ≤ n/2 Conductance of Graph G Uniform Sparsest Cut Problem Given a graph G compute Ф G and find the set S achieving it. Approximation Algorithms: Cheeger’s Inequality [Alon][Alon-Milman] Given a graph G, if the normalized adjacency matrix has second eigen value λ 2 then, A log n approximation algorithm [Leighton-Rao 98-99?]. A sqrt(log n) approximation algorithm using semidefinite programming [Arora-Rao-Vazirani 2004]. Extremely well-studied, many different contexts pseudo-randomness, group theory, online routing, Markov chains, metric embeddings, …

32. A Reverse Reduction Graph (Social Network) Close-knit community Finding Small Non Expanding Sets Suppose there exists is a small community say (0.1% of the population) 99% of whose friends are within the community.. Find one such close-knit community. Theorem [R-Steurer 10] UGC is false  New algorithms to approximate expansion of small sets in graphs

33. STILL OPEN: Reverse reduction from Max Cut or Vertex Cover to Unique Games.

34. What if UGC is false? Could existing algorithmic techniques (LPs/SDPs) disprove the UGC?

35. Could LPs/SDPs disprove the UGC?

36. Question I: Could some small L INEAR P ROGRAM give a better approximation for MaxCut or Vertex Cover thereby disproving the UGC? Probably Not! [Charikar-Makarychev-Makarychev][Schoenebeck-Tulsiani] For MaxCut, for several classes of linear programs, exponential sized linear programs are necessary to even beat the trivial ½ approximation! Question II: Could some small S EMI D EFINITE P ROGRAM give a better approximation for MaxCut or Vertex Cover thereby disproving the UGC? We don’t know.

37. v1v1 v2v2 v3v3 v4v4 v5v5 Max Cut SDP: Embedd the graph on the N - dimensional unit ball, Maximizing ¼ ( Average squared length of the edges ) In the integral solution, all the vectors v i are 1,-1. Thus they satisfy additional constraints For example : (v i – v j ) 2 + (v j – v k ) 2 ≥ (v i – v k ) 2 (the triangle inequality) The Simplest Relaxation for MaxCut Does adding triangle inequalities improve approximation ratio? (and thereby disprove UGC!)

39. Until 2009: Adding a simple constraint on every 5 vectors could yield a better approximation for MaxCut,and disproves UGC! Building on the work of [Khot-Vishnoi],

41. Constraint Satisfaction Problems Max 3 SAT Find an assignment that satisfies the maximum number of clauses. Variables Finite Domain Constraints {x 1,x 2, x 3, x 4, x 5 } {0,1} Clauses Kind of constraints permitted Different CSPs

42. Deeper understanding of the UGC – why it should be true if it is. Why play this game? Connections between SDP hierarchies, Spectral Graph Theory and Graph Expansion. New algorithms based on SDP hierarchies. [Raghavendra-Tan] Improved approximation for MaxBisection using SDP hierarchies [Barak-Raghavendra-Steurer] Algorithms for 2-CSPs on low-rank graphs. New Gadgets for Hardness Reductions: [Barak-Gopalan-Hastad-Meka-Raghavendra-Steurer] A more efficient long code gadget.