1. On the Unique Games Conjecture Subhash Khot Georgia Inst. Of Technology. At FOCS 2005

2. NP-hard Problems Vertex Cover MAX-3SAT Bin-Packing Set Cover Clique MAX-CUT ……………..

3. Approximability : Algorithms A C-approximation algorithm computes (C > 1), for problem instance I, solution A(I) s.t. Minimization problems : A(I)  C  OPT(I) Maximization problems : A(I)  OPT(I) / C

4. Some Known Approximation Algorithms Vertex Cover 2 - approx. MAX-3SAT 8/7 - approx. Random assignment. Packing/Scheduling (1+  ) – approx.   > 0 (PTAS) Set Cover ln n approx. Clique n/log n [Boppana Halldorsson’92] Many more, ref. [Vazirani’01]

5. PCP Theorem [B’85, GMR’89, BFL’91, LFKN’92, S’92,……] [PY’91] [FGLSS’91, AS’92 ALMSS’92] Theorem : It is NP-hard to tell whether a MAX-3SAT instance is * satisfiable (i.e. OPT = 1) or * no assignment satisfies more than 99% clauses (i.e. OPT  0.99). i.e. MAX-3SAT is 1/0.99 = 1.01 hard to approximate. i.e. MAX-3SAT and MAX-SNP-complete problems [PY’91] have no PTAS.

6. Approximability : Towards Tight Hardness Results [Hastad’96] Clique n 1-  [Hastad’97] MAX-3SAT 8/7 -  [Feige’98] Set Cover (1-  ) ln n [Dinur’05] Combinatorial Proof of PCP Theorem !

7. Open Problems in Approximability –Vertex Cover (1.36 vs. 2) [DinurSafra’02] –Coloring 3-colorable graphs (5 vs. n 3/14 ) [ KhannaLinialSafra’93, BlumKarger’97 ] –Sparsest Cut (1 vs. (logn) 1/2 ) [ AroraRaoVazirani’04 ] –Max Cut (17/16 vs 1/0.878… ) [ Håstad’97, GoemansWilliamson’94] ………………………..

8. Unique Games Conjecture [Khot’02] Implies these hardness results : Vertex Cover 2-  [KR’03] Coloring 3-colorable  (1) [DMR’05] graphs (variant of UGC) MAX-CUT 1/0.878.. -  [KKMO’04] Sparsest Cut, Multi-cut [KV’05,  (1) CKKRS’04] Min-2SAT-Deletion [K’02, CKKRS’04]

9. Unique Games Conjecture Led to … [MOO’05] Majority Is Stablest Theorem [KV’05] “Negative type” metrics do not embed into L 1 with O(1) “distortion”. Optimal “integrality gap” for MAX-CUT SDP with “Triangle Inequality”.

10. Integrality Gap : Definition Given : Maximization Problem + Specific SDP relaxation. For every problem instance G, SDP(G)  OPT(G) Integrality Gap = Max G SDP(G) / OPT(G) Constructing gap instance = negative result.

11. Overview of the talk The UGC Hardness of Approximation Results I hope UGC is true Attempts to Disprove : Algorithms Connections/applications : Fourier Analysis Integrality Gaps Metric Embeddings

12. Unique Games Conjecture A maximization problem called “Unique Game” is hard to approximate. “Gap-preserving” reductions from Unique Game  Hardness results for Vertex Cover, MAX-CUT, Graph-Coloring, …..

13. Example of Unique Game OPT = max fraction of equations that can be satisfied by any assignment. x 1 + x 3 = 2 (mod k) 3 x 5 - x 2 = -1 (mod k) x 2 + 5 x 1 = 0 (mod k) UGC  For large k, it is NP-hard to tell whether OPT  99% or OPT  1%

14. 2-Prover-1-Round Game (Constraint Satisfaction Problem ) variables constraints 

15. 2-Prover-1-Round Game (Constraint Satisfaction Problem ) variables k labels Here k=4 constraints 

16. 2-Prover-1-Round Game (Constraint Satisfaction Problem ) variables k labels Here k=4 Constraints = Bipartite graphs or Relations   [k]  [k]

17. 2-Prover-1-Round Game (Constraint Satisfaction Problem ) variables k labels Here k=4 OPT(G) = 7/7 Find a labeling that satisfies max # constraints

18. Hardness of Finding OPT(G) Given a 2P1R game G, how hard is it to find OPT(G) ? PCP Theorem + Raz’s Parallel Repetition Theorem : For every , there is integer k(  ), s.t. it is NP-hard to tell whether a 2P1R game with k = k(  ) labels has OPT = 1 or OPT   In fact k = 1/poly(  )

19. Reductions from 2P1R Game Almost all known hardness results (e.g. Clique, MAX-3SAT, Set Cover, SVP, …. ) are reductions from 2P1R games. Many special cases of 2P1R games are known to be hard, e.g. Multipartite graphs, Expander graphs, Smoothness property, …. What about unique games ?

20. Unique Game = 2P1R Game with Permutations variable k labels Here k=4

21. Unique Game = 2P1R Game with Permutations variable k labels Here k=4 Permutations or matchings  : [k]  [k]

22. OPT(G) = 6/7 Find a labeling that satisfies max # constraints Unique Game = 2P1R Game with Permutations

23. Unique Games Considered before …… [Feige Lovasz’92] Parallel Repetition of UG reduces OPT(G). How hard is approximating OPT(G) for a unique game G ? Observation : Easy to decide whether OPT(G) = 1.

24. MAX-CUT is Special Case of Unique Game Vertices : Binary variables x, y, z, w, ……. Edges : Equations x + y = 1 (mod 2) [Hastad’97] NP-hard to tell whether OPT(MAX-CUT)  17/21 or OPT(MAX-CUT)  16/21

25. Unique Games Conjecture For any , , there is integer k( ,  ), s.t. it is NP-hard to tell whether a Unique Game with k = k( ,  ) labels has OPT  1-  or OPT   i.e. Gap-Unique Game (1- ,  ) is NP-hard.

26. Overview of the talk The UGC Hardness of Approximation Results I hope UGC is true Attempts to Disprove : Algorithms Connections/applications : Fourier Analysis Integrality Gaps Metric Embeddings

27. Case Study : MAX-CUT Given a graph, find a cut that maximizes fraction of edges cut. Random cut : 2-approximation. [GW’94] SDP-relaxation and rounding. min 0 <  < 1  / (arccos (1-2  ) /  ) = 1/0.878 … approximation. [KKMO’04] Assuming UGC, MAX-CUT is 1/0.878… -  hard to approximate.

28. Reduction to MAX-CUT Unique Game Graph H Completeness : OPT(UG) > 1-o(1)    - o(1) cut. Soundness : OPT(UG) < o(1)  No cut with size arccos (1-2  ) /  + o(1) Hardness factor =  / (arccos (1-2  ) /  ) - o(1) Choose best  to get 1/0.878 … (= [GW’94])

29. Reduction from Unique Game Gadget constructed via Fourier theorem + Connecting gadgets via Unique Game instance [DMR’05] “UGC reduces the analysis of the entire construction to the analysis of the gadget”. Gadget = Basic gadget ---> Bipartite gadget ---> Bipartite gadget with permutation

30. Basic Gadget A graph on {0,1} k with specific properties (e.g. cuts, vertex covers, colorability) {0,1} k k = # labels x = 011 Y = 110

31. Basic Gadget : MAX-CUT Weighted graph, total edge weight = 1. Picking random edge : x  R {0,1} k y <-- flip every co-ordinate of x with probability  (   0.8) x {0,1} k y

32. MAX-CUT Gadget : Co-ordinate Cut Along Dimension i Fraction of edges cut = Pr (x,y) [x i  y i ] =  Observation : These are the maximum cuts. x i = 0 x i = 1

33. Bipartite Gadget A graph on {0,1} k  {0,1} k (double cover of basic gadget) x = 011 y’ = 110

34. Cuts in Bipartite Gadget {0,1} k Matching co-ordinate cuts have size = 

35. Bipartite Gadget with Permutation  : [k] -> [k] Co-ordinates in second hypercube permuted via . x = 011 Y ’ = 110  (y’) = 011 Example :  = reversal of co-ordinates.

36. Reduction from Unique Game Variables k labels OPT  1 – o(1) or OPT  o(1) Permutations  : [k]  [k]

37. Instance H of MAX-CUT {0,1} k Vertices Edges  Bipartite Gadget via 

38. Proving Completeness Unique Game Graph H (Completeness) : OPT(UG) > 1-o(1)  H has  - o(1) cut.

39. Completeness : OPT(UG)  1-o(1) label = 2 label = 1 label = 3 label = 1 label = 3 label = 2 Labels = [1,2,3]

40. Completeness : OPT(UG)  1-o(1) {0,1} k Vertices Edges Hypercubes are cut along dimensions = labels. MAX-CUT   - o(1) 

41. Proving Soundness Unique Game Graph H (Soundness) : OPT(UG) < o(1)  H has no cut of size arccos (1-2  ) /  + o(1)

42. MAX-CUT Gadget Cuts = Boolean functions f : {0,1} k  {0,1} Compare boolean functions * that depend only on single co-ordinate vs * where every co-ordinate has negligible “influence” (i.e. “non-junta” functions) {0,1} k x y f(x 1 x 2 …….. x k ) = x i f(x 1 x 2 …….. x k ) = MAJORITY Influence (i, f) = Pr x [ f(x)  f(x+e i ) ]

43. Gadget : “Non-junta” Cuts How large can non-junta cuts be ? i.e. cuts with all influences negligible ? Random Cut : ½ Majority Cut : arccos (1-2  ) /  > ½ [MOO’05] Majority Is Stablest (Best) Any cut slightly better than Majority Cut must have “influential” co-ordinate.

44. Non-junta Cuts in Bipartite Gadget [MOO’05] Any “special” cut with value arccos (1-2  ) /  +  must define a matching pair of influential co-ordinates. {0,1} k

45. Non-junta Cuts in Bipartite Gadget {0,1} k f : {0,1} k --> {0, 1} g : {0,1} k --> {0, 1}  i Infl (i, f), Infl (i, g) >  (1) cut > arccos (1-2  ) /  +  

46. Instance H of MAX-CUT {0,1} k Vertices Edges  Bipartite Gadget via 

47. Proving Soundness Assume arccos (1-2  ) /  +  cut exists. On  /2 fraction of constraints, the bipartite gadget has arccos (1-2  ) /  +  /2 cut.  matching pair of labels on this constraint. This is impossible since OPT(UG) = o(1). Done !

48. Other Hardness Results Vertex Cover Friedgut’s Theorem Every boolean function with low “average sensitivity” is a junta. Sparsest Cut, Min-2SAT Deletion KahnKalaiLinial Every balanced boolean function has a co-ordinate with influence log n/n. Bourgain’s Theorem (inspired by Hastad-Sudan’s 2-bit Long Code test) Every boolean function with low “noise sensitivity” is a junta. Coloring 3-Colorable [MOO’05] inspired. Graphs

49. Basic Paradigm by [BGS’95, Hastad’97]  Hardness results for Clique, MAX-3SAT, ……. Instead of Unique Games, use reduction from general 2P1R Games (PCP Theorem + Raz). Hypercube = Bits in the Long Code [Bellare Goldreich Sudan’95] PCPs with 3 or more queries (testing Long Code). Not enough to construct 2-query PCPs.

50. Why UGC and not 2P1R Games? Power in simplicity. “Obvious” way of encoding a permutation constraint. Basic Gadget ----> Bipartite Gadget with permutation.

51. Overview of the talk The UGC Hardness of Approximation Results I hope UGC is true Attempts to Disprove : Algorithms Connections/applications : Fourier Analysis Integrality Gaps Metric Embeddings

52. I Hope UGC is True Implies all the “right” hardness results in a unifying way. Neat applications of Fourier theorems [Bourgain’02, KKL’88, Friedgut’98, MOO’05] Surprising application to theory of metric embeddings and SDP-relaxations [KV’05]. Mere coincidence ?

53. Supporting Evidence [Feige Reichman’04] Gap-Unique Game (C ,  ) is NP-hard. i.e. For every constant C, there is  s.t. it is NP-hard to tell if a UG has OPT > C  or OPT < . However C  --> 0 as  --> 0.

54. Supporting Evidence [Khot Vishnoi’05] SDP relaxation for Unique Game has integrality gap (1- ,  ).

55. Overview of the talk The UGC Hardness of Approximation Results I hope UGC is true Attempts to Disprove : Algorithms Connections/applications : Fourier Analysis Integrality Gaps Metric Embeddings

56. Disproving UGC means.. For small enough (constant) , given a UG with optimum 1- , algorithm that finds a labeling satisfying (say) 50% constraints.

57. Algorithmic Results Algorithm that finds a labeling satisfying f( , k, n) fraction of constraints. [Khot’02] 1-  1/5 k 2 [Trevisan’05] 1-  1/3 log 1/3 n [Gupta Talwar’05] 1-  log n [CMM’05] 1/k , 1-  1/2 log 1/2 k None of these disproves UGC.

58. Quadratic Integer Program For Unique Game [Feige Lovasz’92] variable k labels  : [k]  [k] u 1, u 2, …, u k  {0,1} v 1, v 2, …, v k  {0,1} u v v i = 1 if Label(v) = i = 0 otherwise

59. Quadratic Program for Unique Games Constraints on edge-set E. Maximize   u i v π(i) (u, v)  E i=1,2,..,k  u  i  [k], u i  {0,1}  u  u i 2 = 1 i  u  i ≠ j, u i u j = 0

60. SDP Relaxation for Unique Games Maximize    u i, v π(i)  (u, v)  E i=1,2,..,k  u  i  [k], u i is a vector.  u  || u i || 2  = 1 i=1,2,..,k  u  i≠j  [k],  u i, u j  = 0

61. [Feige Lovasz’92] OPT(G)  SDP(G)  1. If OPT(G) < 1, then SDP(G) < 1. SDP(G m ) = (SDP(G)) m Parallel Repetition Theorem for UG : OPT(G) < 1  OPT(G m )  0

62. [Khot’02] Rounding Algorithm u1u1 ukuk u2u2 vkvk v2v2 v1v1 r r Label(u) = 2, Label(v) = 2 Pr [ Label(u) = Label(v) ] > 1 -  1/5 k 2  Labeling satisfies 1 -  1/5 k 2 fraction of constraints in expected sense. Random r u v

63. [CMM’05] Algorithm Labeling that satisfies 1/k  fraction of constraints. ( Optimal [KV’05]) vkvk v2v2 v1v1 r u1u1 ukuk u2u2 r All i s.t. u i is “close” to r are taken as candidate labels to u. Pick one of them at random.

64. [Trevisan’05] Algorithm Given a unique game with optimum 1- 1/log n, algorithm finds a labeling that satisfies 50% of constraints. Limit on hardness factors achievable via UGC (e.g. loglog n for Sparsest Cut).

65. [Trevisan’05] Algorithm [Leighton Rao’88] Delete a few constraints and remaining graph has connected components of low diameter. Variables and constraints

66. [Trevisan’05] Algorithm A good algorithm for graphs with low diameter.

67. Overview of the talk The UGC Hardness of Approximation Results I hope UGC is true Attempts to Disprove : Algorithms Connections/applications : Fourier Analysis Integrality Gaps Metric Embeddings

68. Already Covered Let’s move on ….

69. Overview of the talk The UGC Hardness of Approximation Results I hope UGC is true Attempts to Disprove : Algorithms Connections/applications : Fourier Analysis Integrality Gaps Metric Embeddings

70. [KV’05] Integrality Gaps for SDP-relaxations MAX-CUT Sparsest Cut Unique Game Gaps hold for SDPs with “Triangle Inequality”.

71. Integer Program for MAX-CUT Given G(V,E) Maximize ¼  |v i - v j | 2 (i, j)  E  i, v i  {-1,1} Triangle Inequality (Optional) :  i, j, k, |v i - v j | 2 + |v j - v k | 2  |v i - v k | 2

72. Goemans-Williamson’s SDP Relaxation for MAX-CUT Maximize ¼  || v i - v j || 2 (i, j)  E  i, v i  R n, || v i || = 1 Triangle Inequality (Optional) :  i, j, k, || v i - v j || 2 + || v j - v k || 2  || v i - v k || 2

73. Integrality Gap for MAX-CUT [Goemans Williamson’94] Integrality gap  1/0.878.. [Karloff’99] [Feige Schetchman ’01] Integrality gap  1/0.878.. -  SDP solution does not satisfy Triangle Inequality. Does Triangle Inequality make the SDP tighter ? NO if Unique Games Conj. is true !

74. Integrality Gap for Unique Games SDP Unique Game G with OPT(G) = o(1) SDP(G) = 1-o(1) Orthonormal Bases for R k u 1, u 2, …, u k v 1, v 2, …, v k variables k labels Matchings [k]  [k] u v

75. Integrality Gap for MAX-CUT with Triangle Inequality {-1,1} k u 1, u 2, …, u k  u 1  u 2  u 3 ………  u k-1  u k PCP Reduction OPT(G) = o(1) No large cut Good SDP solution

76. Overview of the talk The UGC Hardness of Approximation Results I hope UGC is true Attempts to Disprove : Algorithms Connections/applications : Fourier Analysis Integrality Gaps Metric Embeddings

77. Metrics and Embeddings Metric is a distance function on [n] such that d(i, j) + d(j, k)  d(i, k). Metric d embeds into metric  with distortion   1 if  i, j d(i, j)   (i, j)   d(i, j).

78. Negative Type Metrics Given a set of vectors satisfying Triangle Inequality :  i, j, k, || v i - v j || 2 + || v j - v k || 2  || v i - v k || 2 d(i, j) = || v i - v j || 2 defines a metric. These are called “negative type metrics”. L 1  NEG  METRICS

79. NEG vs L 1 Question [Goemans, Linial’ 95] Conjecture : NEG metrics embed into L 1 with O(1) distortion. Sparsest Cut O(1) Integrality Gap O(1) Approximation [Linial London Rabinovich’94] [Aumann Rabani’98] Unique Games Conjecture [Chawla Krauthgamer Kumar Rabani Sivakumar ’05] [KV’05]  (1) hardness result

80. NEG vs L 1 Lower Bound  ( loglog n) integrality gap for Sparsest Cut SDP. [KhotVishnoi’05, KrauthgamerRabani’05]  A negative type metric that needs distortion  ( loglog n) to embed into L 1.

81. Open Problems (Dis)Prove Unique Games Conjecture. Prove hardness results bypassing UGC. NEG vs L 1, Close the gap.  (log log n) vs  (  log n loglog n) [Arora Lee Naor’04]

82. Open Problems Prove hardness of Min-Deletion version of Unique Games. (log n approx. [GT’05]) Integrality gaps with “k-gonal” inequalities. Is hypercube (Long Code) necessary ?

83. Open Problems More hardness results, integrality gaps, embedding lower bounds, Fourier Analysis, …… [Samorodnitsky Trevisan’05] “Gowers Uniformity, Influence of Variables, and PCPs”. UGC  Boolean k-CSP is hard to approximate within 2 k- log k Independent Set on degree D graphs is hard to approximate within D/poly(log D).

84. Open Problems in Approximability Traveling Salesperson Steiner Tree Max Acyclic Subgraph, Feedback Arc Set Bin-packing (additive approximation) …………………… Recent progress on Edge Disjoint Paths Network Congestion Shortest Vector Problem Asymmetric k-center (log * n) Group Steiner Tree (log 2 n) Hypergraph Vertex Cover ………………

85. Linear Unique Games System of linear equations mod k. x 1 + x 3 = 2 3 x 5 - x 2 = -1 x 2 + 5 x 1 = 0 [KKMO’04] UGC  UGC in the special case of linear equations mod k.

86. Variations of Conjecture 2-to-1 Conjecture [K’02]  -Conjecture [DMR’05]  NP-hard to color 3-colorable graphs with O(1) colors.   [k]  [k]