How to Round Any CSP Prasad Raghavendra University of Washington, Seattle David Steurer, Princeton University (In Principle)

Constraint Satisfaction Problem A Classic Example : Max-3-SAT Given a 3-SAT formula, Find an assignment to the variables that satisfies the maximum number of clauses. Equivalently the largest fraction of clauses

Variables : {x 1, x 2, x 3,x 4, x 5 } Constraints : 4 clauses Constraint Satisfaction Problem Instance : Set of variables. Predicates P i applied on variables Find an assignment that satisfies the largest fraction of constraints. Problem : Domain : {0,1,.. q-1} Predicates :{P 1, P 2, P 3 … P r } P i : [q] k -> {0,1} Max-3-SAT Domain : {0,1} Predicates : P 1 (x,y,z) = x ѵ y ѵ z

Theorem: [Raghavendra 08] Assuming Unique Games Conjecture, For every CSP, “a simple semidefinite program (SDP1) gives the best approximation computable efficiently.” [Raghavendra08] A generic rounding scheme for (SDP1) that is optimal for every CSP under UGC. Independent of UGC, for 2CSPs, the generic rounding scheme for (SDP1) achieves an Approximation Ratio ≥ (1- ² ) Integrality Gap of SDP.

Rounding Algorithm minimum over all instances = value of rounded solution value of SDP solution rounding – ratio A ( ¦ ) (approximation ratio) ≥ (1- ² ) integrality gap ( ¦ ) = value of optimal solution value of SDP solution minimum over all instances For any CSP ¦ and any ² >0, there exists an efficient algorithm A, Unconditionally, the algorithm A as good as all known algorithms for CSPs Very Simple : No Invariance Principle, Dictatorship Tests, Unique Games. Drawbacks Running Time(A) On CSP over alphabet size q, arity k No explicit approximation ratio

Computing Integrality Gaps Theorem: For any CSP ¦ and any ² >0, there exists an algorithm A to compute integrality gap ( ¦ ) within an accuracy ² Running Time(A) On CSP over alphabet size q, arity k

Previous Work SDP ALGORITHMS [Charikar-Makarychev-Makarychev 06] MaxCut [Goemans-Williamson] [Charikar-Wirth] [Lewin-Livnat-Zwick] [Charikar-Makarychev-Makarychev 07] [Hast] [Charikar-Makarychev-Makarychev 07] [Frieze-Jerrum] [Karloff-Zwick] [Zwick SODA 98] [Zwick STOC 98] [Zwick 99] [Halperin-Zwick 01] [Goemans-Williamson 01] [Goemans 01] [Feige-Goemans] [Matuura-Matsui] [Trevisan-Sudan-Sorkin-Williamson] [O’Donnell-Wu] Optimal rounding schemes for MaxCut

ALGORITHM OUTLINE Rounding Any Constraint Satisfaction Problem

Max Cut Max CUT Input : A weighted graph G Find : A cut with maximum fraction of crossing edges

Semidefinite Program Variables : v 1, v 2 … v n | v i | 2 = 1 Maximize Max Cut SDP

v1v1 v2v2 v3v3 v4v4 v5v5 MaxCut Rounding Problem Given a graph on the n - dimensional unit ball, Find the maximum cut of the graph.

Approximation using Finite Models ¦ -CSP Instance = ¦ -CSP Instance = finite variable folding (identifying variables) optimal solution for = finite approximate solution for = unfolding of the assignment constant time Challenge : ensure = finite has a good solution

Approximation using Finite Models [Frieze-Kannan] For a dense instance =, it is possible to construct finite model = finite OPT( = finite ) ≥ (1-ε) OPT( = ) General Method for CSPs What we will do : SDP value ( = finite ) > (1-ε)SDP value ( = ) PTAS for dense instances

Analysis of Rounding Scheme ¦ -CSP Instance = ¦ -CSP Instance = finite SDP value ® SDP value > ® - ² OPT value ¯ rounded value ¯ Hence: rounding-ratio for = < (1+ ² ) integrality-ratio for = finite unfolding

CONSTRUCTING FINITE MODELS (MAXCUT) Rounding Any Constraint Satisfaction Problem

v1v1 v2v2 v3v3 v4v4 v5v5 STEP 1 : Dimension Reduction Pick d = 1/ Є 4 random Gaussian vectors {G 1, G 2,.. G d } Project the SDP solution along these directions. Map vector V V → V’ = (V∙G 1, V∙G 2, … V∙G d ) v1v1 v3v3 v4v4 v5v5 Constant dimensions STEP 2 : Surgery Scale every vector V’ to unit length STEP 3 : Discretization Pick an Є –net for the d dimensional sphere Move every vertex to the nearest point in the Є –net v2v2 v2v2 FINITE MODEL Graph on Є –net points

To Show: SDP value ( = finite ) > (1-ε)SDP value ( = ) Lemma : “Inner Products are almost preserved under random projections” If V’,U’ are random projections of U, V on 1/ ε 4 directions, Pr [ |V∙U – V’∙U’| > ε] < ε 2

STEP 1 : Dimension Reduction Project the SDP solution along 1/ Є 4 random directions. STEP 2 : Surgery Scale every vector V’ to unit length STEP 3 : Discretization Pick an Є –net for the d dimensional sphere Move every vertex to the nearest point in the Є –net For SDP value ( = ) Contribution of an edge e = (U,V) |U-V| 2 = 2-2 V∙U To Show: SDP value ( = finite ) > (1-ε)SDP value ( = ) SDP Vectors for = finite = Corresponding vectors in Є –net STEP 1 With probability > 1- Є 2, | |U-V| 2 - |U’-V’| 2 | < 2Є STEP 2 With probability > 1- 2Є 2, 1+ Є < |V’| 2,|U’| 2 < 1- Є, Normalization changes distance by at most 2Є STEP 3 Changes edge length by at most 2Є

For SDP value ( = ) Contribution of an edge e = (U,V) |U-V| 2 = 2-2 V∙U To Show: SDP value ( = finite ) > (1-ε)SDP value ( = ) SDP Vectors for = finite = Corresponding vectors in Є –net STEP 1 With probability > 1- Є 2, | |U-V| 2 - |U’-V’| 2 | < 2Є STEP 2 With probability > 1- 2Є 2, 1+ Є < |V’| 2,|U’| 2 < 1- Є, Normalization changes distance by at most 2Є STEP 3 Changes edge length by at most 2Є ANALYSIS With probability 1-3Є 2, The contribution of edge e changes by < 6Є In expectation, For (1-3Є 2 ) edges, the contribution of edge e changes by < 6Є SDP value ( = finite ) > SDP value ( = ) - 6Є – 3Є 2

FINITE MODELS FOR GENERAL CSP Rounding Any Constraint Satisfaction Problem

Semidefinite Program for CSPs Variables : For each variable X a Vectors {V (a,0), V (a,1) } For each clause P = (x a ν x b ν x c ), Scalar variables μ (P,000), μ (P,001), μ (P,010), μ (P,100), μ (P,011), μ (P,110), μ (P,101), μ (P,111) X a = 1 V (a,0) = 0 V (a,1) = 1 X a = 0 V (a,0) = 1 V (a,1) = 0 If X a = 0, X b = 1, X c = 1 μ (P,000) = 0μ (P,011) = 1 μ (P,001) = 0μ (P,110) = 0 μ (P,010) = 0μ (P,101) = 0 μ (P,100) = 0μ (P,111) = 0 Objective Function :Constraints : For each clause P, 0 ≤μ (P,α) ≤ 1 For each clause P (x a ν x b ν x c ), For each pair X a, X b in P, consitency between vector and LP variables. V (a,0) ∙V (b,0) = μ (P,000) + μ (P,001) V (a,0) ∙V (b,1) = μ (P,010) + μ (P,011) V (a,1) ∙V (b,0) = μ (P,100) + μ (P,101) V (a,1) ∙V (b,1) = μ (P,100) + μ (P,101)

Semidefinite Relaxation for CSP SDP solution for = : SDP objective: for every constraint Á in = -local distributions ¹ Á over assignments to the variables of Á Example of local distr.: Á = 3XOR(x 3, x 4, x 7 ) x 3 x 4 x 7 ¹ Á … for every variable x i in = -vectors v i,1, …, v i,q constraints (also for first moments) Explanation of constraints : first and second moments of distributions are consistent and form PSD matrix maximize

Strong and Weak STRENGTH For every clause Á in = -local distributions ¹ Á over assignments to the variables of Á Vector variables v i,a within a clause Á satisfy all valid constraints (like triangle inequality) – the inner products are in the integral hull. WEAKNESS The above hard constraint is only for variables that participate together in a clause

Throwing away constraints {v i,a } { μ …} -Infeasible SDP solution for a instance =, it does not satisfy the consistency for a clause P. Consider instance = ‘ = = - {P} Now {v i,a } { μ … } is a good SDP solution for = ‘ Throw away clauses from CSP Throw away constraints from SDP relaxation

v1v1 v2v2 v3v3 v4v4 v5v5 v1v1 v3v3 v4v4 v5v5 Constant dimensions v2v2 v2v2 FINITE MODEL CSP on Є –net points STEP 1 : Dimension Reduction Project the SDP solution along d =1/ Є 4 random directions. STEP 3 : Discretization Pick an Є –net for the d dimensional sphere Move every variable to the nearest point in the Є –net = finite = discretized instance STEP 2 : Throw away Discard clauses for which the corresponding inner products are not preserved within Є. = ‘ = New instance

To Show: SDP value ( = finite ) > (1-ε)SDP value ( = ) SDP Vectors for = finite = Corresponding vectors in Є –net LP variables { μ …}? Problem : The inner products of vectors corresponding to a clause P might not be in the integral hull. ( For example : 3 arbitrary vectors in a Є –net are not guaranteed to satisfy triangle inequality ) The initial SDP solution satisfied all the constraints

STEP 1 : Dimension Reduction Project the SDP solution along d =1/ Є 4 random directions. STEP 3 : Discretization Pick an Є –net for the d dimensional sphere Move every variable to the nearest point in the Є –net = finite = discretized instance STEP 2 : Throw away Discard clauses for which the corresponding inner products are not preserved within Є. = ‘ = New instance From STEP 2, We have discarded clauses for which inner products are not preserved within Є Discarding a clause P Forget about constraints corresponding to P Discretization changes inner product by Є For every remaining clause, all inner products are within 2Є of what it was.

Smoothing Operation Canonical SDP Solution Uniform Distribution over all Integral solutions. Example: V a,0 ∙V a,0 = V a,1 ∙V a,1 = ½ V a,0 ∙V b,0 = V a,0 ∙V b,1 = V a,1 ∙V b,0 = V a,1 ∙V b,1 = 1/4 Є –net Solution SDP Vectors for = finite = Corresponding vectors in Є –net (1-Є) X + = Final SDP solution Integral Hull Є X Є Consider the inner products corresponding to a single clause P SDP Objective value remains roughly the same.

Conclusions Rounding stronger SDPs. More efficient rounding? Can this SDP be solved in constant dimensional space directly? Integrality gaps for stronger SDP relaxation of Unique Games

Thank You

Good finite Models from SDP solutions – Dimension Reduction & Discretization ¦ -CSP Instance = ¦ -CSP Instance = finite SDP solution for = compute Dimension Reduction Project on random low dimensional subspace almost SDP solution for = Discretize Move vectors to closest point on ² -net almost SDP solution for = RnRn RdRd identify variables with same vectors Theorem: SDP value ( = finite ) > SDP value ( = ) Theorem: SDP value ( = finite ) > SDP value ( = ) Idea : use almost SDP solution and do surgery finite number of different vectrs

Constraint Satisfaction Problems (CSP) CSP ¦ finite set of allowed types of constraints Á : [q] k  {0,1} (alphabet [q], arity k) e.g. ¦ = { 3XOR, 3SAT, 3NAE} ¦ -CSP Instance = -variables x 1,…,x n -list of constraints Á of type ¦ on subsets of variables ¦ -CSP Instance = -variables x 1,…,x n -list of constraints Á of type ¦ on subsets of variables Goal : Find assignment x 2 [q] n so as to maximize fraction of satisfied constraints  opt( = ) Examples: Max-Cut, Max-3SAT,… PCP Theorem: NP-hard to distinguish opt( = )=1 and opt( = )<0.9 (even for constant k and q) Approximation Algorithms: Goemans-Williamson, Zwick, CMM, …