Constraint Satisfaction over a Non-Boolean Domain Approximation Algorithms and Unique Games Hardness Venkatesan Guruswami Prasad Raghavendra University of Washington Seattle, WA

Constraint Satisfaction Problem A Classic Example : Max-3-SAT Given a 3-SAT formula, Find an assignment to the variables that satisfies the maximum number of clauses. Equivalently the largest fraction of clauses

Constraint Satisfaction Problem General Definition : Domain : {0,1,.. q-1} Predicates :{P 1, P 2, P 3 … P r } P i : [q] k -> {0,1} Arity : Maximum number of variables per constraint (k) Example : Max-3-SAT Domain : {0,1} Predicates : P 1 (x,y,z) = x ѵ y ѵ z Arity = 3 GOAL : Find an assignment satisfying maximum fraction of constraints

Approximability Max-3-SAT : 7/8 [Karloff-Zwick],[Hastad] Most Max-CSP problems are NP-hard to solve exactly. Different Max-CSP problems are approximable to varying ratios. 3-XOR : ½ [Hastad] Max-Cut : [Goemans-Williamson], [Khot-Kindler-Mossel- O’donnel] Max-2-SAT : 0.94 [Lempel-Livnat- Zwick],[Austrin]

Question : Which Max-CSP is the hardest to approximate? Refined Question : Among all Max-CSP problems over domain [q] ={0,..q-1}, and arity k, which is the hardest to approximate? Clearly, the problems become harder as domain size or the arity grows

PCP Motivation Probabilistically Checkable Proof (A string over alphabet {0,1,..q-1}) Verifier Random bits ACCEPT/REJECT Completeness(C) : If SAT formula is satisfiable, there is a proof that verifier accepts with probability C Soundness (S): For an unsatisfiable formula, no proof is accepted with probability more than S

Among all Max-CSP problems over domain [q] ={0,..q-1}, and arity k, which is the hardest to approximate? PCP Motivation What is the best possible gap between completeness (c) and soundness (s) for a PCP verifier that makes k queries over an alphabet [q] = {0,1,..q-1} ?

Boolean CSPs Hardness: For every k, there is a boolean CSP of arity k, which is NP- hard to approximate better than : Algorithm: Every boolean CSP of arity k, can be approximated to a factor : [Samorodnitsky-Trevisan 2000] simplified by [Hastad-Wigderson] [Engebresten-Holmerin] [Samorodnitsky-Trevisan 2006] Assuming Unique Games Conjecture Random Assignment [Trevisan] [Hast] [Charikar-Makarychev-Makarychev]

This Work : Non-Boolean CSPs UG Hardness: Assuming Unique Games Conjecture, For every k, and a prime number q, there is a CSP of arity k over the domain [q] ={0,1,2,..q-1}, which is NP-hard to approximate better than Algorithm: The algorithm of [Charikar-Makarychev- Makarychev] can be extended to non-boolean domains. Every CSP of arity k over the domain [q] ={0,1,2,..q-1} can be approximated to a factor

Related Work [Raghavendra 08] “Optimal approximation algorithms and hardness results for every CSP, assuming Unique Games Conjecture.” – “Every” so applies to the hardest CSPs too. – Does not give explicit example of hardest CSP, nor the explicit value of the approximation ratio. [Austrin-Mossel 08] “ Assuming Unique Games conjecture, For every prime power q, and k, it is NP-hard to approximate a certain CSP over [q] to a factor > ” – Independent work using entirely different techniques(invariance principle) – Show a more general result, that yields a criteria for Approximation Resistance of a predicate.

Techniques We extend the proof techniques of [Samorodnitsky-Trevisan 2006] to non- boolean domains. To this end, we – Define a subspace linearity test. – Show a technical lemma relating the success probability of a function F to the Gower’s norm of F (similar to the standard proof relating the number of multidimensional arithmetic progressions to the Gower’s norm) Along the way, we make some minor simplifications to [Samorodnitsky-Trevisan 2006]. – (Remove the need for common influences)

Proof Overview

Given a function F : [q] R [q], Make at most k queries to F Based on values of F, Output ACCEPT or REJECT. Distinguish between the following two cases : Dictatorship Testing Problem F is a dictator function F(x 1,… x R ) = x i F is far from every dictator function (No influential coordinate) Pr[ACCEPT ] = Completeness Pr[ACCEPT ] = Soundness Goal : Achieve maximum gap between Completeness and Soundness

UG Hardness Proofs UG Hardness Result: Assuming Unique Games Conjecture, it is NP- hard to approximate a CSP over [q] with arity k to ratio better than C/S Using [Khot-Kindler-Mossel-O’Donnell] reduction. Dictatorship Test Over functions F:[q] R -> [q] Completeness = C Soundness = S # of queries = k For the rest of the talk, we shall focus on Dictatorship Testing.

Testing Dictatorships by Testing Linearity [Samorodnitsky-Trevisan 2006] Fix {0,1} : field on 2 elements k = 2 d Given a function F : {0,1} R -> {0,1} Pick a random affine subspace A of dimension d. Test if F agrees with some affine linear function on the subspace A. Every dictator F(x 1, x 2,.. x R ) = x i is a linear function over vector space {0,1} R Random Assignment : There are 2 d+1 different affine linear functions on A. There are possible functions on A. So a random function satisfies the test with probability

Gower’s Norm For F : {0,1} R -> {0,1}, let f(x) = (-1) F(x). d th Gowers Norm U d (f) = E [ product of f over C ] Expectation over random d- dimensional subcubes C in {0,1} R xx+y 1 x+y 2 x+y 1 +y 2 xx+y 1 x+y 2 x+y 1 +y 2 x x+y 1 x+y 3 + y 2 x+y 1 +y 2 +y 3 x+y 3 x+y 1 +y 3 d-dimensional cube spanned by {x,y 1,y 2,.. y d } is Cubes

More Formally, Intuitively, the d th Gower’s norm measures the correlation of the function f with degree d-1 polynomials. Gower’s Norm

Testing Dictatorships by Testing Linearity [Samorodnitsky-Trevisan 2006] Lemma : If F : {0,1} R -> {0,1} passes the test with probability then f = (-1) F has high d th Gowers Norm. (k=2 d ) Lemma : If a balanced function f : {0,1} R -> {-1,1} has high d th Gowers Norm, then it has an influential coordinate (k=2 d ) Theorem : If a balanced function F : {0,1} R -> {0,1} passes the test with probability then it has an influential coordinate Using Noise sensitivity, There are only a FEW influential coordinates.

Extending to Larger domains Fix [q]: field on q elements(q is a prime). k = q d Given a function F : [q] R -> [q] Pick a random affine subspace A of dimension d. Test if F agrees with some affine linear function on the subspace A. Replace 2 by q in the [Samorodnitsky- Trevisan] dictatorship test.

The Difficulty Lemma : If F : {0,1} R -> {0,1} passes the test with probability then f = (-1) F has high d th Gowers Norm. (k=2 d ) Over {0,1} R, Subcube = Affine subspace. Testing linearity over a random affine subspace, can be easily related to expectation over a random cube. Over [q] R, Subcube ≠ Affine subspace. (2 R points) (q R points)

Success probability of a function F : {0,1} R -> {0,1}, is related to : let f(x) = (-1) F(x). E [ product of f over A ] Expectation over random d-dimensional affine subspace A in [q] R (Affine subspaces are like multidimensional arithmetic progressions) xx+y 1 x+2y 1 x+(q-2)y 1 x+(q-1)y 1 x+y 2 +y 1 x+y 2 +2y 1 x+y 2 + (q-2)y 1 x+y 2 + (q-1)y 1 x x+y 1 x+2y 1 x+ (q-2)y 1 x+ (q-1)y 1 x+2y 2 +y 1 x+2y 2 +2y 1 x+2y 2 + (q-2)y 1 x+2y 2 + (q-1)y 1 x+(q-2)y 2 +y 1 x+(q-2)y 2 +2y 1 x+(q-2)y 2 + (q-2)y 1 x+(q-2)y 2 + (q-1)y 1 x+(q-2)y 2 x+2y 2 x+y 2 x+(q-1)y 2 x+(q-1)y 2 +y 1 x+(q-1)y 2 +2y 1 x+(q-1)y 2 + (q-2)y 1 x+(q-1)y 2 + (q-1)y 1 Multidimensional Progressions E [ product of f over C ] Expectation over random dq-dimensional subcubes C in [q] R

Alternate Lemma Lemma : If F : [q] R -> [q] passes the test with probability then f = (-1) F has high dq th Gower’s Norm. (k=q d ) d-dimensional affine subspace test relates to the dq th Gower’s norm The proof is technical and involves repeated use of the Cauchy-Schwartz inequality. A special case of a more general result by [Green-Tao][Gowers-Wolf], where they define “Cauchy-Schwartz Complexity” of a set of linear forms.

Open Questions CSPs with Perfect Completeness: Which CSP is hardest to approximate, under the promise that the input instance is completely satisfiable? Approximation Resistance: Characterize CSPs for which the best approximation achievable is given by a random assignment.

Thank You

Unique Games A Special Case E2LIN mod p Given a set of linear equations of the form: X i – X j = c ij mod p Find a solution that satisfies the maximum number of equations. x-y = 11 (mod 17) x-z = 13 (mod 17) … …. z-w = 15(mod 17)

Unique Games Conjecture [Khot 02] An Equivalent Version [Khot-Kindler-Mossel-O’Donnell] For every ε> 0, the following problem is NP-hard for large enough prime p Given a E2LIN mod p system, distinguish between: There is an assignment satisfying 1-ε fraction of the equations. No assignment satisfies more than ε fraction of equations.