MaxClique Inapproximability Seminar on HARDNESS OF APPROXIMATION PROBLEMS by Dr. Irit Dinur Presented by Rica Gonen

MaxClique Inapproximability using PCP This talk will present the MaxClique inapproximability proved by Subhash Khot. The inapproximability result uses –a PCP verifier for 3SAT –to determine the size of a clique in graph G with N’ vertices –where G is the product graph of a randomized reduction from 3SAT to graph G.

Talk Outline MaxClique inapproximability – results’ history. Khot result - general structure. Technical background. Khot verifier - construction and proof. MaxClique inapproximability using Khot verifier.

MaxClique Inapproximability – Results’ History.

On the positive side –The best (known) polynomial time approximation achieves an approximation ratio of –It is of the form On the negative side –The first step towards proving strong inapproximability was taken by Feige et al. –They showed a connection between Probability Checkable Proof Systems and inapproximability of MaxClique. –The discovery of the PCP theorem by Arora et al implied that MaxClique is inapproximable within a factor for some constant c > 0 unless P = NP.

On the negative side (cont) –Bellare and Sudan defined an important parameter of PCP called amortized free bit complexity and showed that –Theorem: if NP has probabilistically checkable proofs where the verifier uses logarithmic randomness and amortized free bits, then MaxClique is inapproximable in polynomial time with a factor for any constant > 0 unless NP=ZPP. –They constructed PCPs with 3 amortized free bids and obtained a hardness factor of for clique. –It was improved by Bellare et all to.

On the negative side (cont) –Hastad proved an inapproximability factor. He obtain a PCP verifier that achieves amortized free bit complexity for arbitrarily small constant >0. –Khot paper aim at getting the best subconstant value of in Hastad’s result.

Khot’s Result – General Structure.

Approximating MAX-Clique is NP-hard PCP Theorem: We will reduce 3SAT to MAX-Clique.  = {    l } of clauses over variables y 1,...,y m of range 2 V, Each of  1  l depends on at most 3 variables. The verifier V uses random bits and chooses a uniformly random clause of (* denotes the variable or its compliment), and query q bits of the proof. The proof corresponds to an assignment of the variables appearing in and accepts iff the values satisfy If is SAT then If is unSAT then

Approximating MAX-Clique is NP-hard We will construct a graph, G , that: has a clique of size at least c  there exists an assignment, satisfying the constraints y 1,...,y m that verifier V accepts with probability has a clique of size at most s  there does not exists an assignment, satisfying the constraints y 1,...,y m that verifier V accepts with probability

(, )-co-partite Graph G=(R  Q, E) Comprise independent sets of size < 2^r

Clique Instance: an (, )-co-partite graph G=(R  Q, E) Problem: distinguish between –Good: CL(G) = c –Bad: every set w  V s.t. |w|> s is not a clique.

 3SAT  MAX-Clique y1y1 y1y1 y2y2 y2y2 yiyi yiyi y m-1 ymym ymym 1111 1111 jjjj jjjj llll llllTT T TT FTF T 2^r Construct a graph G  that has 1 independent set  i  , randomly chosen by the verifier V, in which 1 vertex  assignment for  I accepted by the verifier V.

3SAT  MAX-Clique y1y1 y1y1 y2y2 y2y2 yiyi yiyi y m-1 ymym ymym 1111 1111 jjjj jjjj llll llllTT T TT FTF T 2^r Two vertices are connected iff the assignments they represent are consistent

Lemma: If is SAT, clique of size at least c  verifier V accepts with probability  Consider an assignment A that was accepted by the verifier V with probability. It satisfies at least c clauses. For each clause i consider A's restriction to  i ‘s variables The corresponding c vertexes form a clique in G   Any clique of size c in G  implies an assignment satisfying c clauses accepted by the verifier V with probability. 3SAT  MAX-Clique

Lemma: If is unSAT, clique of size at most s  verifier V accepts with probability  Consider an assignment A that was accepted by the verifier V with probability. It satisfies at most s clauses. For each clause i consider A's restriction to  i ‘s variables The corresponding s vertexes form a clique in G   Any clique of size s in G  implies an assignment satisfying s clauses accepted by the verifier V with probability. 3SAT  MAX-Clique Hence: MAX-Clique is NP hard, and MAX- Clique is NP-hard to approximate!

Theorem B.1 (Engebretsen and Holmerin):Theorem B.1 if there is a PCP verifier for 3SAT using r random bits, f free query bits, completeness c and soundness s, then there is a randomized reduction from 3SAT to a graph G with vertices such that: –If the 3sat formula is satisfiable with probability 2/3, G has a clique of size at least –If the formula is unsatisfiable with probability 2/3, maximum clique size in G is at most. Khot Constructs a PCP verifier for 3SAT

Khot wants to use Hadamard code in the PCP verifier Hastad’s result uses long code encodings. Long coed encodes a u-bit string by a -bit string. To improve Hastad’s result Khot uses Hadamard code. Hadamard code encodes a u-bit string by a -bit string and allows randomness efficient checking. Hadamard codes are define using linear functions Khot needs an underlying NP-hard problem that features linear constraints. He defines such a problem called Max-3Lin( ).

The hardness of Max-3Lin( ) Stated according to Hastad theorem: Theorem 2.1:Theorem 2.1 There exist a polynomial time reduction from a SAT formula with n variables to a system L modulo 2 with N variables such that: –If is satisfiable, there exist an assignment to the variables in L that satisfies 1- fraction of equations. –If is unsatisfiable, no assignment can satisfy more than 1/2+ fraction of the equations. –Every equation contains exactly 3 variables and every variable appears in exactly 7 equations. Moreover, the reduction can achieve = for some constant >0 if we allow the running time of the reduction and N to be slightly superpolynomial, i.e.

Khot’s PCP Verifier expects a (Hadamard) encoding of the proof supplied to the Raz Verifier. Details about the Raz Verifier in the Technical background section. Khot’s PCP Verifier construction makes use of the Raz Verifier.

Technical background

The Raz Verifier The Raz Verifier is given an instance L of Max- 3Lin( ). It expects two proofs P and Q. For every set U of u variable, P(U) is a u-bit string giving the values of those variables in some global assignment. For every set W of u equations, Q(W) is a 3u-bit string giving the values of the 3u variables appearing in these u equations.

The Raz Verifier works as follows: It randomly picks variables U= It picks equations W= where equation is chosen randomly from equations containing variable The verifier accepts iff –Q(W) satisfies all the equations (the linear constraints test) –P(U)= (Q(W)), where is the projection from 3u-bit strings to u-bit strings i.e. the values of the variables in P(U) and Q(W) are the same. (the projection test).

Completeness of the Raz Verifier Completeness is. If there is an assignment that satisfies fraction of equations, both P and Q are consistent with this assignment. With probability, all the equations will be satisfied and the verifier will accept.

Soundness of the Raz Verifier When at most 1/2+ fraction of equations in L are satisfiable, the soundness can be upper bounded by Raz’s Parallel Repetition Theorem. Theorem 2.2:Theorem 2.2 There exist an absolute constant < 1 such that soundness of the Raz Verifier for Max-3Lin( ) is at most

Hadamard Codes Hadamard(p) : Hadamard code of is the -bit string. – returns 1 or -1 for every vector a. – has vectors.

Khot verifier - construction and proof.

The Construction of Khot’s PCP Verifier ( ) is given an instance L of Max-3Lin( ). expects proofs P’,Q’ in Hadamard codes. –P’,Q’ are encodings of proofs P,Q –P,Q are supplied to the Raz Verifier. picks a set U of u variables at random. picks k sets independently. –Each set is picked such that it has u equations and every variable appears at least ones in the set.

The Construction of Khot’s PCP Verifier ( ) (cont) Let A be Hadamard code of P(U) Let be Hadamard code of. –Tables are assumed to be folded over respective linear constraints. pickes and randomly. accepts iff for – is the projection function between and U.

Ignoring the linear constraints test If Khot’s PCP verifier accepts the encoded proofs with a good probability, then these proofs can be decoded to construct proofs (P,Q) which the Raz Verifier accepts with a good probability. Folding ensures that the decoding procedure satisfies the linear constraints on W. Thus linear constraint test can be ignored.

Folding Let the string x=Q(W) read by the Raz Verifier satisfy the linear constrains modulo 2, – Let B be Hadamard code of x. Let H be linear subspace spanned by the vectors Let For, (1):(1) – denotes the lexicographically smallest vector in the set of vectors B’ is a folding of B over the linear constrains.

Folding (cont) Decoding of a table B gives with probability. Folding ensures that any given by this decoding procedure satisfies the linear constraints on W. Lemma 2.4: if, then must satisfy the linear constraints, i.e..

Folding (cont) It will be required that the supposed Hadamard codes be folded over the respective constraints. This requirement can be enforced using the following access mechanism. –When the verifier wants to read B(b), –it reads B( ) instead –And “calculates” the value of B(b) from (1).(1)

Analysis Theorem 3.1:Theorem 3.1 The Verifier for Max-3Lin( ) instance L with N variables –Uses r=ulogN +O(ku) random bits. –Queries bits from the proof with f=2k free bits. –Has completeness at least –Has soundness provided

Analysis of random bits and queries picks u variables which are ulogN bits at random picks uk equations which are O(ku) bits at random. Queries k bits and k bits giving 2k free bits Queries bits projection test.

completeness Assume that there is an assignment that satisfies 1- fraction of equations. Proofs P,Q are consistent with this assignment. And encoded with correct Hadamard codes to construct proofs P’,Q’. With probability 1- a single equation can be satisfied. With probability, all the ku equations in the sets will be satisfied.

completeness (cont) With correct Hadamard codes, –Since a projection function maintains the parity Since in a correct proof.

MaxClique inapproximability using Khot verifier.

Inapproximability for MaxClique Theorem 1.2: It is not possible to approximate MaxClique in polynomial time within a factor for some constant unless To prove Theorem 1.2 –Use verifier from Theorem 3.1Theorem 3.1 With superconstant values of u,k And subconstant value of As given by Theorem 2.1Theorem 2.1 –And apply Theorem B.1.Theorem B.1 max-3Lin max-3Lin max-3sat max-3sat max- clique

Inapproximability for MaxClique (cont) The gap between maximum clique sizes in Theorem B.1 is –For some constant –N’ is the size of the graph produced by the reduction in Theorem B.1.

If you are still awake… Soundness and Technical Background

Fourier Transforms Let Function A is called linear if. for every, there is a function defined by –There are vectors in and therefore linear functions. Define an inner product on this space as

Fourier Transforms (cont) The set of all linear functions forms an orthonormal basis for this vector space w.r.t. the above inner product. –Orthogonal: is a constant In summation over half elements are odd and half even Half elements =-1 Half elements =1 –Orthonormal: is either 1 or -1. Both squared =1 functions results in summing 1s It follows that any function A can be uniquely expressed as where are its Fourier coefficients.

Fourier Transforms (cont) A projection function is a function that maps vectors in to some fixed u coordinates. For, let denote the unique vector such that and coordinates of c other than those projected by are 0.

Soundness To prove the soundness, by Theorem 2.2, it is sufficient to show thatTheorem 2.2 –If the soundness is –Then there exist proofs P,QThen there exist proofs P,Q –Which the Raz Verifier accepts with probability ( ). –According to Theorem 2.2 the soundness of the Raz Verifier is at most –Therefore the soundness of can not exceed

Soundness According to Samorodnitsky and Trevisan the acceptance probability of the verifier is given by: (2)

Soundness (cont) If this probability is, there exist a nonempty set such that. –The summation at (2) has elements.(2) –Assume to the contrary that there is no set such that. –Then for every set,. –Meaning –It follows that –Contradicting our assumption

Soundness (cont) Lemma 3.2 by Samorodnitsky and Trevisan enables us to assume that S is of the form [2]x[d] – –Lemma 3.2: if for some non-empty set S, then for some. Two cases of d are considered, even and odd.

The case when S=[2]x[d] and d is even (3) –Since the power does not depend on j –(-1) to the power of even number of times and therefore equals 1. –Similarly

The case when S=[2]x[d] and d is even Using the following Fourier expansions –Since (4) –Since a projection function maintains the parity Similarly (5)

The case when S=[2]x[d] and d is even Substituting (4)and (5) in (3)(4)(5) (3) (6)

The case when S=[2]x[d] and d is even Taking expectation over, only if, the terms in (6) are non-zero.(6) – – and are the orthonormal basis vectors. –If, –(6) will equal 0(6) Taking expectation over, only if, the terms in (6) are non-zero.(6) –If then –Similarly if

The case when S=[2]x[d] and d is even It is concluded that: (7)

The case when S=[2]x[d] and d is odd Using Fourier expansions of and similarly to the case where d is even (8)

Define proofs for the Raz Verifier Remainder of the steps in the soundness proof. Define proofs for the Raz Verifier as follows: –For the set W, pick with probability – Remainder – folding ensures that this satisfies the linear constraints on W. –For a set U, pick sets at random –And pick with probability –If d is even, Define –If d is odd, Pick with probability Define

Soundness The acceptance probability of the Raz Verifier on these proof is the expressions in (7) and (8).(7) (8) There exist at least one choice of proofs P,Q which is accepted by the Raz Verifier with probability at least.

Thank You!

Inapproximability for MaxClique (cont) Construct a PCP verifier for 3SAT as follows: –Using Theorem 2.1 transform a given 3SAT formula to an instance L of Max-3Lin( ) –Using Theorem 3.1 construct a PCP verifier for L –Apply Theorem B.1