1. 1 Design and Analysis of Algorithms Yoram Moses Lecture 13 June 17, 2010 http://www.ee.technion.ac.il/courses/046002

2. 2 The Cook-Levin Theorem

3. 3 Reminder: NP Language: L  {0,1} * Verification relation for L: R  {0,1} *  {0,1} *  If x  L, then there is a “certificate” y s.t. (x,y)  R.  If x  L, then there is no y s.t. (x,y)  R. R is polynomially bounded, if there is a constant c > 0 s.t. |y| ≤ |x| c holds for every (x,y)  R. L is polynomial-time verifiable, if it has a verification relation R that is both  polynomially bounded and  polynomial-time decidable. NP = all polynomial-time verifiable languages.

4. 4 Reminder: NP-Completeness A language L is NP-hard if L ’ ≤ p L for all L ’  NP.  There is a reduction f: {0,1} *  {0,1} * s.t. x  L ’ iff f(x)  L f is polynomial-time computable A language L is NP-complete if both  L  NP and  L is NP-hard

5. 5 Reminder: Boolean Circuits Boolean circuit:  A directed acyclic graph (DAG)  Nodes are also called gates  n input gates (in-degree = 0)  Each input gate is labeled by a distinct Boolean variable (denoted x 1,…,x n ).  1 output gate (out-degree = 0)  Internal gates are labeled by AND / OR / NOT  AND, OR gates have in-degree ≥ 2  NOT gates have in-degree = 1

6. 6 Boolean Circuits: Example x1x1 x3x3 x2x2        input gates internal gates output

7. 7 Reminder: Input Assignments Input Assignment: a vector   {0,1} n  Assigns a binary value to each of the n input variables x 1,…,x n. Circuit evaluation: a mapping C: {0,1} n  {0,1}  Maps every assignment  to an output bit (1 or 0).  C(  ) = evaluation of C on   Evaluation is done in topological order, starting from the input gates.

8. 8 Circuit Evaluation: Example x1x1 x3x3 x2x2        input gates internal gates output 1 1 0 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0

9. 9 Reminder: Circuit-SAT Satisfying assignment: an input assignment  for which C(  ) = 1. Satisfiable circuit: a circuit that has at least one satisfying assignment. Circuit-SAT = language of all satisfiable circuits. Last lecture we showed: Lemma: Circuit-SAT  NP.

10. 10 Cook-Levin Theorem Theorem [ Cook-Levin, 71 ]: Circuit-SAT is NP-hard. Corollary: Circuit-SAT is NP-complete.  Our “First” NP-Complete problem Proof plan: Directly reduce every language L  NP to Circuit-SAT.

11. 11 Proof Plan of Cook-Levin Theorem Suppose that L  NP.  there exist a verification relation R L, a TM M L, and constants c 1,c 2 such that  x  L iff there exists y, with |y| ≤ |x| c 1, s.t. (x,y)  R L.  M L accepts input z iff z=(x,y)  R L.  time(M L,z) = O(|z| c 2 ) Note: inputs z to M L consist of two components:  Part A is x whose size is |x|=m bits  Part B is y whose size is |y|=m c 1 bits

12. 12 Proof Plan of Cook-Levin Theorem For each input x, define a circuit f(x) = C x s.t.  C x can be constructed (computed) in polynomial time  M L accepts (x,y) iff C x (y) = 1. Once we do this, we obtain that x  L   y: M L accepts “x#y”   y: C x (y) = 1  C satisfiable Conclusion: L ≤ p Circuit-SAT Since L is an arbitrary language in NP, Circuit-SAT is NP-hard! (and NPC)

13. 13 Reminder: Turing Machines Hardware:  Control: a finite state automaton  Memory: an infinite tape  Head: read/writes symbols on tape and can move right/left abba    … infinite tape: head q2q2 state

14. 14 Reminder: Turing Machines A Turing machine M consists of:  A set of states Q Start state: q 0 Accept state: q accept Reject state: q reject  An alphabet   A transition function  : Q    Q    { L,R } Machine configuration:  Current state  Content of tape  Position of head a b b a#q 2  …

15. 15 Reminder: Turing Machines Start configuration on input x = x 1 …x n : Halting configurations:  Accepting configuration: control state is q accept  Rejecting configuration: control state is q reject Successive configurations: x 1 #q 0 x2x2   … xnxn … a b b a#q 2  … a b b#q 4 c  …  q 2,a) = (q 4,c,R)

16. 16 Reminder: Turing Machines Computation of M on input x: A sequence of configurations C 1, C 2,…, C k, where  C 1 is the start configuration with x written on the tape  C k is a halting configuration  C i is a successor of C i-1 for i = 2,…,k Acceptance/rejection:  M accepts x if C k is an accepting configuration.  M rejects x if C k is a rejecting configuration. M decides L if both  M halts on every input x, and  M accepts x iff x  L.

17. 17 Tableau M: TM, z: input C 1,…, C k : computation of M on z k = time(M,z) = running time of M on z Note: if time(M,z) = k, head never goes beyond k th cell.  in all configurations C 1,…, C k, cells k+1,k+2,… are blank.  =   { a#q | a  , q  Q }  each configuration C i can be represented as a vector in  k. Tableau of M on x: a k  k matrix, whose rows are C 1,…, C k.

18. 18 Properties of Tableaus Observation 1: in all tableaus T, the value of T[i,j] depends only on the three cells above T[i,j]. The only cells that can change from C j-1 to C j are:  The cell in which the head is located at step j-1  The cell to its left  The cell to its right Note: the dependence is dictated only by . j-1 j i i-1 i+1

19. 19 Properties of Tableaus Observation 2: the dependence of T[i,j] on the three cells above it is the same for all tableaus T of the same Turing machine M. (they all have the same  Conclusion: For every TM, there is a “tableau function” g:  3   s.t. for all tableaus T of M and for all i,j > 1 T[i,j] = g(T[i-1,j-1], T[i,j-1], T[i+1,j-1]) Special care is needed at boundaries:  j = 1: first row of T depends on the input x only  i = 1,k: cells in first/last column depend only on the two cells above them.

20. 20 A Circuit for g Observation 3: Since  has constant size, we can design a circuit C g of constant size that computes the value at T[i,j] for i>1 based on the three cells above it. a circuit C’ g of constant size that computes the value at T[1,j] based on the two cells above it. The same two circuits can be used for all tableaus T of a given Turing machine M. C’gC’g CgCg

21. 21 Simulation of TMs by Circuits M: Turing machine t(n) = max z : |z|=n time(M,z) Theorem: Fix M. For all n, there exists a circuit C n s.t.:  For every input x of length n: C n (z) = 1 iff M accepts z.  C n can be constructed in O(t(n) 2 ) time

22. 22 Proof of Simulation Theorem z: input of length n T z : tableau of M on z. t = t(n) Since time(M,z) ≤ t(n) = t, can assume T z is a t  t matrix.  If time(M,x) = k < t, add t – k copies of last column and last row of original tableau. g: the tableau function of M C g : a circuit that computes g Note: for all j > 1, T x [i,j] can be generated from the two/three cells above it using C g

23. 23 Circuit Construction The circuit C n :  Input: n symbols of z  Output: 1 if M accepts z and 0 otherwise.  For simplicity, we assume wires of C n can carry symbols from  and not just bits. C n will be a combination of t 2 + 1 sub-circuits  A t  t mesh of sub-circuits C i,j, for i,j = 1,…,t C i,j computes (i,j)-th cells of the tableau T z  A “decision circuit” C dec, which determines whether the last row of T z consists of an accepting configuration.

24. 24 Circuit Construction z 1 #q 0 znzn z2z2 C’gC’g CgCg CgCg CgCg CgCg C’gC’g CgCg CgCg CgCg CgCg C’gC’g CgCg CgCg CgCg CgCg C dec   1 z3z3 CgCg CgCg CgCg … … … … … … … … 1 2 2 3 3 t n n+1 t … … … … … …

25. 25 Circuit Construction (cont.) First row of the mesh (j = 1):  C 1,1 : translates input symbol z 1 into symbol ‘z 1 #q 0 ’  C i,1, i = 2,…,n: relays input symbol z i  C i,1, i = n+1,…,t: outputs the constant symbol ‘  ’ Other t-1 rows of the mesh (j > 1):  All sub-circuits are copies of C g and C’ g  C 1,j : gets its inputs from C 1,j-1, C 2,j-1.  C i,j, i = 2,…,t – 1: gets its inputs from C i-1,j-1, C i,j-1, C i+1,j-1.  C t,j : gets its inputs from C t-1,j-1, C t,j-1. C dec :  Gets its inputs from C 1,t,…,C t,t  Outputs 1 if output of one of them consists of a symbol of the form ‘*#q accept ’ and 0 otherwise

26. 26 Correctness Analysis If input to C n is z, then:  C i,j computes T z [i,j]  Therefore, C dec outputs 1 iff q accept appears on the last row of T z  However, q accept appears on last row of T z iff M accepts z Conclusion: C n (z) = 1 iff M accepts z.

27. 27 Running Time Analysis Time to construct each sub-circuit:  Input sub-circuit: O(1) time  Constant sub-circuit: O(1) time  C g, C’ g : O(1) time  C dec : O(t) time Time to wire all sub-circuits together: O(t 2 ) time Total cost of the reduction: O(t 2 ) time

28. 28 Proof of Cook-Levin Theorem Suppose that L  NP.  there exist a verification relation R L, a TM M L, and constants c 1,c 2 s.t.  x  L iff there exists y, |y| ≤ |x| c 1, s.t. (x,y)  R L.  M L accepts input z iff z=(x,y)  R L.  time(M L,m c 1 ) = O(m c 2 ) Note: inputs z to M L consist of two parts:  Part A: x of size m bits  Part B: y of size m c 1 bits

29. 29 Proof of Cook-Levin Theorem For each input x, define a circuit C x as follows:  C x is the same as C m+m c 1 except that the m input wires corresponding to Part A are hard-wired with the symbols of x. By the simulation theorem:  M L accepts (x,y) iff C x (y) = 1.  C x can be constructed in O((m+m c 1 ) 2c 2 ) time. Hence, our reduction is: f(x) = C x  x  L   y: M accepts “x#y”   y: C x (y) = 1  C x satisfiable  C x is computable in polynomial time Conclusion: L ≤ p Circuit-SAT Since L was an arbitrary language in NP, Circuit-SAT is NP-hard!

30. 30 End of Lecture 13