1. Fall 2013 CMU CS 15-855 Computational Complexity Lecture 3 and 4 Non-determinism, NTIME hierarchy threorem, Ladner’s Proof 9/17/2013

2. Robust Time and Space Classes Robust time and space classes: L = SPACE(log n) PSPACE =  k SPACE(n k ) P =  k TIME(n k ) EXP =  k TIME(2 n k )

3. 9/17/2013 Our world picture so far EXP PSPACE P L

4. 9/17/2013 Relationships between classes So far: L µ P µ PSPACE µ EXP believe all containments strict know L ( PSPACE, P ( EXP even before any mention of NP, two major unsolved problems: L = P P = PSPACE ??

5. 9/17/2013 Completeness complexity class C L is C-hard if –every language in C reduces to L language L is C-complete if –L is in C –L is C-hard

6. 9/17/2013 A P-complete problem logspace reduction: f computable by TM that uses O(log n) space –denoted “L 1 ≤ L L 2 ” If L 2 is P-complete, then L 2 in L implies L = P (homework problem)

7. 9/17/2013 A P-complete problem Circuit Value (CVAL): given a variable-free Boolean circuit (gates , , , 0, 1), does it output 1? Theorem: CVAL is P-complete. Proof: –already argued in P –L arbitrary language in P, TM M decides L in n c steps

8. 9/17/2013 A P-complete problem Tableau (configurations written in an array) for machine M on input w: w1/qsw1/qs w2w2 …wnwn _ … w1w1 w2/q1w2/q1 …wnwn _ … w1/q1w1/q1 a…wnwn _ … _/q a _…__ …............ height = time taken = |w| c width = space used ≤ |w| c

9. 9/17/2013 A P-complete problem Important observation: contents of cell in tableau determined by 3 others above it: a/q 1 ba b/q 1 a a a aba b

10. 9/17/2013 A P-complete problem Can build Boolean circuit STEP –input (binary encoding of) 3 cells –output (binary encoding of) 1 cell ab/q 1 a a STEP each output bit is some function of inputs can build circuit for each size is independent of size of tableau

11. 9/17/2013 A P-complete problem |w| c copies of STEP compute row i from i-1 w1/qsw1/qs w2w2 …wnwn _ … w1w1 w2/q1w2/q1 …wnwn _ …............ Tableau for M on input w … … STEP

12. 9/17/2013 A P-complete problem w 1/ q s w2w2 …wnwn _ … STEP............ 1 iff cell contains q accept ignore these This circuit C M, w has inputs w 1 w 2 …w n and C(w) = 1 iff M accepts input w. logspace reduction Size = O(|w| 2c ) w1w1 w2w2 wnwn

13. 9/17/2013 Answer to question   10   101  Can we evaluate an n node Boolean circuit using O(log n) space? NO! (probably) CVAL in L if and only if L = P

14. 9/17/2013 L  EXP PAD L  P A consequence of measuring running time as a function of input length: -.> –suppose L  EXP, and define PAD L = { x# N : x  L, N = 2 |x| k } –TM that decides PAD L : ensure suffix of N #s, ignore #s, then simulate TM that decides L –running time now polynomial ! –<- M reg PAD L  P so M runs in time exp in length of x

15. 9/17/2013 Succinctness converse (intuition only): “succinctness” –suppose L is P-complete –intuitively, some inputs are “hard” -- require full power of P –SUCCINCT L has inputs encoded in different form than L, some exponentially shorter –if “hard” inputs are exponentially shorter, then candidate to be EXP-complete

16. 9/17/2013 Succinct encodings succinct encoding for a directed graph G= (V = {1,2,3,…}, E): a succinct encoding for a variable-free Boolean circuit: ij 1 iff (i, j)  E ij 1 iff wire from gate i to gate j type of gate i type of gate j

17. 9/17/2013 An EXP-complete problem Succinct Circuit Value: given a succinctly encoded variable-free Boolean circuit (gates , , , 0, 1), does it output 1? Theorem: Succinct Circuit Value is EXP- complete. Proof: –in EXP (why?) –L arbitrary language in EXP, TM M decides L in 2 n k steps

18. 9/17/2013 An EXP-complete problem –tableau for input x = x 1 x 2 x 3 …x n : –Circuit C from CVAL reduction has size O(2 2n k ). –TM M accepts input x iff circuit outputs 1 x _ _ _ _ _ height, width 2 n k

19. 9/17/2013 An EXP-complete problem –Can encode C succinctly: if i, j within single STEP circuit, easy to compute output if i, j between two STEP circuits, easy to compute output if one of i, j refers to input gates, consult x to compute output ij 1 iff wire from gate i to gate j type of gate i type of gate j

20. 9/17/2013 Summary Remaining TM details: big-oh necessary. First complexity classes: L, P, PSPACE, EXP First separations (via simulation and diagonalization): P ≠ EXP, L ≠ PSPACE First major open questions: L = P P = PSPACE First complete problems: –CVAL is P-complete –Succinct CVAL is EXP-complete ??

21. 9/17/2013 Nondeterminism: introduction A motivating question: Can computers replace mathematicians? L = { (x, 1 k ) : statement x has a proof of length at most k }

22. 9/17/2013 Nondeterminism: introduction Outline: –nondeterminism –nondeterministic time classes –NP, NP-completeness, P vs. NP –coNP –NTIME Hierarchy –Ladner’s Theorem

23. 9/17/2013 Nondeterminism Recall deterministic TM –Q finite set of states –∑ alphabet including blank: “_” –q start, q accept, q reject in Q –transition function: δ : Q x ∑ ! Q x ∑ x {L, R, -}

24. 9/17/2013 Nondeterminism nondeterministic Turing Machine: –Q finite set of states –∑ alphabet including blank: “_” –q start, q accept, q reject in Q –transition relation ∆  (Q x ∑) x (Q x ∑ x {L, R, -}) given current state and symbol scanned, several choices of what to do next.

25. 9/17/2013 Nondeterminism deterministic TM: given current configuration, unique next configuration nondeterministic TM: given current configuration, several possible next configurations q start x 1 x 2 x 3 …x n q accept q reject x  Lx  L

26. 9/17/2013 Nondeterminism asymmetric accept/reject criterion q start x 1 x 2 x 3 …x n q accept q reject x  L x  L “guess” “computation path”

27. 9/17/2013 Nondeterminism all paths terminate time used: maximum length of paths from root space used: maximum # of work tape squares touched on any path from root

28. 9/17/2013 Nondeterminism NTIME(f(n)) = languages decidable by a multi-tape NTM that runs for at most f(n) steps on any computation path, where n is the input length, and f :N ! N NSPACE(f(n)) = languages decidable by a multi-tape NTM that touches at most f(n) squares of its work tapes along any computation path, where n is the input length, and f :N ! N

29. 9/17/2013 Nondeterminism Focus on time classes first: NP =  k NTIME(n k ) NEXP =  k NTIME(2 n k )

30. 9/17/2013 Poly-time verifiers Very useful alternate definition of NP: Theorem: language L is in NP if and only if it is expressible as: L = { x| 9 y, |y| ≤ |x| k, (x, y)  R } where R is a language in P. poly-time TM M R deciding R is a “verifier” “witness” or “certificate” efficiently verifiable

31. 9/17/2013 Poly-time verifiers Example: 3SAT expressible as 3SAT = {φ : φ is a 3-CNF formula for which  assignment A for which (φ, A)  R} R = {(φ, A) : A is a sat. assign. for φ} –satisfying assignment A is a “witness” of the satisfiability of φ (it “certifies” satisfiability of φ) –R is decidable in poly-time

32. 9/17/2013 Poly-time verifiers L = { x | 9 y, |y| ≤ |x| k, (x, y)  R } Proof: (  ) give poly-time NTM deciding L phase 1: “guess” y with |x| k nondeterministic steps phase 2: decide if (x, y)  R

33. 9/17/2013 Poly-time verifiers Proof: (  ) given L  NP, describe L as: L = { x | 9 y, |y| ≤ |x| k, (x, y)  R } –L is decided by NTM M running in time n k –define the language R = { (x, y) : y is an accepting computation history of M on input x} –check: accepting history has length ≤ |x| k –check: R is decidable in polynomial time –check: M accepts x iff  y, |y| ≤ |x| k, (x, y)  R

34. 9/17/2013 Why NP? not a realistic model of computation but, captures important computational feature of many problems: exhaustive search works contains huge number of natural, practical problems many problems have form: L = { x | 9 y s.t. (x,y) 2 R} problem requirements object we are seeking efficient test: does y meet requirements?

35. 9/17/2013 Why NP? Why not EXP? –too strong! –important problems not complete.

36. 9/17/2013 Relationships between classes Easy: P µ NP, EXP µ NEXP –TM special case of NTM Recall: L  NP iff expressible as L = { x | 9 y, |y| · |x| k s.t. (x,y) 2 R} NP µ PSPACE (try all possible y) The central question: P = NP finding a solution vs. recognizing a solution ?

37. 9/17/2013 NP-completeness Circuit SAT: given a Boolean circuit (gates , ,  ), with variables y 1, y 2, …, y m is there some assignment that makes it output 1? Theorem: Circuit SAT is NP-complete. Proof: –clearly in NP

38. 9/17/2013 CIRCUIT-SAT is NP-complete Theorem: CIRCUIT-SAT is NP-complete CIRCUIT-SAT = {C : C is a Boolean circuit for which there exists a satisfying truth assignment} Proof: –Part 1: need to show CIRCUIT-SAT  NP. can express CIRCUIT-SAT as: CIRCUIT-SAT = {C : C is a Boolean circuit for which  x such that (C, x)  R} R = {(C, x) : C is a Boolean circuit and C(x) = 1}

39. 9/17/2013 CIRCUIT-SAT is NP-complete CIRCUIT-SAT = {C : C is a Boolean circuit for which there exists a satisfying truth assignment} Proof: –Part 2: for each language A  NP, need to give poly-time reduction from A to CIRCUIT-SAT –for a given language A  NP, we know A = {x | 9 y, |y| ≤ |x| k, (x, y)  R } and there is a (deterministic) TM M R that decides R in time g(n) ≤ n c for some c.

40. 9/17/2013 CIRCUIT-SAT is NP-complete Tableau (configurations written in an array) for machine M R on input w = (x, y): w1/qsw1/qs w2w2 …wnwn _ … w1w1 w2/q1w2/q1 …wnwn _ … w1/q1w1/q1 a…wnwn _ … _/q a _…__ …............ height = time taken = |w| c width = space used ≤ |w| c

41. 9/17/2013 CIRCUIT-SAT is NP-complete Important observation: contents of cell in tableau determined by 3 others above it: a/q 1 ba b/q 1 a a a aba b

42. 9/17/2013 CIRCUIT-SAT is NP-complete Can build Boolean circuit STEP –input (binary encoding of) 3 cells –output (binary encoding of) 1 cell ab/q 1 a a STEP each output bit is some function of inputs can build circuit for each size is independent of size of tableau

43. 9/17/2013 CIRCUIT-SAT is NP-complete |w| c copies of STEP compute row i from i-1 w1/qsw1/qs w2w2 …wnwn _ … w1w1 w2/q1w2/q1 …wnwn _ …............ Tableau for M R on input w = (x, y) … … STEP

44. 9/17/2013 CIRCUIT-SAT is NP-complete w 1/ q s w2w2 …wnwn _ … STEP............ 1 iff cell contains q accept ignore these This circuit C M R, w has inputs w 1 w 2 …w n and C(w) = 1 iff M R accepts input w. Size = O(|w| 2c ) w1w1 w2w2 wnwn

45. 9/17/2013 CIRCUIT-SAT is NP-complete –recall: we are reducing language A: A = { x | 9 y, |y| ≤ |x| k, (x, y)  R } to CIRCUIT-SAT. –f(x) produces the following circuit: x1x1 x2x2 …xnxn y1y1 y2y2 …ymym Circuit C M R, w 1 iff (x,y)  R – hardwire input x – leave y as variables

46. 9/17/2013 NP-completeness –Given L  NP of form L = { x | 9 y s.t. (x,y) 2 R} x1x1 x2x2 …xnxn y1y1 y2y2 …ymym CVAL reduction for R 1 iff (x,y)  R –hardwire input x; leave y as variables

47. 9/17/2013 NEXP-completeness Succinct Circuit SAT: given a succinctly encoded Boolean circuit (gates , ,  ), with variables y 1, y 2, …, y m is there some assignment that makes it output 1? Theorem: Succinct Circuit SAT is NEXP- complete. Proof: –same trick as for Succinct CVAL EXP- complete.

48. 9/17/2013 Upwards Collapse Theorem P=NP implies EXP=NEXP –Let L be a NEXP-complete language –PAD L = { x# N : x  L, N = 2 |x| k } remains NEXP complete (ignore padding) –PAD L is in NP –So PAD L is in P, and thus, L im EXP –.–.

49. 9/17/2013 Complement classes In general, if C is a complexity class co-C is the complement class, containing all complements of languages in C –L  C implies (  * - L)  co-C –(  * - L)  C implies L  co-C Some classes closed under complement: –e.g. co-P = P

50. 9/17/2013 coNP Is NP closed under complement? q accept q reject x  L x  L q accept q reject x  Lx  L Can we transform this machine: into this machine?

51. 9/17/2013 coNP “proof system” interpretation: Recall: L  NP iff expressible as L = { x | 9 y, |y| ≤ |x| k, (x, y)  R } “proof” “proof verifier” languages in NP have “short proofs” coNP captures (in its complete problems) problems least likely to have “short proofs”. –e.g., UNSAT is coNP-complete

52. 9/17/2013 coNP P = NP implies NP = coNP Belief: NP ≠ coNP –another major open problem

53. 9/17/2013 NTIME Hierarchy Theorem Theorem (Nondeterministic Time Hierarchy Theorem): For every proper complexity function f(n) ≥ n, and g(n) = ω(f(n+1)), NTIME(f(n)) ( NTIME(g(n)).

54. 9/17/2013 NTIME Hierarchy Theorem inputs Y n Y n n Y n YnYYnnY D : Turing Machines (M, x): does NTM M accept x in f(n) steps? Proof attempt : (what’s wrong?)

55. 9/17/2013 NTIME Hierarchy Theorem Let t(n) be large enough so that can decide if NTM M running in time f(n) accepts 1 n, in time t(n). yny???? nyn???? nyy????... 1 t(n) 1n1n MiMi M i-1 M1M1............ D :... I’m responsible for dealing with NTM M i

56. 9/17/2013 NTIME Hierarchy Theorem Enough time on input 1 t(n) to do the opposite of M i (1 n ): y???? MiMi n D :... 1 t(n) 1n1n

57. 9/17/2013 NTIME Hierarchy Theorem For k in [n…t(n)] can to do same as M i (1 k+1 ) on input 1 k y???? MiMi n D :... 1 t(n) 1n1n

58. 9/17/2013 NTIME Hierarchy Theorem Did we diagonalize against M i ? –if L(M i ) = L(D) then: –equality along all arrows. –contradiction. y???? MiMi n D :... 1 t(n) 1n1n

59. 9/17/2013 NTIME Hierarchy Theorem General scheme: –interval [1...t(1)] kills M 1 –interval [t(1)…t(t(1))] kills M 2 –interval [t i-1 (1)…t i (1)] kills M i Running time of D on 1 n : f(n+1) + time to compute interval containing n conclude D in NTIME(g(n)) ( g(n) = ω(f(n+1)) )

60. 9/17/2013 Ladner’s Theorem Assuming P ≠ NP, what does the world (inside NP) look like? NPC P NP: NPC P NP:

61. 9/17/2013 Ladner’s Theorem Theorem (Ladner): If P ≠ NP, then there exists L  NP that is neither in P nor NP- complete. Proof: “lazy diagonalization” –deal with similar problem as in NTIME Hierarchy proof

62. 9/17/2013 Ladner’s Theorem Can enumerate (TMs deciding) all languages in P. –enumerate TMs so that each machine appears infinitely often –add clock to M i so that it runs in at most n i steps

63. 9/17/2013 Ladner’s Theorem Can enumerate (TMs deciding) all NP- complete languages. –enumerate TMs f i computing all polynomial- time functions –machine N i decides language SAT reduces to via f i if f i is reduction, else SAT Check if fi is correct on a logn sized inputs. If error then behave like SAT now you are SAT on all but finitely many inputs.

64. 9/17/2013 Ladner’s Theorem Our goal: L  NP that is neither in P nor NP- complete MiMi M0M0 inputsL N0N0 NiNi :: ::

65. 9/17/2013 Ladner’s Theorem Top half, assuming P ≠ NP: MiMi M0M0 SAT :: input x input z focus on M i for any x, can always find some z ≥ x on which M i and SAT differ (why?)

66. 9/17/2013 Ladner’s Theorem Bottom half, assuming P ≠ NP: TRIV N0N0 NiNi :: input x input z TRIV = Σ * focus on N i for any x, can always find some z ≥ x on which N i and TRIV differ (why?)

67. 9/17/2013 Ladner’s Theorem Try to “merge”: on input x, either –answer SAT(x) –answer TRIV(x) if can decide which one in P, L  NP SATTRIV MiMi M0M0 L N0N0 NiNi :: ::... ≠ ≠ ≠ ≠

68. 9/17/2013 Ladner’s Theorem General scheme: f(n) slowly increasing function –f(|x|) even: answer SAT(x) –f(|x|) odd: answer TRIV(x) notice choice only depends on length of input… that’s OK L... f(|x|) 0001112222... SAT TRIV

69. 9/17/2013 Ladner’s Theorem 1 st attempt to define f(n) “eager f(n)”: increase at 1 st opportunity Inductive definition: f(0) = 0; f(n) = –if f(n-1) = 2i, trying to kill M i if  z < 1 n s.t. M i (z) ≠ SAT(z), then f(n) = f(n-1) + 1; else f(n) = f(n-1) –if f(n-1) = 2i+1, trying to kill N i if  z < 1 n s.t. N i (z) ≠ TRIV(z), then f(n) = f(n-1) + 1; else f(n) = f(n-1)

70. 9/17/2013 Ladner’s Theorem Problem: eager f(n) too difficult to compute on input of length n, –look at all strings z of length < n –compute SAT(z) or N i (z) for each ! Solution: “lazy” f(n) –on input of length n, only run for 2n steps –if enough time to see should increase (over f(n-1)), do it; else, stay same –(alternate proof: T(1)=f(1), T(2)=f(f(1)), –Interval length 1 to length T(1) kills first SAT reduction