1. CS151 Complexity Theory Lecture 2 April 1, 2004

2. CS151 Lecture 22 Time and Space A motivating question: –Boolean formula with n nodes –evaluate using O(log n) space?    101 depth-first traversal requires storing intermediate values idea: short-circuit ANDs and ORs when possible

3. April 1, 2004CS151 Lecture 23 Time and Space   10   101  Can we evaluate an n node Boolean circuit using O(log n) space?

4. April 1, 2004CS151 Lecture 24 Time and Space Recall: –TIME(f(n)), SPACE(f(n)) Questions: –how are these classes related to each other? –how do we define robust time and space classes? –what problems are contained in these classes? complete for these classes?

5. April 1, 2004CS151 Lecture 25 Outline Why big-oh? Linear Speedup Theorem Hierarchy Theorems Robust Time and Space Classes Relationships between classes Some complete problems

6. April 1, 2004CS151 Lecture 26 Linear Speedup Theorem: Suppose TM M decides language L in time f(n). Then for any є > 0, there exists TM M’ that decides L in time єf(n) + n + 2. Proof: –simple idea: increase “word length” –M’ will have one more tape than M m-tuples of symbols of M ∑ new = ∑ old  ∑ old m many more states

7. April 1, 2004CS151 Lecture 27 Linear Speedup part 1: compress input onto fresh tape... ababbaaa ababbaaa_

8. April 1, 2004CS151 Lecture 28 Linear Speedup part 2: simulate M, m steps at a time bbaababaaab... abbaababaaababa... mm –4 (L,R,R,L) steps to read relevant symbols, “remember” in state –2 (L,R or R,L) to make M’s changes

9. April 1, 2004CS151 Lecture 29 Linear Speedup accounting: –part 1 (copying): n + 2 steps –part 2 (simulation): 6 (f(n)/m) –set m = 6/є –total: єf(n) + n + 2 Theorem: Suppose TM M decides language L in space f(n). Then for any є > 0, there exists TM M’ that decides L in space єf(n) + 2. Proof: same.

10. April 1, 2004CS151 Lecture 210 Time and Space Moral: big-oh notation necessary given our model of computation –Recall: f(n) = O(g(n)) if there exists c such that f(n) ≤ c g(n) for all sufficiently large n. –TM model incapable of making distinctions between time and space usage that differs by a constant. In general: interested in course distinctions not affected by model –e.g. simulation of k-string TM running in time f(n) by single-string TM running in time O(f(n) 2 )

11. April 1, 2004CS151 Lecture 211 Hierarchy Theorems Does genuinely more time permit us to decide new languages? how can we construct a language L that is not in TIME(f(n))… idea: same as “HALT undecidable” diagonalization and simulation

12. April 1, 2004CS151 Lecture 212 Recall proof for Halting Problem Turing Machines inputs Y n Y n n Y n YnYYnnY H’ : box (M, x): does M halt on x? The existence of H which tells us yes/no for each box allows us to construct a TM H’ that cannot be in the table.

13. April 1, 2004CS151 Lecture 213 Time Hierarchy Theorem Turing Machines inputs Y n Y n n Y n YnYYnnY D : box (M, x): does M accept x in time f(n)? TM SIM tells us yes/no for each box in time g(n) rows include all of TIME(f(n)) construct TM D running in time g(2n) that is not in table

14. April 1, 2004CS151 Lecture 214 Time Hierarchy Theorem Theorem (Time Hierarchy Theorem): For every proper complexity function f(n) ≥ n: TIME(f(n))  TIME(f(2n) 3 ). more on “proper complexity functions” later…

15. April 1, 2004CS151 Lecture 215 Proof of Time Hierarchy Theorem Proof: –SIM is TM deciding language { : M accepts x in ≤ f(|x|) steps } –Claim: SIM runs in time g(n) = f(n) 3. –define new TM D: on input if SIM accepts, reject if SIM rejects, accept –D runs in time g(2n)

16. April 1, 2004CS151 Lecture 216 Proof of Time Hierarchy Theorem Proof (continued): –suppose M in TIME(f(n)) decides L(D) M( ) = SIM( ) ≠ D( ) but M( ) = D( ) –contradiction.

17. April 1, 2004CS151 Lecture 217 Proof of Time Hierarchy Theorem Claim: there is a TM SIM that decides { : M accepts x in ≤ f(|x|) steps} and runs in time g(n) = f(n) 3. Proof sketch: SIM has 4 work tapes contents and “virtual head” positions for M’s tapes M’s transition function and state f(|x|) “+”s used as a clock scratch space

18. April 1, 2004CS151 Lecture 218 Proof of Time Hierarchy Theorem contents and “virtual head” positions for M’s tapes M’s transition function and state f(|x|) “+”s used as a clock scratch space –initialize tapes –simulate step of M, advance head on tape 3; repeat. –can check running time is as claimed. Important detail: need to initialize tape 3 in time O(f(n) + n)

19. April 1, 2004CS151 Lecture 219 Proper Complexity Functions Definition: f is a proper complexity function if –f(n) ≥ f(n-1) for all n –there exists a TM M that outputs exactly f(n) symbols on input 1 n, and runs in time O(f(n) + n) and space O(f(n)).

20. April 1, 2004CS151 Lecture 220 Proper Complexity Functions includes all reasonable functions we will work with –log n, √n, n 2, 2 n, n!, … –if f and g are proper then f + g, fg, f(g), f g, 2 g are all proper can mostly ignore, but be aware it is a genuine concern: Theorem:  non-proper f such that TIME(f(n)) = TIME(2 f(n) ).

21. April 1, 2004CS151 Lecture 221 Hierarchy Theorems Does genuinely more space permit us to decide new languages? Theorem (Space Hierarchy Theorem): For every proper complexity function f(n) ≥ log n: SPACE(f(n))  SPACE(f(n)logf(n)). Proof: same ideas.

22. April 1, 2004CS151 Lecture 222 Robust Time and Space Classes What is meant by “robust” class? –no formal definition –reasonable changes to model of computation shouldn’t change class –should allow “modular composition” – calling subroutine in class (for classes closed under complement…)

23. April 1, 2004CS151 Lecture 223 Robust Time and Space Classes Robust time and space classes: L = SPACE(log n) PSPACE =  k SPACE(n k ) P =  k TIME(n k ) EXP =  k TIME(2 n k )

24. April 1, 2004CS151 Lecture 224 Time and Space Classes Problems in these classes:   101 L : FVAL, integer multiplication, most reductions… PSPACE : generalized geography, 2-person games… pasadena athens auckland san francisco oakland davis

25. April 1, 2004CS151 Lecture 225 Time and Space Classes P : CVAL, linear programming, max- flow…   10   101  EXP : SAT, all of NP and much more…

26. April 1, 2004CS151 Lecture 226 Relationships between classes How are these four classes related to each other? Time Hierarchy Theorem implies P  EXP –P  TIME(2 n )  TIME(2 (2n)3 )  EXP Space Hierarchy Theorem implies L  PSPACE –L=SPACE(log n)  SPACE(log 2 n)  PSPACE

27. April 1, 2004CS151 Lecture 227 Relationships between classes Easy: P  PSPACE L vs. P, PSPACE vs. EXP ?

28. April 1, 2004CS151 Lecture 228 Relationships between classes Useful convention: Turing Machine configurations. Any point in computation represented by string: C = σ 1 σ 2 … σ i q σ i+1 σ i+2 … σ m start configuration for single-tape TM on input x: q start x 1 x 2 …x n σ1σ1... state = q σ2σ2 …σiσi σ i+1 …σmσm

29. April 1, 2004CS151 Lecture 229 Relationships between classes easy to tell if C yields C’ in 1 step configuration graph: nodes are configurations, edge (C, C’) iff C yields C’ in one step # configurations for a 2-tape TM (work tape + read-only input) that runs in space t(n) n x t(n) x |Q| x |∑| f(n) input-tape head position work-tape head position state work-tape contents

30. April 1, 2004CS151 Lecture 230 Relationships between classes if t(n) = c log n, at most n x (c log n) x c 0 x c 1 c log n ≤ n k configurations. can determine if reach q accept or q reject from start configuration by exploring config. graph of size n k (e.g. by DFS) Conclude: L  P

31. April 1, 2004CS151 Lecture 231 Relationships between classes if t(n) = n c, at most n x n c x c 0 x c 1 n c ≤ 2 n k configurations. can determine if reach q accept or q reject from start configuration by exploring config. graph of size 2 n k (e.g. by DFS) Conclude: PSPACE  EXP

32. April 1, 2004CS151 Lecture 232 Relationships between classes So far: L  P  PSPACE  EXP believe all containments strict know L  PSPACE, P  EXP even before any mention of NP, two major unsolved problems: L = P P = PSPACE ??

33. April 1, 2004CS151 Lecture 233 A P-complete problem We don’t know how to prove L ≠ P But, can identify problems in P least likely to be in L using P- completeness. need stronger reduction (why?) yes no yes no L1L1 L2L2 f f

34. April 1, 2004CS151 Lecture 234 A P-complete problem logspace reduction: f computable by TM that uses O(log n) space –denoted “L 1 ≤ L L 2 ” If L 2 is P-complete, then L 2 in L implies L = P (homework problem)

35. April 1, 2004CS151 Lecture 235 A P-complete problem Circuit Value (CVAL): given a variable-free Boolean circuit (gates , , , 0, 1), does it output 1? Theorem: CVAL is P-complete. Proof: –already argued in P –L arbitrary language in P, TM M decides L in n k steps

36. April 1, 2004CS151 Lecture 236 A P-complete problem Tableau (configurations written in an array) for machine M on input w: w1/qsw1/qs w2w2 …wnwn _ … w1w1 w2/q1w2/q1 …wnwn _ … w1/q1w1/q1 a…wnwn _ … _/q a _…__ …............ height = time taken = |w| c width = space used ≤ |w| c

37. April 1, 2004CS151 Lecture 237 A P-complete problem Important observation: contents of cell in tableau determined by 3 others above it: a/q 1 ba b/q 1 a a a aba b

38. April 1, 2004CS151 Lecture 238 A P-complete problem Can build Boolean circuit STEP –input (binary encoding of) 3 cells –output (binary encoding of) 1 cell ab/q 1 a a STEP each output bit is some function of inputs can build circuit for each size is independent of size of tableau

39. April 1, 2004CS151 Lecture 239 A P-complete problem |w| c copies of STEP compute row i from i-1 w1/qsw1/qs w2w2 …wnwn _ … w1w1 w2/q1w2/q1 …wnwn _ …............ Tableau for M on input w … … STEP

40. April 1, 2004CS151 Lecture 240 A P-complete problem w 1/ q s w2w2 …wnwn _ … STEP............ 1 iff cell contains q accept ignore these This circuit C M, w has inputs w 1 w 2 …w n and C(w) = 1 iff M accepts input w. logspace reduction Size = O(|w| 2c ) w1w1 w2w2 wnwn

41. April 1, 2004CS151 Lecture 241 Answer to question   10   101  Can we evaluate an n node Boolean circuit using O(log n) space? NO! (probably) CVAL in P if and only if L = P

42. April 1, 2004CS151 Lecture 242 Padding and succinctness Two consequences of measuring running time as function of input length: “padding” –suppose L  EXP, define PAD L = { x# N : x  L, N = 2 |x| k } –same TM decides L (ignore #s) –running time now polynomial !

43. April 1, 2004CS151 Lecture 243 Padding and succinctness converse: “succinctness” –suppose L is P-complete –intuitively, some inputs are “hard” -- require full power of P –SUCCINCT L = input encoded in exponentially shorter form than L –if “hard” inputs encodable this way, then candidate to be EXP-complete

44. April 1, 2004CS151 Lecture 244 An EXP-complete problem succinct encoding for a directed graph G= (V = {1,2,3,…}, E): a succinct encoding for a variable-free Boolean circuit: ij 1 iff (i, j)  E ij 1 iff wire from gate i to gate j type of gate i type of gate j

45. April 1, 2004CS151 Lecture 245 An EXP-complete problem Succinct Circuit Value: given a succinctly encoded variable-free Boolean circuit (gates , , , 0, 1), does it output 1? Theorem: Succinct Circuit Value is EXP- complete. Proof: –in EXP (why?) –L arbitrary language in EXP, TM M decides L in 2 n k steps

46. April 1, 2004CS151 Lecture 246 An EXP-complete problem –tableau for input x = x 1 x 2 x 3 …x n : –Circuit C from CVAL reduction has size O(2 2n k ). –TM M accepts input x iff circuit outputs 1 x _ _ _ _ _ height, width 2 n k

47. April 1, 2004CS151 Lecture 247 An EXP-complete problem –Can encode C succinctly: if i, j within single STEP circuit, easy to compute output if i, j between two STEP circuits, easy to compute output if one of i, j refers to input gates, consult x to compute output ij 1 iff wire from gate i to gate j type of gate i type of gate j

48. April 1, 2004CS151 Lecture 248 Summary Remaining TM details: big-oh necessary. First complexity classes: L, P, PSPACE, EXP First separations (via simulation and diagonalization): P ≠ EXP, L ≠ PSPACE First major open questions: L = P P = PSPACE First complete problems: –CVAL is P-complete –Succinct CVAL is EXP-complete ??

49. April 1, 2004CS151 Lecture 249 Summary EXP PSPACE P L