Complexity Theory Lecture 3 Lecturer: Moni Naor

Recap Last week: Non deterministic communication complexity Probabilistic communication complexity Their Relationship Lots of techniques This: Space Complexity

Summary of techniques and ideas Probabilistic method : to show that a combinatorial object exists: choose a random one and show that it satisfies desired properties with Prob>0 –Constructive vs. non-constructive methods Union bound: define a collection of bad events A 1, A 2, … A n Pr[ [ i A i ] ·  i Pr[A i ] · n Pr[A i ] Simulating large sample spaces by small ones Reversing roles via Minimax Theorem

Perfect Hash Functions A family H of functions is (n,k,ℓ) perfect if 8 h 2 H h:{1,…,n}  {1,…, ℓ} 8 S ½ {1,…,n} |S|=k 9 h 2 H such that h is 1-1 on S A non-deterministic protocol for k- disjointness using an (n,2k,2k) family H of functions Alice : guess h 2 H and send description hoping that h is perfect for x [ y compute h(i) for all i 2 x and send 2k bit vector C x where C x [j]=1 iff 9 i 2 x such that h(i)=j Bob : compute h(i) for all i 2 y and see whether C x and C y intersect Accept only if do not intersect If |x  y| ¸ 1 always reject. If |x  y| ¸ 0 and h is perfect for x [ y – accept Complexity: log|H| + 2k

Existence of Perfect Hash Families: The Probabilistic Method For a fixed S ½ {1,…,n} of size 2k and choose random h:{1,…,n}  {1,…,2k} Pr[h is perfect for S] = k!/(2k) 2k ¼ e -2k Suppose we choose m random h:{1,…,n}  {1,…,2k} Let event A S be `` no h in the collection is perfect for S” Pr[A S ] · (1- e -2k ) m We are interested in showing Pr[ [ S A S ] < 1 This implies that there is a choice of the that is a perfect family of hash function Pr[ [ S A S ] ·  S Pr[A S ] · ( n 2k ) Pr[A S ] The probabilistic method! Union Bound

The parameters Set m= e 2k log ( n 2k ). Then Pr[A S ] · (1- e -2k ) m · (1- e -2k ) e 2k log ( n 2k ) =1/( n 2k ) This means that communication complexity is 2k +log m = 2k +2k +log (k log n) 2 O(k+log log n) Classical constructions: Fredman, Komlos & Szemeredi More modern one: Rasmus Pagh

Public coins model What if Alice and Bob have access to a joint source of bits. Possible view: distribution over deterministic protocols Let R  pub (f): be the minimum cost of a public coins protocol computing f correctly with probability at least 1-  for any input (x,y) Example: R  pub (Equality)=(log  Theorem : for any Boolean f: R  (f) is R  pub (f)+O(log n + log 1/  Proof : choose t = 8n/  2 assignments to the public string…

Simulating large sample spaces Want to find among all possible public random strings a small collection of strings on which the protocol behave similarly on all inputs Choose m random strings For input (x,y) event A x,y is more than (  +  ) of the m strings fail the protocol Pr[A x,y ] · e -2  2 t < 2 -2n Pr[ [ x,y A x,y ] ·  x,y Pr[A x,y ] < 2 2n 2 -2n =1 Good 1-  Bad  Collection that should resemble probability of success on ALL inputs

Distributional Complexity Let  be a probability distribution on X x Y and  >0. The ( ,  )- distributional complexity of f ( D   (f) ) is the cost of the best deterministic protocol that is correct on 1-  of the inputs weighted by  Theorem : for any f: R  pub (f)=max  D   (f) Protocols of depth d Inputs Is the given protocol correct on the given input Von Neumann’s Minimax Theorem: For all matrices M: max p min q p T M q = min q max p p T M q

Space Complexity: deterministic and non-deterministic Non deterministic TM: transition function  X Q   X Q  X {left,right} may be non-deterministic. The TM is considered to `accept’ if any combination of decisions leads to an accepts state can think of computation as tree and need one accepting leaf Space : L 2 SPACE(f(n)) if there is a TM which recognizes L and –uses no more than f(|x|) cells of its work tapes during the computation on input x Cell is used if it scanned by the tape head Non deterministic Space : L 2 N SPACE(f(n)) if there is a non deterministic TM which recognizes L and –Halts on all branches –Uses no more than f(|x|) cells of its work tapes on any of the branches during the computation on input x Cell is used if it scanned by the tape head

NSPACE and Paths in Graphs NSPACE(f(n) models naturally search in ``implicit” graphs on 2 f(n) nodes Example : the Rush Hour™ Game: given a configuration is it possible to get the red car out Can be solved in NSPACE(n) n=configuration size

Space and Time take I Theorem : if a NTM N always halts and uses space f(n) ¸ log n, then it runs in time 2 O(f(n)) consider C, a configuration of N –Heads positions –Working tape content –Output tape content Except for the input tape– there are 2 O(f(n)) x f(n) o(1) x n possible configurations For any input, in a computation cannot have two equal configurations occurring – –might lead to an infinite branch

Savitch’s Theorem Theorem : for f(n) proper complexity function f(n) ¸ log n NSPACE(f(n)) µ SPACE(f 2 (n)) Cannot hope to consider all possible branchings: There might be 2 f(n) nondeterministic choices in a single branch and cannot store all of them Let L be decided by NTM N in NSPACE(f(n)) –C denotes a configuration of N Heads positions Working tape content Output tape content Define reachable(c 1,c 2,t) to be true iff N can transition from configuration c 1 to c 2 within t steps x 2 L iff reachable(c start, c accept, 2 f(n) )

Solving reachable(c start, c accept, 2 f(n) ) Idea: reachable(c 1,c 2, t) iff there is a midpoint configuration c 3 such that –reachable(c 1,c 3, t/2) and –reachable(c 3,c 2, t/2) To compute both reachable(c 1,c 3, t/2) and reachable(c 1,c 3, t/2) can use the same space (except some accounting)

`Program’ for reachable(c start, c accept, t) reachable on input (c 1, c 2, t) If t=1 test whether c 1 =c 2 or whether N can move from c 1 to c 2 in a single step. If t>1: for each configuration c 3 of N using space f(n) –run reachable(c 1,c 3, t/2) –run reachable(c 3,c 2, t/2) –Accept if both steps accept If exhausted all configuration without accepting `reject’

`Program’ for reachable(c start, c accept, t) reachable on input (c 1, c 2, t) If t=1 test whether c 1 =c 2 or whether N can move from c 1 to c 2 in a single step. If t>1: for each configuration c 3 of N using space f(n) –run reachable(c 1,c 3, t/2) –run reachable(c 3,c 2, t/2) –Accept if both steps accept If exhausted all configuration without accepting `reject’ Runtime stack has depth f(n) with O(f(n)) space per call

PSPACE = NPSPACE Let PSPACE = [ k =1 1 SPACE(n k ) and let NPSPACE = [ k =1 1 NSPACE(n k ) Corollary of Savitch’s Theorem : PSPACE = NPSPACE Open problem: improve the theorem (even by small amount)

Reductions and Completeness Want to convert the problem of deciding A into deciding B Oracle for A – can ask whether x 2 A Oracle Machine with oracle for A: M A Cook-Turing reducible : a language A is Cook-Turing reducible to B ( A · T B ) if there exists a polynomial time M such that M B recognizes A Many-One (Karp Levin) reducible : a language A is Many-One reducible to B ( A · M B ) if there exists a function F computable in polynomial time such that x 2 A iff F(x) 2 B Log space reducible : a language A is Many-One reducible to B ( A · L B ) if there exists a function F 2 FSPACE(log n) such that x 2 A iff F(x) 2 B Property : for all A and B: A · L B ) A · M B ) A · T B Property : all reducibility notions are transitive

Log-space reductions Input Tape Output Tape Work Tape Read only Write only Log Space for read and write This is what we count Closed under composition! Do not write the output explicitly. Instead consider G’(x,i)= ith bit of G(X)

PSPACE Completeness Definition : Language L is PSPACE-complete iff: 1. L is in PSPACE 2. Every language L’ in PSPACE is reducible to L via a polynomial time reduction (Cook Turing) Complete problems are the `hardest’ problems in a complexity class They represent it Reduction must be easy relative to the complexity class Property : If L is PSPACE-complete and L is in P, then P = PSPACE

The TQBF Problem Totally Quantified Boolean Formula Boolean formula: expression containing Boolean variables x i Constants 0 and 1 Boolean operations: Ç, Æ, : In a Quantified Boolean formula in addition quantifiers: Universal (for all) 8 –The statement 8 x  means: for every value of x the resulting statement  is true Existential (there exists) 9 –The statement 9 x  means: for some value of x the resulting statement  is true Can assume all quantifiers appear at the beginning of the formula (prenex normal form) If each variable appears within the scope of some quantifier – fully quantified Statement always either true or false TQBF = {  :  is a true fully quantified Boolean formula}

Theorem: TQBF is PSPACE Complete TQBF is in PSPACE: to see if  is in TQBF, If contains no quantifiers – expression with only constants –can evaluate it. If  = 9 x  recursively –set x=0 and check  –set x=1 and check  –If either one answers `true’ answer `true’. Otherwise `false’ If  = 8 x  recursively –set x=0 and check  –set x=1 and check  –If both answer `true’ answer `true’ ’. Otherwise `false’ Reuse space!

TQBF is PSPACE Complete Every language L in PSPACE is reducible to TQBF via a polynomial time reduction Suppose NTM N decides L is space n k Goal: Given (N,w), construct a Boolean formula  such that:  is in TQBF, N has an accepting path on w size of  should be n k’

Given (N,w), consider a 2 n k by n k tableau similar to that of Cook’s theorem N ’’s configuration time c 1,1 c 1,2 c 1,3 c 1,4 … c 1,n k c 2,1 c 2,2 c 2,3 c 2,4 … c 2,n k. c n k,, 1 c n k,2 c n k,3 c n k,4 … c n k,n k 122nk122nk

N ’s configuration time # q 0 w 1 w 2 … w n U U … # # c 2,2 c 2,3 c 2,4 … c 2,n k 1 # # c 3,2 c 3,3 c 3,4 … c 3,n k 1 # # any accept configuration# If N accepts w, then there is a legal tableau that looks like this: 122nk122nk Current state and implicit location of the tape head …TQBF is PSPACE Complete

Want  c 1,c 2,t which is in TQBF iff N can transition between configurations c 1 and c 2 within at most t steps Construct  c start,c accept, 2 n k recursively: First try: guess c 3 as mid point:  c 1,c 2,t = 9 c 3 :  c 1,c 3,t/2 Æ  c 3,c 2,t/2 Exponential size formula But only used existential quantifiers!  c 1,c 2,t = 9 c 3 8 (c 4,c 5 ) 2 {(c 1,c 3 ), (c 3,c 2 )}:  c 4,c 5,t/2

PSPACE Completeness Many problems proved to be PSPACE Complete –In particular games Games closely related to quantifiers: The formula game Let  = 9 x 1 8 x 2 9 x 3 … 8 x n [  ] be a prenex formula Player A and Player E take turns in selecting values for the variables –A for the 8 bound variables and –E for the 9 bound variables –At the end Player E wins if  is TRUE and A wins if  is FALSE Formula-game= {  : Player E has a winning strategy in the formula game played on  } Theorem : Formula-game is PSPACE Complete

Game of Go To players alternate placing black and white pebbles Objective: create large safe groups of your color –Monochromatic connected subgraph of the grid and capture as many pebbles of the other color by surrounding them Theorem : deciding whether a given configuration is a win for white is PSPACE-Complete

Pebbling a DAG Rules Inputs: a pebble can be place on input (source) vertex at any time Computation : a pebble can be placed on any non-input vertex if all its immediate predecessor have pebbles A pebble can be removed at any time Goal : find a strategy to pebble all the outputs while using a few pebbles and a few steps Theorem : Minimizing pebbles is PSPACE-complete Gilbert, Lengaur, Tarjan, Sicomp 1980 Sources Outputs Models computation with registers

Homework on Pebbling Depth of a DAG : distance of the input node and output node that are furthest apart (if they are connected). Let d be the maximum indegree Show that any DAG of depth c and can be pebbled using O(c d ) pebbles. Show that any tree can be pebbled using O(d log n) pebbles.

Generalized Rush Hour Given a rush hour configuration can you get the special car out? Recently proved theorem: Rush Hour is PSPACE-Complete (Baum and Flake) Implications: Hard to solve Even if you invested/wasted time and solved it, how do you convince your friend that it is possible ? How do you demonstrate that there is no solution? Homework : suggest a probabilistic protocol for verifying that a solution exists. –The prover may be willing to work hard, but not the verifier. –the probability of catching a cheating prover is non-zero (but might be very small) for any cheater –If the prover is not cheating then the verifier is always convinced by the proof.

L and NL Log-Space = SPACE(log(n)) NL = NSPACE(log(n)) Log(n) space for work Input is read-only

Log-space reductions Input Tape Output Tape Work Tape Read only Write only Log Space for read and write This is what we count Closed under composition! Do not write the output explicitly. Instead consider G’(x,i)= ith bit of G(X)

NL-completeness Language L is NL-complete iff 1.L is in NL 2.Every language L’ in NL is log-space reducible to L PATH = {(G,s,t)| G is a directed graph with a path that starts at vertex s and ends at vertex t } –How is the graph given (adjacency matrix, list of outgoing edges…? Theorem : PATH is NL-complete What about undirected connectivity?

Sublogarithmic Space Are there interesting languages requiring less than logarithmic space? –Regular languages –Other? A  (log log n) language: {x=b 0 $ b 2 $ … $ b k | b i =binary representation of i} Algorithm: –Check the first and last blocks –Check that each block is +1 from previous To avoid using too much space on bogus strings: check for i=1.. that the last i bits cycle through all possibilities

No languages between regular and O(log log n) Theorem : For any f(n) 2 o(log log n) Space(f(n)) = Space(O(1)) Proof : Configuration is –i position on tape –c content of working tapes heads and direction No configuration can repeat itself Ow. Machine does not stop For word x let C(x,i) = c 1, c 2, … c r sequence of configurations machine passes through on position i Number of different C(x,i) ~ 2 f(n)2 f(n) < n for sufficiently large n Let x S be shortest word requiring more than S space Idea: if there is 1 · i<j · n such that C(x S,i) = C(x S,j) then can cut it out resulting in shorter string with similar behavior C1C1 C2C2 C3C3

Is sublogarithmic space interesting No equivalent RAM Not clear that it models easiness/simplicity –Unlike regular languages Other models for simple languages –Parallel complexity

Non-deterministic space is closed under complementation The Immerman-Szelepcsenyi Theorem For simplicity start with NL=Co-NL States of NTM correspond to directed graph –Problem: how to prove non-reachability Idea: to prove that a room is empty show the whereabouts of all potential occupiers Show how to count in NL –Sufficient for non-reachability: can make sure graph is acyclic For directed graph G=(V,E) let R(G,s,i) = {u| reachable(s,u,i) } Main Lemma : there exists a log space non deterministic machine for computing |R(G,s,i)| –The machine never outputs a wrong answer but may output `fail’ u is reachable from s in i moves in G

NL Algorithm for |R(G,s,i)| given |R(G,s,i-1)| Simple: go over all nodes u=1,2 … n : if reachable(s,u,i) increase counter How to determine reachable(s,u,i) with no errors either way? NL Algorithm for determining reachable(s,u,i) Input: |R(G,s,i-1)| Output: `yes’, `no’, `fail’ go over nodes v=1,2 … n –if reachable(s,v,i-1): m := m+1 If (v,u) 2 G output `yes’ If m < |R(G,s,i-1)| output `fail’ –Else: output `no’ Not a recursive call! Guess the path and verify

Wrapping up: For i=1,2 … n-1 compute |R(G,s,i)| –If ever the output is `fail’ then `fail’ Each call requires simply |R(G,s,i-1)| –same log n bits Knowing |R(G,s,n)| allows us to determine whether reachable(s,accept,n) with no errors Theorem : If f(n) ¸ log n is a proper complexity function, then NSPACE(f(n)) = Co-NSPACE(f(n))