Complexity Theory Lecture 5 Lecturer: Moni Naor

Recap Last week: Probabilistic Space and Time Complexity Undirected Connectivity is in randomized logspace This week: Probabilistic Complexity 1.Schwatz-Zippel 2.Approximating Counting Problems Plus Alternation

Famous Markov Chain: PageRank algorithm [Brin and Page 98] Good authorities should be pointed by good authorities Random walk on the web graph –pick a page at random –with probability α follow a random outgoing link –with probability 1- α jump to a random page Rank according to the stationary distribution

Hot off the press news Omer Reingold, Undirected ST-Connectivity in Log-Space, Available: Electronic Colloquium on Computational Complexity, Report TR Important Web Resources on Complexity: ECCC: Lance Fortnow’s Computational Complexity Web Log:

Probabilistic Variants of P RP : one-sided Error For all x 2 L we have Pr[M stops with `yes ’]>1/2 For all x  L we have Pr[M stops with `no ’]=1 BPP : two-sided Error For all x 2 L we have Pr[M stops with `yes ’]>2/3 For all x  L we have Pr[M stops with `no ’]>2/3 ZPP : No Error but stopping time is only expected polynomial time Allows Amplication (error reduction) Allows Amplification as long as difference is at least 1/p(n)

The Schwartz-Zippel Algorithm/Theorem Theorem : Let Q(x 1, x 2, …, x n ) 2 F(x 1, x 2, …, x n ) be a non-zero multivariate polynomial of total degree d. Fix any finite set S µ F and choose r 1, r 2, …, r n 2 R S. Then Pr[Q(r 1, r 2, …, r n )=0] · d/|S| Proof : by induction on n Useful when Q is not given explicitly and want to test equality to 0 Determinant of a matrix

Matching in Bipartite Graphs Let G=(V,U,E) b e a bipartite graph, |V|=|U|=n Define A to be the n £ n matrix with |E| variables A ij = x ij if (i,j) 2 E and 0 otherwise Theorem (Edmonds): G has a perfect matching iff det(A)≠0 Proof : det(A)=   2 S n sgn(  ) A 1,  (1) A 1,  (2) … A 1,  (n) If there is no matching always zero If there is a matching cannot be canceled by another term

Algorithm for deciding whether a matching exists in bipartite graphs Fix a prime P larger than 2n Choose {r ij } (i,j) 2 E 2 R GF[P] Compute det(A({r ij } (i,j) )) If non-zero declare matching. Ow, no matching In general computing det(A) more expensive than running combinatorial algorithm for matching Not true in parallel computation

Branching Programs A Branching Program (BP) for on n Boolean variables x 1, x 2, … x n function is a DAG with outdegree 0 or 2. –One source –Each sink is labeled in {0,1} –Each internal node is labeled with a variable x i and the two outgoing edge are labeled with 0 and 1 Given an assignment: computation proceeds by traversing the resulting path to a sink Example: decision trees are all BPs Question: Which functions have polynomial sized BPs? Observation: If a function is in LogSpace, then it has a polynomial sized BP

Types of Branching Programs A read-once BP (ROBP) is one where on every path from source to sink at no variable appears more than once Equivalence problem for BPs: Given two branching programs, do they compute the same function? Homework: for general BPs the equivalence problem is Co-NP Complete Theorem: for read once BPs the equivalence problem is in Co-RP

Arithmetization of BPs For a given BP B define a multivariable polynomial P B over the rationals –Each Boolean variable x i – variable X i –For each path from source to sink labeled 1 add a monomial with variables on the path For occurrence of x i put X i and of : x i put (1-X i ) Claim : for any assignment a 2 {0,1} n we have that B(a) = P B (a) Since each assignment makes only one path non-zero Claim : for any rational assignment q to X 1,X 2,…, X n possible to evaluate P B (q) in polynomial time

Computing P B To evaluate P b at point q=(q 1, q 2, …, q n ) : Assign source value 1 If a node labeled with x i has already been assigned with value v, assign outgoing edges –Edge labeled with 1 gets assigned v q i –Edge labeled with 0 gets assigned v (1-q i ) If all the incoming edges of a node have been assigned: node is assigned their sum

An algorithm for the equivalence of read- once BPs Input B 1 and B 2. Output are they equivalent Pick a subset of 2n values Choose random assignment q to variables Check whether P B 1 (q) = P B 2 (q) Key point: for ROBP P B 1 (X) ≡ P B 2 (X) iff B 1 and B 2 are equivalent Proof: Since P B 1 (X) and P B 2 (X) are multilinear polynomials the Schwartz Zippel Theorem says they will agree with probability at most 1/2

Methods for separating LogSpace from P (and beyond) Prove lower bounds on branching programs If for any problem in P you prove a super polynomial lower bound of the smallest BP for it, then LogSpace ( P If for any problem in P you prove that a short BP implies a lower bound on the size – a time-space tradeoff on almost any imaginable machine BP are the non-uniform variant of log-space There are lower bounds separating read-once from read-twice BP

Probabilistic Approximation Natural usage of randomization: approximating number of solutions: –Pick a random assignment and see what happens Can define class PP: PP : unbounded two-sided Error For all x 2 L we have Pr[M stops with `yes ’]>1/2 For all x  L we have Pr[M stops with `no ’]>1/2 Prove: NP [ Co-NP ½ PP

Counting Problems A counting problem: given an instance x of a decision problem how many witness does it have? Examples – given a formula how many satisfying assignment does it have CNF DNF –Given a graph: How many spanning trees does it have How many perfect matchings How many Hamiltonian Cycles

#P A function f is in #P if there exists a NTM M –running in polynomial time – M(x) has f(x) accepting paths for all x 2 {0,1} n #PAll above problems are in #P #P Claim : NP-Hard to compute all functions in #P #P Claim: for all f 2 FP we have f 2 #P FP=#P Claim: if FP=#P then P=NP

Approximation Schemes for #P Polynomial Randomized approximation scheme ( PRAS ) for a function f in #P: A probabilistic machine A that get x and  and in time polynomial time in |x| returns an answer Pr[(1-  )f(x) · A(x,  ) · (1+  ) f(x)] ¸ 2/3 Fully Polynomial Randomized approximation scheme ( FPRAS ): in time polynomial time in |x| and 1/  Homework : if a function f in has an fpras then there is an algorithm that in addition to x and  get  and –returns a correct approximation with probability at least 1-  –in time polynomial time in |x|, 1/  and log 1/ 

Approximating satisfying assignments to DNFs Given a DNF formula F=D 1 Ç D 2 Ç …, D m count how many satisfying assignments First attempt: For ℓ times –generate an assignment at random and test whether satisfies F –Let Y i =1 if satisfies and 0 otherwise Output Z = 2 n 1/ ℓ  i=1 ℓ Y i Claim E[Z]=#F Claim : if  = #F/2 n and ℓ ¸ 4/(  2  ) log 2/  then approximation is good with probability 1-  Not good enough when  is small

Approximating DNFs Good news: for each D i easy to compute # of sat assignments Second attempt: –Sum up # D i What to do about overlapping assignments? –Idea: assign each satisfying assignment to the lexicographically first clause it satisfies –Lost the ability to compute # of sat assignments exactly Instead: approximate for each D i number of lex first satisfying assignments by – generating random satisfying assignment and testing for lex first Effectively: obtained a problem with  ¸ 1/m

Circuits Let B be a collection of Booleans functions A Circuit for on n Boolean variables x 1, x 2, … x n over Basis B is a DAG with –Each source is labeled with a literal –Unique sink –Each internal node is labeled with a function from B and has the appropriate indegree A circuit compute a function in the natural way Which (families of) functions have circuits whose size is bounded by polynomial in the input size? – B ={ :, Ç, Æ } Claim : all f 2 P have polynomial size circuits Proof: via the `usual’ tableau. Size of circuit T(n)S(n). Using Oblivious Turing Machines T(n) log T(n) Claim : most Boolean function f on n variables do not have polynomial sized circuits For any polynomial p(n) most functions need more p(n) gates Need  (2 n /n) gates to compute all functions on n variables, which is tight Can determine the inputs to a given gate by a log space machine

Polynomial Sized Circuits for BPP Theorem: any f 2 BPP has a polynomial size circuit –There exists a sequence of circuits C 1, C 2, … and a polynomial p(n) such that: circuit C n computes f on inputs of size n circuit C n is of size at most p(n) Several proofs: Construction of hitting set (for f 2 RP ) –Claim that a sequence r 1, r 2, … r n exists where for each x 2 L at least one r i says `yes’ Simulating large sample spaces by small ones Amplification –Reduce the error so that a single assignment will be good for all x

Polynomial Sized Circuits for BPP Theorem: any f 2 BPP has a polynomial size circuit Simulating large sample spaces Want to find a small collection of strings on which the PTM behaves as on the large collection –If the PTM errs with probability at most , then should err on at most  +  of the small collection Choose m random strings For input x event A x is more than (  +  ) of the m strings fail the PTM Pr[A x ] · e -2  2 m < 2 -2n Pr[ [ x A x ] ·  x Pr[A x ] < 2 n 2 -2n =1 Good 1-  Bad  Collection that should resemble probability of success on ALL inputs Chernoff

Alternation Non-determinism: a configuration leads to acceptance if there is an accepting leaf in the computation –Similar to Or Can also consider leads to acceptance if all leaves are accepting –Similar to And What if we alternate between the modes

Alternating Turing Machines An Alternating Turing Machines (ATM) is a non-deterministic TM whose state S ={S AND [ S OR } are partitioned into two sets For input x consider the tree of computations where each node is a configuration The ATM is accepting if the root leads to an accepting configuration: –A leaf is accepting or not –A node in state s 2 S AND leads to acceptance if both its children leads to acceptance –A node in state s 2 S OR leads to acceptance if one of its children leads to acceptance

Alternating Time Classes ATIME(f(n))= class of language decided by an ATM where: All computations halt after at most f(|x|) steps ASPACE(f(n))= class of language decided by an ATM where: All computation halt Never use more than f(|x|) cells AL AL = ASPACE(log n) AL = P Theorem: AL = P Theorem: by simulating circuit Important point: P Important point: if f 2 P then the circuit is constructable in log space Simulate the circuit from the output towards the inputs