1. Complexity Theory Lecture 1 Lecturer: Moni Naor

2. Computational Complexity Theory Study the resources needed to solve computational problems –Computer time –Computer memory –Communication –Parallelism –Randomness –… Identify problems that are infeasible to compute by any reasonable machine Taxonomy: classify problems into classes with similar properties wrt the resource requirements –Help find the most efficient algorithm for a problem

3. Goal of Complexity Theory Given a computational problem – determine the best method of solving it For a given computational setting – determine which computation tasks can be executed State of the art: very far from these goals –Cannot provide unconditional interesting lower bounds for almost any interesting problem –But:…

4. Complexity, Algorithms and Cryptography Complexity Theory has developed a taxonomy allowing to determine for many problems Cryptography Algorithms Complexity

5. Brief History of Complexity Theory Logical foundations 1930s: Computability Theory: –Kurt Gödel –Alan Turing –Alonzo Church 1940s 1950s: work on circuit complexity (Shannon) 1960s: first papers dealing explicitly with complexity issues: –Definitions of polynomial time (Cobham, Edmonds, Peterson) –Time hierarchies, Speedup Theorems, Abstract Complexity Measures Early 1970s: formulation of the P=NP? question, NP-Completeness an industry (Cook, Karp, Levin) Since then – an explosion 1956 Gödel’s letter to von Neumann with the P=NP question Developments in the Soviet Union

6. What will we learn in this course Time and space hierarchies Non-determinism and NP-Completeness Relativization (oracles) Alternation Space Complexity Randomness Interactive Proofs PCP Complexity of counting Concrete Complexity models: –Communication Complexity –Pebbling –Branching Programs Circuit Complexity P, RP, BPP BQP NP, co-NP, PH, #P, PSPACE, IP, AM, PCP NC, AC, L, NL

7. Course Information Complexity Theory - Winter 2004/5 Instructor: Moni Naor Grader: Asaf Nussbaum When: Thursdays, 14:00--17:00 (3 points) Where: Ziskind 1 Course web page : www.wisdom.weizmann.ac.il/~naor/COURSE/complexity.html Prerequisites: familiarity with algorithms, data structures, probability theory, and linear algebra, at an undergraduate level; a basic course in computability is assumed. Requirements: – Homework : There will be around ten homework assignments and a final test. Homework assignments should be turned in on time (usually two weeks after they are given)! Try and do as many problems from each set. You may discuss the problems with other students, but the write-up MUST be individual. You may look up any material, but should cite any external source. – Exam : The exam will be in class.

8. Bibliography No textbook for the course. A lot of background and relevant material is available in Christos Papadimitriou, Computational Complexity, Cambridge, 1994 Michael Sipser, Introduction to the Theory of Computation, 1997 Garey and Johnson. Computers and Intractability: A Guide to the Theory of NP- Completeness. New York: W. H. Freeman, 1979. Two online courses close in nature to our course: Lecturers : Steven Rudich and Avrim Blum, University : CMU Lecturer : Luca Trevisan, Computational Complexity, University : UC Berkeley.

9. Things you should already know Turing Machine –One-tape vs. Multi-tape –Defined by ( ,Q,  )  finite alphabet including blank symbol State space including q start, q accept and q reject transition function  X Q   X Q  X {left,right} –A configuration is represented as a string of symbols from  [ Q  1  2 … q …  m –Multi-tape machine has Read-only input tape Write-only output tape Several working tapes

10. Things you should already know Recognition or decision of language L by a TM M –if M accepts every string in L and rejects all strings not in L –A language L is recursive or decidable if there is a TM M which recognizes it Acceptance of language L by a TM M –if M accepts every string in L and does not accept strings not in L does not necessarily halt on all inputs A language L is recursive enumerable (r.e.) if there is a TM M which accepts it

11. Turing Machines are general Truing Machines can simulate many other models and machines RAM – Random Access Machines –Java.. Circuits – if the circuit itself is uniform Various logical systems Turing machines: the language L u = { |M accepts x} is r.e. – there is a (universal) TM accepting it Simulation are pretty efficient Counterexample: quantum

12. Time and Space Let f:N  N by any function. Time : L 2 TIME(f(n)) if there is a (multi-tape) TM which recognizes L and uses no more than f(|x|) steps on input x Space : L 2 SPACE(f(n)) if there is a (multi-tape) TM which recognizes L and uses no more than f(|x|) cells of its work tapes during the computation on input x –Cell is used if it scanned by the tape head Time and space can be defined for function computation as well No point in accuracy in the time function Theorem (Linear Speedup): for any L 2 TIME(f(n)) and  >0 there is an equivalent TM M’ requiring only  f(n)+n steps Conclusion : lets talk about O(f(n))

13. Proper complexity functions f(n) is a proper complexity function if –f(n)≥ f(n-1) –There is a TM which on input x outputs a string of length exactly f(|x|) while running it time O(|x|+f(|x|)) and space O(f(|x|)) Exercise: show that if f(n) and g(n) are proper complexity functions, then so are – f(n) + g(n) –f(n)g(n) Clocked Simulation Theorem : for every proper complexity function f There exists a TM M f running in time f 3 (|Mx|) that on input ( M,x) accepts iff M(x) accepts within f(|x|) steps Can improve f 3 (n) to f(n) log f(n)

14. Hierarchy Theorems and Diagonalization Diagonalization – proof technique invented by Cantor to prove the uncountability of the real numbers Theorem (Time Hierarchy) : for any proper complexity functions f(n) and g(n) ≥ f(n) log(f(n)) TIME(f(n)) is properly contained in TIME(g(2n)) Proof: Recall M f. Consider L = {<M|M f (MM) rejects} We have L 2 TIME(g(2n)) but L is not in TIME(f(n)) Corollary: TIME(n k ) is properly contained in TIME(n k+1 )

15. Communication Complexity Alice x2Xx2X Bob y2Yy2Y Let f:X x Y  Z Input is split between two participants Want to compute outcome: z=f (x,y) while exchanging as few bits as possible

16. A protocol is defined by the communication tree z 0 z 1 z 2 z 3 z 4 z 5 z 6 z 7... Alice: 0 Bob: 1 Alice: 0 Bob: 1 z5z5

17. A Protocol A protocol P over domain X x Y with range Z is a binary tree where –Each internal node v is labeled with either a v :X  {0,1} or b v :Y  {0,1} –Each leaf is labeled with an element z 2 Z The value of protocol P on input (x,y) is the label of the leaf reached by starting from the root and walking down the tree. At each internal node labeled a v walk – left if a v (x)=0 – right if a v (x)=1 At each internal node labeled b v walk – left if b v (y)=0 – right if b v (y)=1 –The cost of protocol P on input (x,y) is the length of the path taken on input (x,y) –The cost of protocol P is the maximum path length

18. Motivation for studying communication complexity Originally for studying VLSI questions Connection with Turing Machines Data structures and the cell probe model Boolean circuit depth …

19. Communication Complexity of a function For a function f:X x Y  Z the (deterministic) communication complexity of f ( D(f) ) is the minimum cost of protocol P over all protocols that compute f Observation : For any function f:X x Y  Z D(f) ≤ log |X| + log |Z| Example : let x,y µ {1,…,n} and let f(x,y)=max{x [ y} Then D(f) · 2 log n

20. Median let x,y µ {1,…,n} and let MED(x,y) be the median of the multiset x [ y If the size is even then element ranked |x [ y|/2 Claim : D(MED) is O(log 2 n) protocol idea : do a binary search on the value, each party reporting how many are above te current guess Homework : D(MED) is O(log n) protocol idea : each party proposes a candidate See which one is larger - no need to repeat bits

21. Combinatorial Rectangles A combinatorial rectangle in X x Y is a subset R µ X x Y such that R= A x B for some A µ X and B µ Y Proposition: R µ X x Y is a combinatorial rectangle iff (x 1,y 1 ) 2 R and (x 2,y 2 ) 2 R implies that (x 1,y 2 ) 2 R For Protocol P and node v let R v be the set of inputs (x,y) reaching v Claim: For any protocol P and node v the set R v is a combinatorial rectangle Claim: For any given the transcript of an exchange between Alice an Bob possible (but not x and y ) possible to determine z=f(x,y)

22. Fooling Sets For f:X x Y  Z a subset R µ X x Y is f - monochromatic if f is fixed on R Observation: any protocol P induces a partition of X x Y into f - monochromatic rectangles. The number of rectangles is the number of leaves in P A set S µ X x Y is a fooling set for f if there exists a z 2 Z where –For every (x,y) 2 S, f(x,y)=z –For every distinct (x 1,y 1 ), (x 2,y 2 ) 2 S either f(x 1,y 2 )≠z or f(x 2,y 1 )≠z Property: no two elements of a fooling set S can be in the same monochromatic rectangle Lemma: if f has a fooling set of size t, then D(f) ≥ log 2 t

23. Applications Equality : Alice and Bob each hold x,y 2 {0,1} n –want to decide whether x=y or not. Fooling set for Equality S={(w,w)|w 2 {0,1} n } Conclusion : D(Equality) ¸ n Disjointness : let x,y µ {1,…,n} and let –DISJ(x,y)=1 if |x  y| ¸ 1 and –DISJ(x,y)=0 otherwise Fooling set for Disjointness S={(A,comp(A))|A µ {1,…,n} } Conclusion : D(DISJ) ¸ n

24. Applications to Time-Space lower bounds in Turing Machines Let Palindromes={xx R | x 2  *} Palindromes 2 TIME(n) Palindromes 2 SPACE(log n) But what about simultaneous linear time and log space? Lemma : let f: {0,1} n x {0,1} n  {0,1} be a function with communication complexity D(f). For any multi-tape TM running in time T(n) and space S(n) accepting all string {x0 n y| |y|=|x|=n, f(x,y)=1} and rejecting all strings {x0 n y| |y|=|x|=n, f(x,y)=0} We have D(f) = O(T(n) S(n)/n) Proof : joint simulation of the TM by Alice and Bob. To get time-space lower bound on Palindromes use Equality Consider {x0 n x R | x 2  *, |x|=n}

25. Rank lower bound For f:X x Y  {0,1} let M f tbe the |X| x |Y| matrix where entry f(x,y) has value f(x,y). The rank of f, rank(f), is the rank of M f over the reals. Theorem : For any f:X x Y  {0,1} D(f) ≥ log 2 rank(f) Proof: For any protocol P M f =  ℓ leaf M ℓ M ℓ is the matrix corresponding to the rectangle of leaf ℓ Examples: – Equality M Equality is the identity matrix – rank is 2 n

26. Inner Product Let x,y 2 {0,1} n IP(x,y)=  i=1 n x i y i mod 2 What is rank(M IP )? Let N=M IP M IP. Then entry ( x,y) in N is  z 2 {0,1} n which is # z’s where = =1 –2 n-2 if x≠y –2 n-1 if x=y –0 if x or y is 0 Hence rank(N) ¸ 2 n -1. Since rank(M IP M IP ) · min{rank(M IP },rank(M IP }} rank(M IP } ¸ 2 n -1 and D(IP) ¸ n

27. Non-determinism and covers Can a protocol tree be very unbalanced? –Technique for balancing: given a protocol with t leaves there is a protocol with communication complexity O(log t) Is the monotone rectangles lower bound tight? Consider instead of partition into monochromatic rectangles a cover by monochromatic rectangles The rectangles are not necessary disjoint For z 2 Z a z-cover handles only the inputs (x,y) where f(z,y)=z This corresponds to non-deterministic complexity A non-deterministic protocol for verifying that f(x,y)=z: Alice: Guess a rectangle R intersecting row x, send name to Bob Bob: verify that R intersects column y tell Alice Accept only if Bob approve Complexity: log # of z-rectangles