CS151 Complexity Theory Lecture 6 April 15, 2015.

Slides:



Advertisements
Similar presentations
COMPLEXITY THEORY CSci 5403 LECTURE XVI: COUNTING PROBLEMS AND RANDOMIZED REDUCTIONS.
Advertisements

Circuit and Communication Complexity. Karchmer – Wigderson Games Given The communication game G f : Alice getss.t. f(x)=1 Bob getss.t. f(y)=0 Goal: Find.
The Communication Complexity of Approximate Set Packing and Covering
1 NP-completeness Lecture 2: Jan P The class of problems that can be solved in polynomial time. e.g. gcd, shortest path, prime, etc. There are many.
The Theory of NP-Completeness
Great Theoretical Ideas in Computer Science for Some.
INHERENT LIMITATIONS OF COMPUTER PROGRAMS CSci 4011.
Having Proofs for Incorrectness
Complexity 26-1 Complexity Andrei Bulatov Interactive Proofs.
1 L is in NP means: There is a language L’ in P and a polynomial p so that L 1 · L 2 means: For some polynomial time computable map r : 8 x: x 2 L 1 iff.
Complexity 18-1 Complexity Andrei Bulatov Probabilistic Algorithms.
CS151 Complexity Theory Lecture 5 April 13, 2015.
February 23, 2015CS21 Lecture 201 CS21 Decidability and Tractability Lecture 20 February 23, 2015.
CS151 Complexity Theory Lecture 7 April 20, 2004.
CS151 Complexity Theory Lecture 5 April 13, 2004.
On the Hardness of Graph Isomorphism Jacobo Tor á n SIAM J. Comput. Vol 33, p , Presenter: Qingwu Yang April, 2006.
CPSC 689: Discrete Algorithms for Mobile and Wireless Systems Spring 2009 Prof. Jennifer Welch.
Arithmetic Hardness vs. Randomness Valentine Kabanets SFU.
CS151 Complexity Theory Lecture 7 April 20, 2015.
Complexity 19-1 Complexity Andrei Bulatov More Probabilistic Algorithms.
1 CSE 417: Algorithms and Computational Complexity Winter 2001 Lecture 23 Instructor: Paul Beame.
Analysis of Algorithms CS 477/677
CSE 421 Algorithms Richard Anderson Lecture 27 NP Completeness.
Chapter 11: Limitations of Algorithmic Power
CS151 Complexity Theory Lecture 15 May 18, CS151 Lecture 152 Outline IP = PSPACE Arthur-Merlin games –classes MA, AM Optimization, Approximation,
CS151 Complexity Theory Lecture 6 April 15, 2004.
Complexity ©D. Moshkovitz 1 And Randomized Computations The Polynomial Hierarchy.
Toward NP-Completeness: Introduction Almost all the algorithms we studies so far were bounded by some polynomial in the size of the input, so we call them.
1 The Theory of NP-Completeness 2012/11/6 P: the class of problems which can be solved by a deterministic polynomial algorithm. NP : the class of decision.
Nattee Niparnan. Easy & Hard Problem What is “difficulty” of problem? Difficult for computer scientist to derive algorithm for the problem? Difficult.
Theory of Computing Lecture 17 MAS 714 Hartmut Klauck.
INHERENT LIMITATIONS OF COMPUTER PROGRAMS CSci 4011.
CS151 Complexity Theory Lecture 13 May 11, Outline proof systems interactive proofs and their power Arthur-Merlin games.
Week 10Complexity of Algorithms1 Hard Computational Problems Some computational problems are hard Despite a numerous attempts we do not know any efficient.
CSE 024: Design & Analysis of Algorithms Chapter 9: NP Completeness Sedgewick Chp:40 David Luebke’s Course Notes / University of Virginia, Computer Science.
1 Lower Bounds Lower bound: an estimate on a minimum amount of work needed to solve a given problem Examples: b number of comparisons needed to find the.
EMIS 8373: Integer Programming NP-Complete Problems updated 21 April 2009.
Graph Colouring L09: Oct 10. This Lecture Graph coloring is another important problem in graph theory. It also has many applications, including the famous.
Umans Complexity Theory Lectures Lecture 1a: Problems and Languages.
1 Chapter 34: NP-Completeness. 2 About this Tutorial What is NP ? How to check if a problem is in NP ? Cook-Levin Theorem Showing one of the most difficult.
Fall 2013 CMU CS Computational Complexity Lectures 8-9 Randomness, communication, complexity of unique solutions These slides are mostly a resequencing.
Umans Complexity Theory Lectures Lecture 7b: Randomization in Communication Complexity.
CS 3343: Analysis of Algorithms Lecture 25: P and NP Some slides courtesy of Carola Wenk.
Computability & Complexity II Chris Umans Caltech.
CSCI 3130: Formal languages and automata theory Andrej Bogdanov The Chinese University of Hong Kong The Cook-Levin.
CSCI-256 Data Structures & Algorithm Analysis Lecture Note: Some slides by Kevin Wayne. Copyright © 2005 Pearson-Addison Wesley. All rights reserved. 29.
CS6045: Advanced Algorithms NP Completeness. NP-Completeness Some problems are intractable: as they grow large, we are unable to solve them in reasonable.
NP-Completeness Note. Some illustrations are taken from (KT) Kleinberg and Tardos. Algorithm Design (DPV)Dasgupta, Papadimitriou, and Vazirani. Algorithms.
Complexity 24-1 Complexity Andrei Bulatov Interactive Proofs.
1 The Theory of NP-Completeness 2 Review: Finding lower bound by problem transformation Problem X reduces to problem Y (X  Y ) iff X can be solved by.
The NP class. NP-completeness Lecture2. The NP-class The NP class is a class that contains all the problems that can be decided by a Non-Deterministic.
The NP class. NP-completeness
NP-Completeness (2) NP-Completeness Graphs 4/13/2018 5:22 AM x x x x x
P & NP.
Topic 36: Zero-Knowledge Proofs
Probabilistic Algorithms
Information Complexity Lower Bounds
Umans Complexity Theory Lectures
NP-Completeness (2) NP-Completeness Graphs 7/23/ :02 PM x x x x
NP-Completeness (2) NP-Completeness Graphs 7/23/ :02 PM x x x x
NP-Completeness Yin Tat Lee
Computability and Complexity
CS 154, Lecture 6: Communication Complexity
NP-Completeness (2) NP-Completeness Graphs 11/23/2018 2:12 PM x x x x
NP-Completeness Yin Tat Lee
CS21 Decidability and Tractability
CS151 Complexity Theory Lecture 7 April 23, 2019.
Instructor: Aaron Roth
Switching Lemmas and Proof Complexity
NP-Completeness (2) NP-Completeness Graphs 7/9/2019 6:12 AM x x x x x
Presentation transcript:

CS151 Complexity Theory Lecture 6 April 15, 2015

CLIQUE = { (G, k) | G is a graph with a clique of size ≥ k } (clique = set of vertices every pair of which are connected by an edge) CLIQUE is NP-complete. April 15, 2015

Circuit lower bounds We think that NP requires exponential-size circuits. Where should we look for a problem to attempt to prove this? Intuition: “hardest problems” – i.e., NP-complete problems April 15, 2015

Circuit lower bounds Formally: if any problem in NP requires super-polynomial size circuits then every NP-complete problem requires super-polynomial size circuits Proof idea: poly-time reductions can be performed by poly-size circuits using a variant of CVAL construction April 15, 2015

such that x  L implies x’  L for all x ¹ x’. Monotone problems Definition: monotone language = language L  {0,1}* such that x  L implies x’  L for all x ¹ x’. flipping a bit of the input from 0 to 1 can only change the output from “no” to “yes” (or not at all) April 15, 2015

Monotone problems some NP-complete languages are monotone e.g. CLIQUE (given as adjacency matrix): others: HAMILTON CYCLE, SET COVER… but not SAT, KNAPSACK… April 15, 2015

Monotone circuits A restricted class of circuits: Definition: monotone circuit = circuit whose gates are ANDs (), ORs (), but no NOTs can compute exactly the monotone fns. monotone functions closed under AND, OR April 15, 2015

Monotone circuits A question: Do all poly-time computable monotone functions have poly-size monotone circuits? recall: true in non-monotone case April 15, 2015

S  V, |S| = k ISCLIQUE(S) Monotone circuits A monotone circuit for CLIQUEn,k Input: graph G = (V,E) as adj. matrix, |V|=n variable xi,j for each possible edge (i,j) ISCLIQUE(S) = monotone circuit that = 1 iff S  V is a clique: i,j  S xi,j CLIQUEn,k computed by monotone circuit: S  V, |S| = k ISCLIQUE(S) April 15, 2015

Monotone circuits Size of this monotone circuit for CLIQUEn,k: when k = n1/4, size is approximately: April 15, 2015

Monotone circuits 2Ω(n1/8). Theorem (Razborov 85): monotone circuits for CLIQUEn,k with k = n1/4 must have size at least 2Ω(n1/8). Proof: rest of lecture April 15, 2015

Proof idea “method of approximation” suppose C is a monotone circuit for CLIQUEn,k build another monotone circuit CC that “approximates” C gate-by-gate  April 15, 2015

Proof idea on test collection of positive/negative instances of CLIQUEn,k: local property: few errors at each gate global property: many errors on test collection Conclude: C has many gates April 15, 2015

Notation i ( j,k  Xi xj,k) * input: graph G = (V, E) variable xj,k for each potential edge (j, k) CC(X1, X2, … Xm), where Xi  V, means: i ( j,k  Xi xj,k) * For example: CC(X1, X2, … Xm) where the Xi range over all k-subsets of V this is the obvious monotone circuit for CLIQUEn,k from a previous slide. *[CC( ) = 0; ( j,k  ; xj,k) = 1] April 15, 2015

Preview approximate circuit CC(X1, X2, … Xm) n = # nodes k = n1/4 = size of clique h = n1/8 = max. size of subsets Xi this is “global property” that ensures lots of errors many graphs G with no k-cliques, but clique on Xi of size h G Xi April 15, 2015

Preview approximate circuit CC(X1, X2, … Xm) p = n1/8log n M = (p – 1)hh! max # of subsets is M (so m ≤ M) critical for “local property” that ensures few errors at each gate April 15, 2015

Building CC CC (“crude circuit”) for circuit C defined inductively as follows: CC for single variable xj,k is just CC({ j, k }) no errors yet! CC for circuit C of form: “approximate OR” of CC for C’, CC for C’’  C’ C’’ April 15, 2015

Building CC  CC for circuit C of form: “approximate AND” of CC for C’, CC for C’’ “approximate OR” and “approximate AND” steps introduce errors  C’ C’’ April 15, 2015

CC(X1,X2,…Xm’) CC(Y1,Y2,…Ym’’) Approximate OR  C’ C’’ CC(X1,X2,…Xm’) CC(Y1,Y2,…Ym’’) exact OR: CC(X1,X2,…Xm’,Y1,Y2,…Ym’’) set sizes still ≤ h may be up to 2M sets; need to reduce to M April 15, 2015

Approximate OR throw away sets? bad:many errors throw away overlapping sets? – better throw away special configuration of overlapping sets – best April 15, 2015

Sunflowers Definition: (h, p)-sunflower is a family of p sets (“petals”) each of size at most h, such that intersection of every pair is a subset S (the “core”). April 15, 2015

Sunflowers Lemma (Erdös-Rado): Every family of more than M = (p-1)hh! sets, each of size at most h, contains an (h, p)-sunflower. Proof: not hard in Papadimitriou, elsewhere April 15, 2015

CC(pluck(X1,X2,…Xm’,Y1,Y2,…Ym’’) ) Approximate OR CC(X1,X2,…Xm’) CC(Y1,Y2,…Ym’’) exact OR: CC(X1,X2,…Xm’,Y1,Y2,…Ym’’) while more than M sets, find (h, p)-sunflower; replace with its core (“pluck”) approximate OR: CC(pluck(X1,X2,…Xm’,Y1,Y2,…Ym’’) )  C’ C’’ April 15, 2015

Approximate AND  CC(X1,X2,…Xm’) CC(Y1,Y2,…Ym’’) (close to) exact AND: CC( {(Xi  Yj) : 1 ≤ i ≤ m’, 1 ≤ j ≤ m’’} ) some sets may be larger than h; discard them may be up to M2 sets. While > M sets, find (h, p)-sunflower; replace with its core (“pluck”) approximate AND: CC( pluck ( {(XiYj) : |XiYj| ≤ h } )) C’ C’’ April 15, 2015

Test collection Positive instances: all graphs G on n nodes with a k-clique and no other edges. G April 15, 2015

Test collection Negative instances: k-1 colors color each node uniformly at random with one of the colors edge (x, y) iff x, y different colors no k-clique include graphs in their multiplicities makes analysis easier (k-1)-partite graph April 15, 2015

“Local” analysis “false positive”: negative example gate is supposed to output 0, but our CC outputs 1 Lemma: each approximation step introduces at most M2(k-1)n/2p false positives. April 15, 2015

“Local” analysis  Proof: case 1: OR CC(X1,X2,…Xm’) CC(Y1,Y2,…Ym’’) CC(pluck(X1,X2,…Xm’,Y1,Y2,…Ym’’)) given “plucking”: replace Z1… Zp with Z bad case: clique on Z, and each petal is missing at least one edge C’ C’’ April 15, 2015

“Local” analysis what is the probability of a repeated color in each Zi but no repeated colors in Z? Pr[R(Z1)R(Z2)…R(Zp) R(Z)] ≤ Pr[R(Z1)R(Z2)…R(Zp)|R(Z)] (definition of conditional probability) = i Pr[R(Zi) | R(Z)] (independent events given no repeats in Z) ≤ i Pr[R(Zi)] (obviously larger) event R(S) = repeated colors in S April 15, 2015

“Local” analysis for every pair of vertices in Zi, probability of same color is 1/(k-1) R(Zi) ≤ (h choose 2)/(k-1) ≤ ½ i Pr[R(Zi)] ≤ (½)p # negative examples is (k-1)n # false positives in given plucking step is at most (½)p(k-1)n at most M plucking steps # false positives at OR ≤ M(½)p(k-1)n April 15, 2015

“Local” analysis  case 2: AND CC(X1,X2,…Xm’) CC(Y1,Y2,…Ym’’) CC(pluck( {(XiYj) : |XiYj| ≤ h } )) discarding sets (XiYj) larger than h can only make circuit accept fewer examples no false positives here C’ C’’ April 15, 2015

“Local” analysis up to M2 pluckings each introduces at most (½)p(k-1)n false positives (previous slides) # false positives at AND ≤ M2(½)p(k-1)n April 15, 2015

“Local” analysis “false negative”: positive example; gate is supposed to output 1, but our CC outputs 0 Lemma: each approximation step introduces at most false negatives. April 15, 2015

“Local” analysis Proof:  Case 1: OR plucking can only make circuit accept more examples no false negatives here. Case 2: AND CC(X1,X2,…Xm’) CC(Y1,Y2,…Ym’’) CC(pluck( {(XiYj) : |XiYj| ≤ h } )) for positive examples: clique on Xi and clique on Yj ) clique on Xi[Yj (no false negatives until discard Xi[Yj sets)  C’ C’’ April 15, 2015

“Local” analysis discarding set Z = (XiYj) larger than h may introduce false negatives any clique that includes Z is a problem; there are at most such positive examples, since |Z|>h at most M2 such deletions we’ve seen plucking doesn’t matter April 15, 2015

R(X) ≤ (h choose 2)/(k-1) ≤ ½ “Global” analysis Lemma: every non-trivial CC outputs 1 on at least ½ of the negative examples. Proof: CC contains some set X of size at most h accepts all neg. examples with different colors in X probability X has repeated colors is R(X) ≤ (h choose 2)/(k-1) ≤ ½ probability over negative examples that CC accepts is at least ½. April 15, 2015

Finishing up First possibility: trivial CC, rejects all positive examples every positive example must have been false negative at some gate number of gates must be at least: April 15, 2015

Finishing up Second possibility: CC accepts at least ½ of negative examples every negative example must have been false positive at some gate number of gates must be at least: April 15, 2015

Both quantities are at least 2Ω(n1/8) Finishing up Both quantities are at least 2Ω(n1/8) April 15, 2015

Conclusions A question (true in non-monotone case): Do all poly-time computable monotone functions have poly-size monotone circuits? if yes, then we would have just proved P ≠ NP why? April 15, 2015

Conclusions unfortunately, answer is no Razborov later showed similar (super-polynomial) lower bound for MATCHING, which is in P… April 15, 2015

Next up… randomness 3 examples of the power of randomness communication complexity polynomial identity testing complexity of finding unique solutions randomized complexity classes April 15, 2015

1. Communication complexity two parties: Alice and Bob function f:{0,1}n x {0,1}n  {0,1} Alice holds x  {0,1}n; Bob holds y  {0,1}n Goal: compute f(x, y) while communicating as few bits as possible between Alice and Bob count number of bits exchanged (computation free) at each step: one party sends bits that are a function of held input and received bits so far April 15, 2015

Communication complexity simple function (equality): EQ(x, y) = 1 iff x = y simple protocol: Alice sends x to Bob (n bits) Bob sends EQ(x, y) to Alice (1 bit) total: n + 1 bits (works for any predicate f) April 15, 2015

Communication complexity Can we do better? deterministic protocol? probabilistic protocol? at each step: one party sends bits that are a function of held input and received bits so far and the result of some coin tosses required to output f(x, y) with high probability over all coin tosses April 15, 2015

Communication complexity Theorem: no deterministic protocol can compute EQ(x, y) while exchanging fewer than n+1 bits. Proof: “input matrix”: Y = {0,1}n X = {0,1}n f(x,y) April 15, 2015

Communication complexity assume 1 bit sent at a time, alternating (same proof works in general setting) A sends 1 bit depending only on x: Y = {0,1}n inputs x causing A to send 1 X = {0,1}n inputs x causing A to send 0 April 15, 2015

Communication complexity B sends 1 bit depending only on y and received bit: Y = {0,1}n inputs y causing B to send 1 X = {0,1}n inputs y causing B to send 0 April 15, 2015

Communication complexity at end of protocol involving k bits of communication, matrix is partitioned into at most 2k combinatorial rectangles bits sent in protocol are the same for every input (x, y) in given rectangle conclude: f(x,y) must be constant on each rectangle April 15, 2015

Communication complexity Y = {0,1}n Matrix for EQ: 1 1 1 X = {0,1}n 1 any partition into combinatorial rectangles with constant f(x,y) must have 2n + 1 rectangles protocol that exchanges ≤ n bits can only create 2n rectangles, so must exchange at least n+1 bits. April 15, 2015

Communication complexity protocol for EQ employing randomness? Alice picks random prime p in {1...4n2}, sends: p (x mod p) Bob sends: (y mod p) players output 1 if and only if: (x mod p) = (y mod p) April 15, 2015

Communication complexity O(log n) bits exchanged if x = y, always correct if x ≠ y, incorrect if and only if: p divides |x – y| # primes in range is ≥ 2n # primes dividing |x – y| is ≤ n probability incorrect ≤ 1/2 Randomness gives an exponential advantage!! April 15, 2015

2. Polynomial identity testing Given: polynomial p(x1, x2, …, xn) as arithmetic formula (fan-out 1): * multiplication (fan-in 2) addition (fan-in 2) negation (fan-in 1) - * * + - x1 x2 x3 … xn April 15, 2015

Polynomial identity testing Question: Is p identically zero? i.e., is p(x) = 0 for all x  Fn (assume |F| larger than degree…) “polynomial identity testing” because given two polynomials p, q, we can check the identity p  q by checking if (p – q)  0 April 15, 2015

Polynomial identity testing try all |F|n inputs? may be exponentially many multiply out symbolically, check that all coefficients are zero? may be exponentially many coefficients can randomness help? i.e., flip coins, allow small probability of wrong answer April 15, 2015

Polynomial identity testing Lemma (Schwartz-Zippel): Let p(x1, x2, …, xn) be a total degree d polynomial over a field F and let S be any subset of F. Then if p is not identically 0, Prr1,r2,…,rnS[ p(r1, r2, …, rn) = 0] ≤ d/|S|. April 15, 2015