CS151 Complexity Theory Lecture 3 April 10, 2017.

Slides:



Advertisements
Similar presentations
Part VI NP-Hardness. Lecture 23 Whats NP? Hard Problems.
Advertisements

Complexity Theory Lecture 6
CS151 Complexity Theory Lecture 4 April 11, 2013.
CS151 Complexity Theory Lecture 3 April 6, CS151 Lecture 32 Introduction A motivating question: Can computers replace mathematicians? L = { (x,
Fall 2013 CMU CS Computational Complexity Lecture 5 Savich’s theorem, and IS theorems. These slides are mostly a resequencing of Chris Umans’ slides.
Theory of Computing Lecture 16 MAS 714 Hartmut Klauck.
The diagonalization method The halting problem is undecidable Decidability.
Complexity 15-1 Complexity Andrei Bulatov Hierarchy Theorem.
Complexity 13-1 Complexity Andrei Bulatov Hierarchy Theorem.
1 Introduction to Computability Theory Lecture12: Decidable Languages Prof. Amos Israeli.
CS151 Complexity Theory Lecture 3 April 6, Nondeterminism: introduction A motivating question: Can computers replace mathematicians? L = { (x,
March 2, 2015CS21 Lecture 231 CS21 Decidability and Tractability Lecture 23 March 2, 2015.
FORMAL LANGUAGES, AUTOMATA AND COMPUTABILITY Read sections 7.1 – 7.3 of the book for next time.
CS151 Complexity Theory Lecture 2 April 3, Time and Space A motivating question: –Boolean formula with n nodes –evaluate using O(log n) space?
CS Master – Introduction to the Theory of Computation Jan Maluszynski - HT Lecture NP-Completeness Jan Maluszynski, IDA, 2007
January 28, 2015CS21 Lecture 101 CS21 Decidability and Tractability Lecture 10 January 28, 2015.
Umans Complexity Theory Lectures Lecture 2c: EXP Complete Problem: Padding and succinctness.
February 20, 2015CS21 Lecture 191 CS21 Decidability and Tractability Lecture 19 February 20, 2015.
CS151 Complexity Theory Lecture 2 April 1, CS151 Lecture 22 Time and Space A motivating question: –Boolean formula with n nodes –evaluate using.
1 The Cook-Levin Theorem Zeph Grunschlag. 2 Announcements Last HW due Thursday Please give feedback about course at oracle.seas.columbia.edu/wces oracle.seas.columbia.edu/wces.
INHERENT LIMITATIONS OF COMPUTER PROGRAMS CSci 4011.
INHERENT LIMITATIONS OF COMPUTER PROGRAMS CSci 4011.
חישוביות וסיבוכיות Computability and Complexity Lecture 7 TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: A AAAA.
February 18, 2015CS21 Lecture 181 CS21 Decidability and Tractability Lecture 18 February 18, 2015.
Computational Complexity Theory Lecture 2: Reductions, NP-completeness, Cook-Levin theorem Indian Institute of Science.
Theory of Computing Lecture 17 MAS 714 Hartmut Klauck.
Fall 2013 CMU CS Computational Complexity Lecture 3 and 4 Non-determinism, NTIME hierarchy threorem, Ladner’s Proof 9/17/2013.
CSCI 3160 Design and Analysis of Algorithms Tutorial 10 Chengyu Lin.
Theory of Computing Lecture 21 MAS 714 Hartmut Klauck.
Umans Complexity Theory Lectures Lecture 1c: Robust Time & Space Classes.
Fall 2013 CMU CS Computational Complexity Lecture 2 Diagonalization, 9/12/2013.
NP-complete Languages
The NP class. NP-completeness Lecture2. The NP-class The NP class is a class that contains all the problems that can be decided by a Non-Deterministic.
 2005 SDU Lecture14 Mapping Reducibility, Complexity.
Umans Complexity Theory Lectures Lecture 3c: Nondeterminism: Ladner’s Theorem: If P≠ NP then ∃ NP Languages neither NP Complete nor in P.
The NP class. NP-completeness
Graphs 4/13/2018 5:25 AM Presentation for use with the textbook, Algorithm Design and Applications, by M. T. Goodrich and R. Tamassia, Wiley, 2015 NP-Completeness.
Umans Complexity Theory Lectures
NP-Completeness NP-Completeness Graphs 5/7/ :49 PM x x x x x x x
Probabilistic Algorithms
Part VI NP-Hardness.
Computational Complexity Theory
Umans Complexity Theory Lectures
CS154, Lecture 7: Turing Machines.
The diagonalization method The halting problem is undecidable
Turing Machines Acceptors; Enumerators
Intro to Theory of Computation
NP-Completeness NP-Completeness Graphs 11/16/2018 2:32 AM x x x x x x
Summary.
CS151 Complexity Theory Lecture 2 April 5, 2017.
Computability & Complexity I
NP-Completeness NP-Completeness Graphs 12/3/2018 2:46 AM x x x x x x x
CS21 Decidability and Tractability
CS21 Decidability and Tractability
CS154, Lecture 13: P vs NP.
Recall last lecture and Nondeterministic TMs
CS21 Decidability and Tractability
Umans Complexity Theory Lectures
CS21 Decidability and Tractability
CS21 Decidability and Tractability
Our First NP-Complete Problem
Umans Complexity Theory Lectures
CS151 Complexity Theory Lecture 1 April 2, 2019.
Instructor: Aaron Roth
Instructor: Aaron Roth
Instructor: Aaron Roth
CS151 Complexity Theory Lecture 5 April 16, 2019.
CS151 Complexity Theory Lecture 4 April 12, 2019.
Intro to Theory of Computation
CS151 Complexity Theory Lecture 4 April 8, 2004.
Presentation transcript:

CS151 Complexity Theory Lecture 3 April 10, 2017

Padding and succinctness Two consequences of measuring running time as function of input length: “padding” suppose L  EXP, and define PADL = { x#N : x  L, N = 2|x|k } TM that decides PADL: ensure suffix of N #s, ignore #s, then simulate TM that decides L running time now polynomial ! April 10, 2017

Padding and succinctness converse (intuition only): “succinctness” suppose L is P-complete intuitively, some inputs are “hard” -- require full power of P SUCCINCTL has inputs encoded in different form than L, some exponentially shorter if “hard” inputs are exponentially shorter, then candidate to be EXP-complete April 10, 2017

Succinct encodings succinct encoding for a directed graph G= (V = {1,2,3,…}, E): a succinct encoding for a variable-free Boolean circuit: 1 iff (i, j)  E i j 1 iff wire from gate i to gate j type of gate i type of gate j i j April 10, 2017

An EXP-complete problem Succinct Circuit Value: given a succinctly encoded variable-free Boolean circuit (gates , , , 0, 1), does it output 1? Theorem: Succinct Circuit Value is EXP-complete. Proof: in EXP (why?) L arbitrary language in EXP, TM M decides L in 2nk steps April 10, 2017

An EXP-complete problem tableau for input x = x1x2x3…xn: Circuit C from CVAL reduction has size O(22nk). TM M accepts input x iff circuit outputs 1 x _ _ _ _ _ height, width 2nk April 10, 2017

An EXP-complete problem Can encode C succinctly: if i, j within single STEP circuit, easy to compute output if i, j between two STEP circuits, easy to compute output if one of i, j refers to input gates, consult x to compute output 1 iff wire from gate i to gate j type of gate i type of gate j i j April 10, 2017

Summary Remaining TM details: big-oh necessary. First complexity classes: L, P, PSPACE, EXP First separations (via simulation and diagonalization): P ≠ EXP, L ≠ PSPACE First major open questions: L = P P = PSPACE First complete problems: CVAL is P-complete Succinct CVAL is EXP-complete ? ? April 10, 2017

Summary EXP PSPACE P L April 10, 2017

Nondeterminism: introduction A motivating question: Can computers replace mathematicians? L = { (x, 1k) : statement x has a proof of length at most k } April 10, 2017

Nondeterminism: introduction Outline: nondeterminism nondeterministic time classes NP, NP-completeness, P vs. NP coNP NTIME Hierarchy Ladner’s Theorem April 10, 2017

Nondeterminism Recall deterministic TM Q finite set of states ∑ alphabet including blank: “_” qstart, qaccept, qreject in Q transition function: δ : Q x ∑ ! Q x ∑ x {L, R, -} April 10, 2017

Nondeterminism nondeterministic Turing Machine: Q finite set of states ∑ alphabet including blank: “_” qstart, qaccept, qreject in Q transition relation ∆  (Q x ∑) x (Q x ∑ x {L, R, -}) given current state and symbol scanned, several choices of what to do next. April 10, 2017

Nondeterminism deterministic TM: given current configuration, unique next configuration qstartx1x2x3…xn qstartx1x2x3…xn x L x  L qaccept qreject nondeterministic TM: given current configuration, several possible next configurations April 10, 2017

Nondeterminism asymmetric accept/reject criterion qstartx1x2x3…xn “computation path” x  L x  L “guess” qaccept qreject asymmetric accept/reject criterion April 10, 2017

Nondeterminism all paths terminate time used: maximum length of paths from root space used: maximum # of work tape squares touched on any path from root April 10, 2017

Nondeterminism NTIME(f(n)) = languages decidable by a multi-tape NTM that runs for at most f(n) steps on any computation path, where n is the input length, and f :N ! N NSPACE(f(n)) = languages decidable by a multi-tape NTM that touches at most f(n) squares of its work tapes along any computation path, where n is the input length, and f :N ! N April 10, 2017

Nondeterminism Focus on time classes first: NP = k NTIME(nk) NEXP = k NTIME(2nk) April 10, 2017

Poly-time verifiers Very useful alternate definition of NP: “witness” or “certificate” Very useful alternate definition of NP: Theorem: language L is in NP if and only if it is expressible as: L = { x| 9 y, |y| ≤ |x|k, (x, y)  R } where R is a language in P. poly-time TM MR deciding R is a “verifier” efficiently verifiable April 10, 2017

R = {(φ, A) : A is a sat. assign. for φ} Poly-time verifiers Example: 3SAT expressible as 3SAT = {φ : φ is a 3-CNF formula for which  assignment A for which (φ, A)  R} R = {(φ, A) : A is a sat. assign. for φ} satisfying assignment A is a “witness” of the satisfiability of φ (it “certifies” satisfiability of φ) R is decidable in poly-time April 10, 2017

L = { x | 9 y, |y| ≤ |x|k, (x, y)  R } Poly-time verifiers L = { x | 9 y, |y| ≤ |x|k, (x, y)  R } Proof: () give poly-time NTM deciding L phase 1: “guess” y with |x|k nondeterministic steps phase 2: decide if (x, y)  R April 10, 2017

Poly-time verifiers Proof: () given L  NP, describe L as: L = { x | 9 y, |y| ≤ |x|k, (x, y)  R } L is decided by NTM M running in time nk define the language R = { (x, y) : y is an accepting computation history of M on input x} check: accepting history has length ≤ |x|k check: R is decidable in polynomial time check: M accepts x iff y, |y| ≤ |x|k, (x, y)  R April 10, 2017

Why NP? not a realistic model of computation problem requirements object we are seeking not a realistic model of computation but, captures important computational feature of many problems: exhaustive search works contains huge number of natural, practical problems many problems have form: L = { x | 9 y s.t. (x,y) 2 R} efficient test: does y meet requirements? April 10, 2017

Why NP? Why not EXP? too strong! important problems not complete. April 10, 2017

Relationships between classes Easy: P µ NP, EXP µ NEXP TM special case of NTM Recall: L  NP iff expressible as L = { x | 9 y, |y| · |x|k s.t. (x,y) 2 R} NP µ PSPACE (try all possible y) The central question: P = NP finding a solution vs. recognizing a solution ? April 10, 2017

NP-completeness Circuit SAT: given a Boolean circuit (gates , , ), with variables y1, y2, …, ym is there some assignment that makes it output 1? Theorem: Circuit SAT is NP-complete. Proof: clearly in NP April 10, 2017

NP-completeness Given L  NP of form L = { x | 9 y s.t. (x,y) 2 R} … xn y1 y2 … ym 1 iff (x,y)  R CVAL reduction for R hardwire input x; leave y as variables April 10, 2017

NEXP-completeness Succinct Circuit SAT: given a succinctly encoded Boolean circuit (gates , , ), with variables y1, y2, …, ym is there some assignment that makes it output 1? Theorem: Succinct Circuit SAT is NEXP-complete. Proof: same trick as for Succinct CVAL EXP-complete. April 10, 2017

Complement classes In general, if C is a complexity class co-C is the complement class, containing all complements of languages in C L  C implies (* - L)  co-C (* - L)  C implies L  co-C Some classes closed under complement: e.g. co-P = P April 10, 2017

coNP Is NP closed under complement? Can we transform this machine: x  L Can we transform this machine: x  L x  L x  L qaccept qreject into this machine? qaccept qreject April 10, 2017

L = { x | 9 y, |y| ≤ |x|k, (x, y)  R } coNP “proof system” interpretation: Recall: L  NP iff expressible as L = { x | 9 y, |y| ≤ |x|k, (x, y)  R } “proof verifier” “proof” languages in NP have “short proofs” coNP captures (in its complete problems) problems least likely to have “short proofs”. e.g., UNSAT is coNP-complete April 10, 2017

coNP NP ≠ coNP P = NP implies NP = coNP Belief: another major open problem April 10, 2017

NTIME Hierarchy Theorem Theorem (Nondeterministic Time Hierarchy Theorem): For every proper complexity function f(n) ≥ n, and g(n) = ω(f(n+1)), NTIME(f(n)) ( NTIME(g(n)). April 10, 2017

NTIME Hierarchy Theorem inputs Proof attempt : (what’s wrong?) Y Turing Machines n (M, x): does NTM M accept x in f(n) steps? Y n n Y n D : n Y n Y Y n Y April 10, 2017

NTIME Hierarchy Theorem Let t(n) be large enough so that can decide if NTM M running in time f(n) accepts 1n, in time t(n). I’m responsible for dealing with NTM Mi 1n 1t(n) . . . M1 y n y ? ? ? ? ... ... Mi-1 n y n ? ? ? ? Mi n y y ? ? ? ? . . . D : April 10, 2017

NTIME Hierarchy Theorem Enough time on input 1t(n) to do the opposite of Mi(1n): 1n 1t(n) . . . Mi y ? ? ? ? D : . . . n April 10, 2017

NTIME Hierarchy Theorem For k in [n…t(n)] can to do same as Mi(1k+1) on input 1k 1n 1t(n) . . . Mi y ? ? ? ? D : . . . n April 10, 2017

NTIME Hierarchy Theorem Did we diagonalize against Mi? if L(Mi) = L(D) then: equality along all arrows. contradiction. 1n 1t(n) . . . Mi y ? ? ? ? D : . . . n April 10, 2017

NTIME Hierarchy Theorem General scheme: interval [1...t(1)] kills M1 interval [t(1)…t(t(1))] kills M2 interval [ti-1(1)…ti(1)] kills Mi Running time of D on 1n: f(n+1) + time to compute interval containing n conclude D in NTIME(g(n)) (g(n) = ω(f(n+1))) April 10, 2017

Ladner’s Theorem Assuming P ≠ NP, what does the world (inside NP) look like? NP: NP: NPC NPC P P April 10, 2017

Ladner’s Theorem Theorem (Ladner): If P ≠ NP, then there exists L  NP that is neither in P nor NP-complete. Proof: “lazy diagonalization” deal with similar problem as in NTIME Hierarchy proof April 10, 2017

Ladner’s Theorem Can enumerate (TMs deciding) all languages in P. enumerate TMs so that each machine appears infinitely often add clock to Mi so that it runs in at most ni steps April 10, 2017

Ladner’s Theorem Can enumerate (TMs deciding) all NP-complete languages. enumerate TMs fi computing all polynomial-time functions machine Ni decides language SAT reduces to via fi if fi is reduction, else SAT (details omitted…) April 10, 2017

Ladner’s Theorem Our goal: L  NP that is neither in P nor NP-complete Mi : : M0 L inputs N0 : : Ni April 10, 2017

Ladner’s Theorem Top half, assuming P ≠ NP: focus on Mi for any x, can always find some z ≥ x on which Mi and SAT differ (why?) Mi : : M0 SAT input x input z April 10, 2017

Ladner’s Theorem Bottom half, assuming P ≠ NP: TRIV = Σ* input z focus on Ni for any x, can always find some z ≥ x on which Ni and TRIV differ (why?) input x TRIV N0 : : Ni April 10, 2017

Ladner’s Theorem Try to “merge”: on input x, either . . . answer SAT(x) answer TRIV(x) if can decide which one in P, L  NP Mi SAT TRIV : : M0 ≠ ≠ . . . L ≠ ≠ N0 : : Ni April 10, 2017

Ladner’s Theorem General scheme: f(n) slowly increasing function f(|x|) even: answer SAT(x) f(|x|) odd: answer TRIV(x) notice choice only depends on length of input… that’s OK . . . SAT L f(|x|) 1 1 1 2 2 2 2 . . . TRIV April 10, 2017

Ladner’s Theorem 1st attempt to define f(n) “eager f(n)”: increase at 1st opportunity Inductive definition: f(0) = 0; f(n) = if f(n-1) = 2i, trying to kill Mi if 9 z < 1n s.t. Mi(z) ≠ SAT(z), then f(n) = f(n-1) + 1; else f(n) = f(n-1) if f(n-1) = 2i+1, trying to kill Ni if 9 z < 1n s.t. Ni(z) ≠ TRIV(z), then f(n) = f(n-1) + 1; else f(n) = f(n-1) April 10, 2017

Ladner’s Theorem Problem: eager f(n) too difficult to compute on input of length n, look at all strings z of length < n compute SAT(z) or Ni(z) for each ! Solution: “lazy” f(n) on input of length n, only run for 2n steps if enough time to see should increase (over f(n-1)), do it; else, stay same (alternate proof: give explicit f(n) that grows slowly enough…) April 10, 2017

Ladner’s Theorem I’m sup-posed to ensure Mi is killed I finally have enough time to check input z I notice z did the job, increase f to k+1 Key: n eventually large enough to notice completed previous stage Mi ≠ . . . L . . . f 1 1 k k k k k input z < x input x suppose k = 2i April 10, 2017

Ladner’s Theorem Inductive definition of f(n) f(0) = 0 f(n): for n steps compute f(0), f(1), f(2),… . . . . . . L . . . . . . f 1 1 k k k got this far in n steps input x, |x| = n April 10, 2017

Ladner’s Theorem if k = 2i: for n steps try (lex order) to find z s.t. SAT(z) ≠ Mi(z) and f(|z|) even if found, f(n) = f(n-1)+1 else f(n-1) if k = 2i + 1: for n steps try (lex order) to find z s.t. TRIV(z) ≠ Ni(z) and f(|z|) odd April 10, 2017

L = { x | x  SAT if f(|x|) even, Ladner’s Theorem Finishing up: L = { x | x  SAT if f(|x|) even, x  TRIV if f(|x|) odd } L  NP since f(|x|) can be computed in O(n) time April 10, 2017

Ladner’s Theorem suppose Mi decides L suppose Ni decides L f gets stuck at 2i L  SAT for z : |z| > no implies SAT  P. Contradiction. suppose Ni decides L f gets stuck at 2i+1 L  TRIV for z : |z| > no implies L(Ni)  P. Contradiction. (last of diagonalization…) April 10, 2017