CS151 Complexity Theory Lecture 3 April 6, 2004. CS151 Lecture 32 Introduction A motivating question: Can computers replace mathematicians? L = { (x,

Slides:



Advertisements
Similar presentations
COMPLEXITY THEORY CSci 5403 LECTURE VII: DIAGONALIZATION.
Advertisements

Part VI NP-Hardness. Lecture 23 Whats NP? Hard Problems.
CS151 Complexity Theory Lecture 4 April 11, 2013.
INHERENT LIMITATIONS OF COMPUTER PROGRAMS CSci 4011.
Lecture 16: Relativization Umans Complexity Theory Lecturess.
Fall 2013 CMU CS Computational Complexity Lecture 5 Savich’s theorem, and IS theorems. These slides are mostly a resequencing of Chris Umans’ slides.
FORMAL LANGUAGES, AUTOMATA AND COMPUTABILITY
Theory of Computing Lecture 16 MAS 714 Hartmut Klauck.
The diagonalization method The halting problem is undecidable Decidability.
Complexity 15-1 Complexity Andrei Bulatov Hierarchy Theorem.
Computability and Complexity 22-1 Computability and Complexity Andrei Bulatov Hierarchy Theorem.
Complexity 13-1 Complexity Andrei Bulatov Hierarchy Theorem.
1 Introduction to Computability Theory Lecture12: Decidable Languages Prof. Amos Israeli.
P and NP Sipser (pages ). CS 311 Fall Polynomial time P = ∪ k TIME(n k ) … P = ∪ k TIME(n k ) … TIME(n 3 ) TIME(n 2 ) TIME(n)
1 Introduction to Computability Theory Lecture11: Variants of Turing Machines Prof. Amos Israeli.
1 Introduction to Computability Theory Lecture13: Mapping Reductions Prof. Amos Israeli.
CS151 Complexity Theory Lecture 3 April 6, Nondeterminism: introduction A motivating question: Can computers replace mathematicians? L = { (x,
March 2, 2015CS21 Lecture 231 CS21 Decidability and Tractability Lecture 23 March 2, 2015.
Complexity ©D.Moshkovitz 1 Turing Machines. Complexity ©D.Moshkovitz 2 Motivation Our main goal in this course is to analyze problems and categorize them.
CS151 Complexity Theory Lecture 7 April 20, 2004.
CS151 Complexity Theory Lecture 5 April 13, 2004.
FORMAL LANGUAGES, AUTOMATA AND COMPUTABILITY Read sections 7.1 – 7.3 of the book for next time.
CS151 Complexity Theory Lecture 12 May 6, CS151 Lecture 122 Outline The Polynomial-Time Hierarachy (PH) Complete problems for classes in PH, PSPACE.
NP-Completeness CS 51 Summer 2008 TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: A AA A A AAA.
Computability and Complexity 20-1 Computability and Complexity Andrei Bulatov Class NL.
CS Master – Introduction to the Theory of Computation Jan Maluszynski - HT Lecture NP-Completeness Jan Maluszynski, IDA, 2007
January 28, 2015CS21 Lecture 101 CS21 Decidability and Tractability Lecture 10 January 28, 2015.
Umans Complexity Theory Lectures Lecture 2c: EXP Complete Problem: Padding and succinctness.
February 20, 2015CS21 Lecture 191 CS21 Decidability and Tractability Lecture 19 February 20, 2015.
CS151 Complexity Theory Lecture 2 April 1, CS151 Lecture 22 Time and Space A motivating question: –Boolean formula with n nodes –evaluate using.
1 The Cook-Levin Theorem Zeph Grunschlag. 2 Announcements Last HW due Thursday Please give feedback about course at oracle.seas.columbia.edu/wces oracle.seas.columbia.edu/wces.
Theory of Computing Lecture 19 MAS 714 Hartmut Klauck.
INHERENT LIMITATIONS OF COMPUTER PROGRAMS CSci 4011.
INHERENT LIMITATIONS OF COMPUTER PROGRAMS CSci 4011.
Definition: Let M be a deterministic Turing Machine that halts on all inputs. Space Complexity of M is the function f:N  N, where f(n) is the maximum.
חישוביות וסיבוכיות Computability and Complexity Lecture 7 TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: A AAAA.
February 18, 2015CS21 Lecture 181 CS21 Decidability and Tractability Lecture 18 February 18, 2015.
Computational Complexity Theory Lecture 2: Reductions, NP-completeness, Cook-Levin theorem Indian Institute of Science.
Theory of Computing Lecture 17 MAS 714 Hartmut Klauck.
Fall 2013 CMU CS Computational Complexity Lecture 3 and 4 Non-determinism, NTIME hierarchy threorem, Ladner’s Proof 9/17/2013.
Theory of Computing Lecture 21 MAS 714 Hartmut Klauck.
1 Design and Analysis of Algorithms Yoram Moses Lecture 11 June 3, 2010
Fall 2013 CMU CS Computational Complexity Lectures 8-9 Randomness, communication, complexity of unique solutions These slides are mostly a resequencing.
P Vs NP Turing Machine. Definitions - Turing Machine Turing Machine M has a tape of squares Each Square is capable of storing a symbol from set Γ (including.
Lecture 12 P and NP Introduction to intractability Class P and NP Class NPC (NP-complete)
Computability & Complexity II Chris Umans Caltech.
Umans Complexity Theory Lectures Lecture 1c: Robust Time & Space Classes.
Chapter 11 Introduction to Computational Complexity Copyright © 2011 The McGraw-Hill Companies, Inc. Permission required for reproduction or display. 1.
Fall 2013 CMU CS Computational Complexity Lecture 2 Diagonalization, 9/12/2013.
NP-complete Languages
CSCI 2670 Introduction to Theory of Computing December 2, 2004.
Umans Complexity Theory Lectures Lecture 1b: Turing Machines & Halting Problem.
Complexity ©D.Moshkovitz 1 Our First NP-Complete Problem The Cook-Levin theorem A B C.
The NP class. NP-completeness Lecture2. The NP-class The NP class is a class that contains all the problems that can be decided by a Non-Deterministic.
Recall last lecture and Nondeterministic TMs Ola Svensson.
 2005 SDU Lecture14 Mapping Reducibility, Complexity.
Umans Complexity Theory Lectures Lecture 3c: Nondeterminism: Ladner’s Theorem: If P≠ NP then ∃ NP Languages neither NP Complete nor in P.
Umans Complexity Theory Lectures
CS151 Complexity Theory Lecture 3 April 10, 2017.
CS21 Decidability and Tractability
CS154, Lecture 13: P vs NP.
Recall last lecture and Nondeterministic TMs
CS21 Decidability and Tractability
Umans Complexity Theory Lectures
Umans Complexity Theory Lectures
Instructor: Aaron Roth
Instructor: Aaron Roth
CS151 Complexity Theory Lecture 4 April 12, 2019.
Intro to Theory of Computation
CS151 Complexity Theory Lecture 4 April 8, 2004.
Presentation transcript:

CS151 Complexity Theory Lecture 3 April 6, 2004

CS151 Lecture 32 Introduction A motivating question: Can computers replace mathematicians? L = { (x, 1 k ) : statement x has a proof of length at most k }

April 6, 2004CS151 Lecture 33 Introduction This lecture: –nondeterminism –nondeterministic time classes –NP, NP-completeness, P vs. NP –coNP –NTIME Hierarchy –Ladner’s Theorem

April 6, 2004CS151 Lecture 34 Nondeterminism Recall deterministic TM –Q finite set of states –∑ alphabet including blank: “_” –q start, q accept, q reject in Q –transition function: δ : Q x ∑  Q x ∑ x {L, R, -}

April 6, 2004CS151 Lecture 35 Nondeterminism nondeterministic Turing Machine: –Q finite set of states –∑ alphabet including blank: “_” –q start, q accept, q reject in Q –transition relation ∆  (Q x ∑) x (Q x ∑ x {L, R, -}) given current state and symbol scanned, several choices of what to do next.

April 6, 2004CS151 Lecture 36 Nondeterminism deterministic TM: given current configuration, unique next configuration nondeterministic TM: given current configuration, several possible next configurations q start x 1 x 2 x 3 …x n q accept q reject x  Lx  L

April 6, 2004CS151 Lecture 37 Nondeterminism asymmetric accept/reject criterion q start x 1 x 2 x 3 …x n q accept q reject x  L x  L “guess” “computation path”

April 6, 2004CS151 Lecture 38 Nondeterminism all paths terminate time used: maximum length of paths from root space used: maximum # of work tape squares touched on any path from root

April 6, 2004CS151 Lecture 39 Nondeterminism NTIME(f(n)) = languages decidable by a multi-tape NTM that runs for at most f(n) steps on any computation path, where n is the input length, and f :N  N NSPACE(f(n)) = languages decidable by a multi-tape NTM that touches at most f(n) squares of its work tapes along any computation path, where n is the input length, and f :N  N

April 6, 2004CS151 Lecture 310 Nondeterminism Focus on time classes first: NP =  k NTIME(n k ) NEXP =  k NTIME(2 n k )

April 6, 2004CS151 Lecture 311 Poly-time verifiers Very useful alternate definition of NP: Theorem: language L is in NP if and only if it is expressible as: L = { x |  y, |y| ≤ |x| k, (x, y)  R } where R is a language in P. poly-time TM M R deciding R is a “verifier” “witness” or “certificate” efficiently verifiable

April 6, 2004CS151 Lecture 312 Poly-time verifiers Example: 3SAT expressible as 3SAT = {φ : φ is a 3-CNF formula for which  assignment A for which (φ, A)  R} R = {(φ, A) : A is a sat. assign. for φ} –satisfying assignment A is a “witness” of the satisfiability of φ (it “certifies” satisfiability of φ) –R is decidable in poly-time

April 6, 2004CS151 Lecture 313 Poly-time verifiers L = { x |  y, |y| ≤ |x| k, (x, y)  R } Proof: (  ) give poly-time NTM deciding L phase 1: “guess” y with |x| k nondeterministic steps phase 2: decide if (x, y)  R

April 6, 2004CS151 Lecture 314 Poly-time verifiers Proof: (  ) given L  NP, describe L as: L = { x |  y, |y| ≤ |x| k, (x, y)  R } –L is decided by NTM M running in time n k –define the language R = { (x, y) : y is an accepting computation history of M on input x} –check: accepting history has length ≤ |x| k –check: R is decidable in polynomial time –check: M accepts x iff  y, |y| ≤ |x| k, (x, y)  R

April 6, 2004CS151 Lecture 315 Why NP? not a realistic model of computation but, captures important computational feature of many problems: exhaustive search works contains huge number of natural, practical problems many problems have form: L = { x |  y s.t. (x, y)  R } problem requirements object we are seeking efficient test: does y meet requirements?

April 6, 2004CS151 Lecture 316 Why NP? Why not EXP? –too strong! –important problems not complete.

April 6, 2004CS151 Lecture 317 Relationships between classes Easy: P  NP, EXP  NEXP –TM special case of NTM Recall: L  NP iff expressible as L = { x |  y, |y| ≤ |x| k, (x, y)  R } NP  PSPACE (try all possible y) The central question: P = NP recognizing a solution vs. finding a solution ?

April 6, 2004CS151 Lecture 318 NP-completeness Circuit SAT: given a Boolean circuit (gates , ,  ), with variables y 1, y 2, …, y m is there some assignment that makes it output 1? Theorem: Circuit SAT is NP-complete. Proof: –clearly in NP

April 6, 2004CS151 Lecture 319 NP-completeness –Given L  NP of form L = { x |  y s.t. (x, y)  R } x1x1 x2x2 …xnxn y1y1 y2y2 …ymym CVAL reduction for R 1 iff (x,y)  R –hardwire input x; leave y as variables

April 6, 2004CS151 Lecture 320 NEXP-completeness Succinct Circuit SAT: given a succinctly encoded Boolean circuit (gates , ,  ), with variables y 1, y 2, …, y m is there some assignment that makes it output 1? Theorem: Succinct Circuit SAT is NEXP- complete. Proof: –same trick as for Succinct CVAL EXP- complete.

April 6, 2004CS151 Lecture 321 Complement classes In general, if C is a complexity class co-C is the complement class, containing all complements of languages in C –L  C implies (  * - L)  co-C –(  * - L)  C implies L  co-C Some classes closed under complement: –e.g. co-P = P

April 6, 2004CS151 Lecture 322 coNP Is NP closed under complement? q accept q reject x  L x  L q accept q reject x  Lx  L Can we transform this machine: into this machine?

April 6, 2004CS151 Lecture 323 coNP “proof system” interpretation: Recall: L  NP iff expressible as L = { x |  y, |y| ≤ |x| k, (x, y)  R } “proof” “proof verifier” languages in NP have “short proofs” coNP captures (in its complete problems) problems least likely to have “short proofs”. –e.g., UNSAT is coNP-complete

April 6, 2004CS151 Lecture 324 coNP P = NP implies NP = coNP Belief: NP ≠ coNP –another major open problem

April 6, 2004CS151 Lecture 325 NTIME Hierarchy Theorem Theorem (Nondeterministic Time Hierarchy Theorem): For every proper complexity function f(n) ≥ n, and g(n) = ω(f(n+1)), NTIME(f(n))  NTIME(g(n)).

April 6, 2004CS151 Lecture 326 NTIME Hierarchy Theorem inputs Y n Y n n Y n YnYYnnY H f ’ : Turing Machines (M, x): does NTM M accept x in f(n) steps? Proof attempt #1: (what’s wrong?)

April 6, 2004CS151 Lecture 327 NTIME Hierarchy Theorem Let t(n) be large enough so that can decide if NTM M running in time f(n) accepts 1 n, in time t(n). yny???? nyn???? nyy????... 1 t(n) 1n1n MiMi M i-1 M1M D :... I’m respon- sible for dealing with NTM M i (because I can!)

April 6, 2004CS151 Lecture 328 NTIME Hierarchy Theorem Enough time on input 1 t(n) to do the opposite of M i (1 n ): y???? MiMi n D :... 1 t(n) 1n1n

April 6, 2004CS151 Lecture 329 NTIME Hierarchy Theorem For k in [n…t(n)] can to do same as M i (1 k+1 ) on input 1 k y???? MiMi n D :... 1 t(n) 1n1n

April 6, 2004CS151 Lecture 330 NTIME Hierarchy Theorem Did we diagonalize against M i ? –if M i simulates D then: –equality along all arrows. –contradiction. y???? MiMi n D :... 1 t(n) 1n1n

April 6, 2004CS151 Lecture 331 NTIME Hierarchy Theorem General scheme: –interval [1...t(1)] kills M 1 –interval [t(1)…t(t(1))] kills M 2 –interval [t i-1 (1)…t i (1)] kills M i Running time of D on 1 n : f(n+1) + time to compute interval containing n conclude D in NTIME(g(n))

April 6, 2004CS151 Lecture 332 Ladner’s Theorem Assuming P ≠ NP, what does the world (inside NP) look like? NPC P NP: NPC P NP:

April 6, 2004CS151 Lecture 333 Ladner’s Theorem Theorem (Ladner): If P ≠ NP, then there exists L  NP that is neither in P nor NP- complete. Proof: “lazy diagonalization” –deal with similar problem as in NTIME Hierarchy proof

April 6, 2004CS151 Lecture 334 Ladner’s Theorem Can enumerate (TMs deciding) all languages in P. –enumerate TMs so that each machine appears infinitely often –add clock to M i so that it runs in at most n i steps

April 6, 2004CS151 Lecture 335 Ladner’s Theorem Can enumerate (TMs deciding) all NP- complete languages. –enumerate TMs f i computing all polynomial- time functions –machine N i decides language SAT reduces to via f i if f i is reduction, else SAT (details omitted…)

April 6, 2004CS151 Lecture 336 Ladner’s Theorem Our goal: L  NP that is neither in P nor NP- complete MiMi M0M0 inputsL N0N0 NiNi :: ::

April 6, 2004CS151 Lecture 337 Ladner’s Theorem Top half, assuming P ≠ NP: MiMi M0M0 SAT :: input x input z focus on M i for any x, can always find some z ≥ x on which M i and SAT differ (why?)

April 6, 2004CS151 Lecture 338 Ladner’s Theorem Bottom half, assuming P ≠ NP: TRIV N0N0 NiNi :: input x input z TRIV = Σ * focus on N i for any x, can always find some z ≥ x on which N i and SAT differ (why?)

April 6, 2004CS151 Lecture 339 Ladner’s Theorem Try to “merge”: on input x, either –answer SAT(x) –answer TRIV(x) if can decide which one in P, L  NP SATTRIV MiMi M0M0 L N0N0 NiNi :: ::... ≠ ≠ ≠ ≠

April 6, 2004CS151 Lecture 340 Ladner’s Theorem General scheme: f(n) slowly increasing function –f(|x|) even: answer SAT(x) –f(|x|) odd: answer TRIV(x) notice choice only depends on length of input… that’s OK L... f(|x|) SAT TRIV

April 6, 2004CS151 Lecture 341 Ladner’s Theorem 1 st attempt to define f(n) “eager f(n)”: increase at 1 st opportunity Inductive definition: f(0) = 0; f(n) = –if f(n-1) = 2i, trying to kill M i if  z < 1 n s.t. M i (z) ≠ SAT(z), then f(n) = f(n-1) + 1; else f(n) = f(n-1) –if f(n-1) = 2i+1, trying to kill N i if  z < 1 n s.t. N i (z) ≠ TRIV(z), then f(n) = f(n-1) + 1; else f(n) = f(n-1)

April 6, 2004CS151 Lecture 342 Ladner’s Theorem Problem: eager f(n) too difficult to compute on input of length n, –look at all strings z of length < n –compute SAT(z) or N i (z) for each ! Solution: “lazy” f(n) –on input of length n, only run for 2n steps –if enough time to see should increase (over f(n-1)), do it; else, stay same –(alternate proof: give explicit f(n) that grows slowly enough…)

April 6, 2004CS151 Lecture 343 Ladner’s Theorem Key: n eventually large enough to notice completed previous stage L f(|x|) 0011kkk... MiMi input x input z < x ≠ suppose k = 2i I’m sup- posed to ensure M i is killed I finally have enough time to check input z I notice z did the job, increase f to k+1

April 6, 2004CS151 Lecture 344 Ladner’s Theorem Inductive definition of f(n) –f(0) = 0 –f(n): for n steps compute f(0), f(1), f(2),… L f 0011kkk... input x, |x| = n got this far in n steps

April 6, 2004CS151 Lecture 345 Ladner’s Theorem –if k = 2i: for n steps try (lex order) to find z s.t. SAT(z) ≠ M i (z) and f(|z|) even if found, f(n) = f(n-1)+1 else f(n-1) –if k = 2i + 1: for n steps try (lex order) to find z s.t. TRIV(z) ≠ N i (z) and f(|z|) odd if found, f(n) = f(n-1)+1 else f(n-1)

April 6, 2004CS151 Lecture 346 Ladner’s Theorem Finishing up: L = { x | x  SAT if f(|x|) even, x  TRIV if f(|x|) odd } L  NP since f(|x|) can be computed in O(n) time

April 6, 2004CS151 Lecture 347 Ladner’s Theorem suppose M i decides L –f gets stuck at 2i –L  SAT for z : |z| > n o –implies SAT  P. Contradiction. suppose N i decides L –f gets stuck at 2i+1 –L  TRIV for z : |z| > n o –implies L(N i )  P. Contradiction. (last of diagonalization…)

April 6, 2004CS151 Lecture 348 Summary nondeterminism nondeterministic time classes: NP, coNP, NEXP NTIME Hierarchy Theorem: NP ≠ NEXP major open questions: P = NP NP = coNP ??

April 6, 2004CS151 Lecture 349 Summary NP-“intermediate” problems –technique: delayed diagonalization complete problems: –circuit SAT is NP-complete –UNSAT is coNP-complete –succinct circuit SAT is NEXP-complete

April 6, 2004CS151 Lecture 350 Summary EXP PSPACE P L NEXP NP coNP coNEXP