Robbie Hott Department of Computer Science University of Virginia Theory of Computation CS3102 – Spring 2015 A tale of computers, math, problem solving, life, love and tragic death

PSPACE-complete QBF The Extended Chomsky Hierarchy Finite {a,b} Regular a* Det. CF a n b n Context-free ww R PanbncnPanbncn NP PSPACE EXPSPACE Recognizable Not Recognizable H H Decidable Presburger arithmetic NP-complete SAT Not finitely describable  ** EXPTIME EXPTIME-complete Go EXPSPACE-complete =RE  Turing Context sensitive LBA H* degrees

Computability Complexity ~1800s – 1960s 1900: Hilbert’s Problems 1936: Turing’s Computable Numbers 1957: Chomsky’s Syntactic Structures (Mostly) “Dead” field 1960s – 2150? 1960s: Hartmanis and Stearns: Complexity class 1971: Cook/Levin, Karp: P=NP? 1976: Knuth’s O, Ω, Θ Very Open and Alive Decidable Undecidable Intractable Tractable

Resource-Bounded Computation Previously: can something be done? Now: how efficiently can it be done? Goal: conserve computational resources: Time, space, other resources?

Resource-Bounded Computation Def: L is decidable within time O(t(n)) if some TM M that decides L always halts on all w  * within O(t(|w|)) steps / time. Def: L is decidable within space O(s(n)) if some TM M that decides L always halts on all w  * while never using more than O(s(|w|)) space / tape cells.

Complexity Classes Def: DTIME(t(n))={L | L is decidable within time O(t(n)) by some deterministic TM} Def: NTIME(t(n))={L | L is decidable within time O(t(n)) by some non-deterministic TM} Def: DSPACE(s(n))={L | L decidable within space O(s(n)) by some deterministic TM} Def: NSPACE(s(n))={L | L decidable within space O(s(n)) by some non-deterministic TM}

Examples of Space & Time Usage Let L 1 ={0 n 1 n | n>0}: For 1-tape TM’s: L 1  DTIME(n 2 ) L 1  DSPACE(n) L 1  DTIME(n log n) For 2-tape TM’s: L 1  DTIME(n) L 1  DSPACE(log n)

Examples of Space & Time Usage Let L 2 =  * L 2  DTIME(n) Theorem: every regular language is in DTIME(n) L 2  DSPACE(1) Theorem: every regular language is in DSPACE(1) L 2  DTIME(1) Let L 3 ={w$w | w in  *} L 3  DTIME(n 2 ) L 3  DSPACE(n) L 3  DSPACE(log n) Extra Credit!

Multi-Tape vs. One-Tape TM Is one more powerful than the other?

Space is Tape Independent Theorem: The space does not depend on the # tapes. Proof: Note: This does not asymptotically increase the overall space (but can increase the total time). Theorem: A 1-tape TM can simulate a t(n)-time-bounded k-tape TM in time O(k  t 2 (n)). Idea: Tapes can be “interlaced” space-efficiently:

Multi-Tape vs. One-Tape TM Are there problems that are in TIME(t(n)) for a multi-tape TM, but not in TIME(t(n)) for a one-tape TM?

Copy Input Problem Input: w, a string of N bits Output: ww Obvious multi-tape algorithm that involves 2N steps: N steps: walk over the input, copying it to the second tape N steps: continue to move right, copying the second tape contents onto the input tape after the input Best (?) single-tape algorithm that involves ~ 2N 2 steps: N iterations: move over the input, marking each symbol N steps: move to the first non-blank square, write that symbol N steps: move back to the rightmost marked input symbol Intuitively is seems impossible to do much better, but hard to prove!

Theory is about Big Questions If little tweaks to our model change the answers, we might as well focus on answering the practical questions for a real system and specific problem instance instead.

Making things Robustier? Find a more robust computing model than a TM – Church-Turing thesis says all mechanical models are equivalent (computing power) to a TM – But, this doesn ’ t mean there might not be better models for complexity Make the complexity classes bigger – Define a complexity class big enough so the little tweaks to TMs do not change the answers

Complexity Class P P = U k DTIME(n k ) P is the class of languages that can be decided in Polynomial Time on a deterministic, single- tape Turing machine.

Classes in P a) TIME(n 2 ) b) TIME(O(n 7 )) c) TIME(O(2 n )) d)Class of languages that can be decided in Polynomial Time by a 2-tape TM e)Class of languages that can be decided in Polynomial Time by a nondeterministic TM Unknown! This is the P = NP question. Focus of next class… Yes! We can simulate each step of a 2- tape TM by making 2 passes over the whole tape ~ 2(n+t(n)) (See Theorem 7.8)

P Examples What are some examples of problems in P?

Traveling Salesperson Problem You have to visit n cities You want to make the shortest trip How could you do this? What if you had a machine that could guess?

Non-deterministic polynomial time Deterministic Polynomial Time: The TM takes at most O(n c ) steps to accept a string of length n Non-deterministic Polynomial Time: The TM takes at most O(n c ) steps on each computation path to accept a string of length n

Complexity Class NP NP = U k NTIME(n k ) P is the class of languages that can be decided in Polynomial Time on a non-deterministic, single-tape Turing machine.

Complexity Class NP Properties of NP – Easy to verify an answer (verifier in P) – Hard to compute an answer

NP Example Subset Sum Problem – Given a set of integers, determine if a subset sums to 0 Example: {-7, -3, -2, 5, 8, 11, 15} – Verify that {-3, -2, 5} is a solution

NP Example? Traveling Salesperson Problem (decision version) – Given a list of n cities to visit and a distance k, is there a route to visit all cities with distance less than k?

NP Example Traveling Salesperson Problem (decision version) – Even the simpler version is in NP – Verifier: given an ordered list of cities, sum up distance and check < k – Solver: Non-determinism: at each city, “guess” the next correctly. After visiting all, verify cost less than k ( linear algorithm ) Deterministic: check all possible routes ( exponential time )

NP Example? Traveling Salesperson Problem (decision version) – Given a list of n cities to visit and a distance k, is there a route to visit all cities with distance less than k?

NP Example? Traveling Salesperson Problem (decision version) – Given a list of n cities to visit and a distance k, is there a route to visit all cities with distance less than k? Traveling Salesperson Problem – Given a list of n cities to visit and a distance k, find a route to visit all cities with distance less than k?

The Class P and the Class NP P = { L | L is accepted by a deterministic Turing Machine in polynomial time } NP = { L | L is accepted by a non-deterministic Turing Machine in polynomial time } They are sets of languages

Other Special Space Classes Def: PSPACE =  DSPACE(n k )  k>1 PSPACE  deterministic polynomial space Def: NPSPACE =  NSPACE(n k )  k>1 NPSPACE  non-deterministic polynomial space Theorem: PSPACE  NPSPACE (obvious) Theorem: PSPACE  NPSPACE (not obvious)

Space/Time Relationships Theorem: DTIME(f(n))  DSPACE(f(n)) Theorem: DTIME(f(n))  DSPACE(f(n) / log(f(n))) Theorem: NTIME(f(n))  DTIME(c f(n) ) for some c depending on the language. Theorem: DSPACE(f(n))  DTIME(c f(n) ) for some c, depending on the language. Theorem [Savitch]: NSPACE(f(n))  DSPACE(f 2 (n)) Corollary: PSPACE  NPSPACE Theorem: NSPACE(n r )  DSPACE(n r+  )  r>0,  >0

PSPACE-complete QBF The Extended Chomsky Hierarchy Finite {a,b} Regular a* Det. CF a n b n Context-free ww R PanbncnPanbncn NP PSPACE EXPSPACE Recognizable Not Recognizable H H Decidable Presburger arithmetic NP-complete SAT Not finitely describable  ** EXPTIME EXPTIME-complete Go EXPSPACE-complete =RE  Turing Context sensitive LBA H* degrees

Theorem: At least two of the following conjectures are true: LLLL NLPNLP PNPPNP NP  PSPACE PSPACE  EXPTIME EXPTIME  NEXPTIME NEXPTIME  EXPSPACE Open problems! Theorem: P  SPACE(n) Open: P  SPACE(n) ? Open: SPACE(n)  P ? Open: NSPACE(n)  DSPACE(n) ?

P vs NP? Are non-deterministic Turing machines really more powerful (efficient) than deterministic ones? Essence of P vs NP problem

Does Non-Determinism matter? DFA ≈ NFA DFA not ≈ NFA (PDA) Finite Automata? No! Push Down Automata? Yes!

Perelman 2006 ??

Progress P = NP if every NP problem has a deterministic polynomial algorithm We could find an algorithm for every NP problem Seems… hard… We could use polynomial time reductions to find the “ hardest ” problems and just work on those

Reductions Real world examples: – Finding your way around the city reduces to reading a map – Traveling from Richmond to Cville reduces to driving a car – Other suggestions?

Polynomial time reductions PARTITION = { n 1,n 2,… n k | we can split the integers into two sets which sum to half } SUBSET-SUM = { | there exists a subset which sums to m } 1) If I can solve SUBSET-SUM, how can I use that to solve an instance of PARTITION? 2) If I can solve PARTITION, how can I use that to solve an instance of SUBSET-SUM?

Polynomial Reductions 1) Partition REDUCES to Subset-Sum – Partition < p Subset-Sum 2) Subset-Sum REDUCES to Partition – Subset-Sum < p Partition Therefore they are equivalently hard

How long does the reduction take? How could you take advantage of an exponential time reduction?

NP-Completeness How would you define NP-Complete? They are the “ hardest ” problems in NP P NP NP-Complete

Definition of NP-Complete Q is an NP-Complete problem if: 1) Q is in NP 2) every other NP problem polynomial time reducible to Q

Getting Started How do you show that EVERY NP problem reduces to Q? One way would be to already have an NP-Complete problem and just reduce from that P1P1 P2P2 P3P3 P4P4 Mystery NP- Complete Problem Q

SAT SAT = { f | f is a Boolean Formula with a satisfying assignment } Is SAT in NP?

Cook-Levin Theorem (1971) SAT is NP-Complete If you want to see the proof it is Theorem 7.37 in Sipser (assigned reading!) or you can take CS 660 – Graduate Theory. You are not responsible for knowing the proof.

3-SAT 3-SAT = { f | f is in Conjunctive Normal Form, each clause has exactly 3 literals and f is satisfiable } 3-SAT is NP-Complete (2-SAT is in P)

NP-Complete To prove a problem is NP-Complete show a polynomial time reduction from 3-SAT Other NP-Complete Problems: – PARTITION – SUBSET-SUM – CLIQUE – HAMILTONIAN PATH (TSP) – GRAPH COLORING – MINESWEEPER (and many more)

NP-Completeness Proof Method To show that Q is NP-Complete: 1) Show that Q is in NP 2) Pick an instance, R, of your favorite NP- Complete problem (ex: Φ in 3-SAT) 3) Show a polynomial algorithm to transform R into an instance of Q

Savitch’s Theorem Theorem: NSPACE(f(n))  DSPACE (f 2 (n)) Proof: Simulation: idea is to aggressively conserve and reuse space while sacrificing (lots of) time. Consider a sequence of TM states in one branch of an NSPACE(f(n))-bounded computation: Computation time / length is bounded by c f(n) (why?) We need to simulate this branch and all others too! Q: How can we space-efficiently simulate these? A: Use divide-and-conquer with heavy space-reuse! Walter Savitch

Savitch’s Theorem Pick a midpoint state along target path: Verify it is a valid intermediate state by recursively solving both subproblems. Iterate for all possible midpoint states! The recursion stack depth is at most log(c f(n) )=O(f(n)) Each recursion stack frame size is O(f(n)).  total space needed is O(f(n) * f(n))=O(f 2 (n)) Note: total time is exponential (but that’s OK).  non-determinism can be eliminated by squaring the space: NSPACE(f(n))  DSPACE (f 2 (n)) Walter Savitch

Savitch’s Theorem Corollary: NPSPACE = PSPACE Proof:NPSPACE =  NSPACE(n k ) k>1   DSPACE(n 2k ) k>1 =  DSPACE(n k ) k>1 = PSPACE i.e., polynomial space is invariant with respect to non-determinism! Q: What about polynomial time? A: Still open! (P=NP) Walter Savitch