1. Nathan Brunelle Department of Computer Science University of Virginia www.cs.virginia.edu/~njb2b/theory Theory of Computation CS3102 – Spring 2014 A tale of computers, math, problem solving, life, love and tragic death

2. I’ll be Absent Next Week (But you shouldn’t be) Tuesday: Jack Wadden Micron Automata Processor Thursday: Robbie Hott P and NP

3. Oracles Originated in Turing’s Ph.D. thesis Named after the “Oracle of Apollo” at Delphi, ancient Greece Black-box subroutine / language Can compute arbitrary functions Instant computations “for free” Can greatly increase computation power of basic TMs E.g., oracle for halting problem

4. A special case of “hyper-computation” Allows “what if” analysis: assumes certain undecidable languages can be recognized An oracle can profoundly impact the decidability & tractability of a language Any language / problem can be “relativized” WRT an arbitrary oracle Undecidability / intractability exists even for oracle machines! Turing Machines with Oracles Theorem [Turing]: Some problems are still not computable, even by Turing machines with an oracle for the halting problem!

5. Reaching a Contradiction What happens if we run D on its own description, ? Define D ( ) = Construct a TM that: Outputs the opposite of the result of simulating H on input (M, ) Assume there exists some TM H that decides A TM. If M accepts its own description, D( ) rejects. If M rejects its own description, D( ) accepts. If D accepts its own description, D( ) rejects. If D rejects its own description, D( ) accepts. substituting D for M… O is a A TM “Oracle” M* is Relativized to O H* is Relativized to O D* is Relativized to O * * * * * * * * * * * * * * * * * * * * * *

6. Ø Turing (1937); studied by Post (1944) and Kleene (1954) Quantifies the non-computability (i.e., algorithmic unsolvability) of (decision) problems and languages Some problems are “more unsolvable” than others! Turing Degrees Emil Post 1897-1954 Alan Turing 1912-1954 Stephen Kleene 1909-1994 Students of Alonzo Church: H H H*H* Turing degree 0 Turing degree 1 Turing degree 2 Defines computation “relative” to an oracle. “Relativized computation” - an infinite hierarchy! A “relativity theory of computation”! Georg Cantor 1845-1918

7. Turing degree of a set X is the set of all Turing-equivalent (i.e., mutually-reducible) sets: an equivalence class [X] Turing degrees form a partial order / join-semilattice [0]: the unique Turing degree containing all computable sets For set X, the “Turing jump” operator X’ is the set of indices of oracle TMs which halt when using X as an oracle [0’]: Turing degree of the halting problem H; [0’’]: Turing degree of the halting problem H* for TMs with oracle H. Turing Degrees Emil Post 1897-1954 Alan Turing 1912-1954 Stephen Kleene 1909-1994 Students of Alonzo Church: Turing jump Turing jump

8. Turing Degrees Emil Post 1897-1954 Alan Turing 1912-1954 Stephen Kleene 1909-1994 Students of Alonzo Church: Turing jump Turing jump Each Turing degree is countably infinite (has exactly  0 sets) There are uncountably many (2  0 ) Turing degrees A Turing degree X is strictly smaller than its Turing jump X’ For a Turing degree X, the set of degrees smaller than X is countable; set of degrees larger than X is uncountable (2  0 ) For every Turing degree X there is an incomparable degree (i.e., neither X  Y nor Y  X holds). There are 2  0 pairwise incomparable Turing degrees For every degree X, there is a degree D strictly between X and X’ so that X < D < X’ (there are actually  0 of them) The structure of the Turing degrees semilattice is extremely complex!

10. PSPACE-complete QBF The Extended Chomsky Hierarchy Finite {a,b} Regular a* Det. CF a n b n Context-free ww R PanbncnPanbncn NP PSPACE EXPSPACE Recognizable Not Recognizable H H Decidable Presburger arithmetic NP-complete SAT Not finitely describable  ** EXPTIME EXPTIME-complete Go EXPSPACE-complete =RE  Turing Context sensitive LBA H* degrees

11. Complexity Classes 9*9=81 99*99*99= 970299 999*999*999*999= 996005996001 =10 12 Number of particles in the universe: about a googol (10 100 ) After just 10 iterations: 10 110 The scientists owe Ali G an apology

12. Today - End Decidable Undecidable Tractable: “Decidable in a reasonable amount of time and space”

13. Computability Complexity ~1800s – 1960s 1900: Hilbert’s Problems 1936: Turing’s Computable Numbers 1957: Chomsky’s Syntactic Structures (Mostly) “Dead” field 1960s – 2150? 1960s: Hartmanis and Stearns: Complexity class 1971: Cook/Levin, Karp: P=NP? 1976: Knuth’s O, Ω, Θ Very Open and Alive Decidable Undecidable Intractable Tractable

14. Complexity Classes Computability Classes: sets of problems (languages) that can be solved (decided/recognized) by a given machine Complexity Classes: sets of problems (languages) that can be solved (decided) by a given machine (usually a TM) within a limited amount of time or space How many complexity classes are there? Infinitely many! “Languages that can be decided by some TM using less than 37 steps” is a complexity class

15. Order Notation O( f ),  ( f ), o( f ),  ( f ) These notations define sets of functions – Generally: functions from positive integer to real We are interested in how the size of the outputs relates to the size of the inputs

16. Big O Intuition: the set O(f) is the set of functions that grow no faster than f – More formal definition coming soon Asymptotic growth rate – As input to f approaches infinity, how fast does value of f increase – Hence, only the fastest-growing term in f matters: O(12n 2 + n)  O(n 3 ) O(n)  O(63n + log n – 423)

17. Formal Definition f  O (g) means: There are positive constants c and n 0 such that f(n)  cg(n) for all values n  n 0.

18. O Examples x  O (x 2 ) ? Yes, c = 1, n 0 =2 works fine. 10x  O (x) ? Yes, c = 11, n 0 =2 works fine. x 2  O (x) ? No, no matter what c and n 0 we pick, cx 2 > x for big enough x f (n)  O (g (n)) means: there are positive constants c and n 0 such that f(n)  cg(n) for all values n  n 0.

19. Lower Bound:  (Omega) f(n) is  (g (n)) means: There are positive constants c and n 0 such that f (n)  cg(n) for all n  n 0. Difference from O, this was 

20. Theta (“Order of”)

21. Summary Big-O: there exist c, n 0 > 0 such that f(n)  cg(n) for all n  n 0. Omega (  ): there exist c, n 0 > 0 s.t. f(n)  cg(n) for all n  n 0. Theta (  ): both O and  are true When you were encouraged to use Big-O in cs201/cs216 to analyze the running time of algorithms, what should you have been using?

22. Resource-Bounded Computation Previously: can something be done? Now: how efficiently can it be done? Goal: conserve computational resources: Time, space, other resources?

23. Resource-Bounded Computation Def: L is decidable within time O(t(n)) if some TM M that decides L always halts on all w  * within O(t(|w|)) steps / time. Def: L is decidable within space O(s(n)) if some TM M that decides L always halts on all w  * while never using more than O(s(|w|)) space / tape cells.

24. Complexity Classes Def: DTIME(t(n))={L | L is decidable within time O(t(n)) by some deterministic TM} Def: NTIME(t(n))={L | L is decidable within time O(t(n)) by some non-deterministic TM} Def: DSPACE(s(n))={L | L decidable within space O(s(n)) by some deterministic TM} Def: NSPACE(s(n))={L | L decidable within space O(s(n)) by some non-deterministic TM}

25. Examples of Space & Time Usage Let L 1 ={0 n 1 n | n>0}: For 1-tape TM’s: L 1  DTIME(n 2 ) L 1  DSPACE(n) L 1  DTIME(n log n) For 2-tape TM’s: L 1  DTIME(n) L 1  DSPACE(log n)

26. Examples of Space & Time Usage Let L 2 =  * L 2  DTIME(n) Theorem: every regular language is in DTIME(n) L 2  DSPACE(1) Theorem: every regular language is in DSPACE(1) L 2  DTIME(1) Let L 3 ={w$w | w in  *} L 3  DTIME(n 2 ) L 3  DSPACE(n) L 3  DSPACE(log n)