Overview of the theory of computation Episode 3 0 Turing machines The traditional concepts of computability, decidability and recursive enumerability.

Slides:



Advertisements
Similar presentations
Turing Machines Memory = an infinitely long tape Persistent storage A read/write tape head that can move around the tape Initially, the tape contains only.
Advertisements

CS 345: Chapter 9 Algorithmic Universality and Its Robustness
Variants of Turing machines
CS 461 – Nov. 9 Chomsky hierarchy of language classes –Review –Let’s find a language outside the TM world! –Hints: languages and TM are countable, but.
THE CHURCH-TURING T H E S I S “ TURING MACHINES” Pages COMPUTABILITY THEORY.
CSCI 4325 / 6339 Theory of Computation Zhixiang Chen Department of Computer Science University of Texas-Pan American.
1 COMP 382: Reasoning about algorithms Unit 9: Undecidability [Slides adapted from Amos Israeli’s]
Foundations of (Theoretical) Computer Science Chapter 3 Lecture Notes (Section 3.2: Variants of Turing Machines) David Martin With.
1 Introduction to Computability Theory Lecture12: Decidable Languages Prof. Amos Israeli.
1 Introduction to Computability Theory Lecture15: Reductions Prof. Amos Israeli.
1 Introduction to Computability Theory Lecture12: Reductions Prof. Amos Israeli.
1 Introduction to Computability Theory Lecture11: Variants of Turing Machines Prof. Amos Israeli.
1 Introduction to Computability Theory Lecture13: Mapping Reductions Prof. Amos Israeli.
CPSC 411, Fall 2008: Set 12 1 CPSC 411 Design and Analysis of Algorithms Set 12: Undecidability Prof. Jennifer Welch Fall 2008.
Complexity ©D.Moshkovitz 1 Turing Machines. Complexity ©D.Moshkovitz 2 Motivation Our main goal in this course is to analyze problems and categorize them.
Reducibility A reduction is a way of converting one problem into another problem in such a way that a solution to the second problem can be used to solve.
CS5371 Theory of Computation Lecture 11: Computability Theory II (TM Variants, Church-Turing Thesis)
Programming the TM qa  (,q) (,q) q1q1 0q1q1 R q1q1 1q1q1 R q1q1  h  Qa  (,q) (,q) q1q1 0q2q2  q1q1 1q3q3  q1q1  h  q2q2 0q4q4 R q2q2 1q4q4.
CHAPTER 4 Decidability Contents Decidable Languages
CS5371 Theory of Computation Lecture 10: Computability Theory I (Turing Machine)
Finite State Machines Data Structures and Algorithms for Information Processing 1.
Prof. Busch - LSU1 Turing Machines. Prof. Busch - LSU2 The Language Hierarchy Regular Languages Context-Free Languages ? ?
1 Introduction to Computability Theory Lecture11: The Halting Problem Prof. Amos Israeli.
Theory of Computation. Computation Computation is a general term for any type of information processing that can be represented as an algorithm precisely.
Reduction Episode 9 0 The operation of reduction and the relation of reducibility Examples of reductions The variety of reduction concepts and their systematization.
CSCI 2670 Introduction to Theory of Computing November 4, 2004.
MA/CSSE 474 Theory of Computation More Reduction Examples Non-SD Reductions.
MA/CSSE 474 Theory of Computation Enumerability Reduction.
THE CHURCH-TURING T H E S I S “ TURING MACHINES” Part 1 – Pages COMPUTABILITY THEORY.
The Church-Turing Thesis Chapter 3 Giorgi Japaridze Theory of Computability.
1Computer Sciences Department. Book: INTRODUCTION TO THE THEORY OF COMPUTATION, SECOND EDITION, by: MICHAEL SIPSER Reference 3Computer Sciences Department.
Automata & Formal Languages, Feodor F. Dragan, Kent State University 1 CHAPTER 3 The Church-Turing Thesis Contents Turing Machines definitions, examples,
CS 3813: Introduction to Formal Languages and Automata Chapter 12 Limits of Algorithmic Computation These class notes are based on material from our textbook,
1 Turing’s Thesis. 2 Turing’s thesis: Any computation carried out by mechanical means can be performed by a Turing Machine (1930)
 2005 SDU Lecture13 Reducibility — A methodology for proving un- decidability.
CSC 3130: Automata theory and formal languages Andrej Bogdanov The Chinese University of Hong Kong Turing Machines.
TM Design Macro Language D and SD MA/CSSE 474 Theory of Computation.
Turing -Recognizable vs. -Decidable
Interactive computability Episode 13 0 Hard-play machines (HPMs) Easy-play machines (EPMs) Definition of interactive computability The interactive version.
Donghyun (David) Kim Department of Mathematics and Computer Science North Carolina Central University 1 Chapter 4 Decidability Some slides are in courtesy.
1 Turing Machines - Chap 8 Turing Machines Recursive and Recursively Enumerable Languages.
Lecture 16b Turing Machines Topics: Closure Properties of Context Free Languages Cocke-Younger-Kasimi Parsing Algorithm June 23, 2015 CSCE 355 Foundations.
Automata & Formal Languages, Feodor F. Dragan, Kent State University 1 CHAPTER 3 The Church-Turing Thesis Contents Turing Machines definitions, examples,
1 CD5560 FABER Formal Languages, Automata and Models of Computation Lecture 12 Mälardalen University 2007.
1 Chapter 9 Undecidability  Turing Machines Coded as Binary Strings  Universal Turing machine  Diagonalizing over Turing Machines  Problems as Languages.
Donghyun (David) Kim Department of Mathematics and Computer Science North Carolina Central University 1 Chapter 5 Reducibility Some slides are in courtesy.
Turing Machines Sections 17.6 – The Universal Turing Machine Problem: All our machines so far are hardwired. ENIAC
Turing Machines CS 130 Theory of Computation HMU Textbook: Chap 8.
1 Turing Machines. 2 The Language Hierarchy Regular Languages Context-Free Languages ? ?
The Church-Turing Thesis Chapter Are We Done? FSM  PDA  Turing machine Is this the end of the line? There are still problems we cannot solve:
FORMAL LANGUAGES, AUTOMATA, AND COMPUTABILITY * Read chapter 4 of the book for next time * Lecture9x.ppt.
Universal Turing Machine
Chapters 11 and 12 Decision Problems and Undecidability.
Modeling Arithmetic, Computation, and Languages Mathematical Structures for Computer Science Chapter 8 Copyright © 2006 W.H. Freeman & Co.MSCS SlidesTuring.
The Acceptance Problem for TMs
Theory of Computability
(Universal Turing Machine)
CSC 4170 Theory of Computation Turing reducibility Section 6.3.
Theory of Computation Turing Machines.
Decidable Languages Costas Busch - LSU.
CSC 4170 Theory of Computation Mapping Reducibility Section 5.3.
CSCI 2670 Introduction to Theory of Computing
Turing Machines Complexity ©D.Moshkovitz.
Theory of Computability
Theory of Computability
CSC 4170 Theory of Computation Mapping Reducibility Section 5.3.
Theory of Computability
Variants of Turing machines
Mathematical and computability-theoretic preliminaries
Turing Machines Everything is an Integer
Presentation transcript:

Overview of the theory of computation Episode 3 0 Turing machines The traditional concepts of computability, decidability and recursive enumerability The limitations of the power of Turing machines The Church-Turing thesis Mapping reducibilty Turing reducibility Kolmogorov complexity

Components of a Turing machine (TM) Input tapeRead-only # 2 $ - -Work tape Read-write - blank symbol Output tapeNo direct access Control (transition function) Has a finite number of states, two of which, Start and Halt, are special.

How a TM works Input tape Work tape Output tape At the beginning, the machine is in its Start state, the input tape has an input on it, the other tapes are blank, and the two scanning heads are in their leftmost positions. Such a situation is shown above, with input being 20. After the computation starts, the machine goes from one state to another, writes on the work tape, and moves its scanning heads left or right according to its transition function, which precisely prescribes what to do in each particular case, depending on the current state and the contents of the two scanned cells (on the input and work tapes). If and when the machine enters the Halt state, it halts, and whatever string w is to the left of the work-tape scanning head, will be automatically copied onto the output tape. Such a string w is considered to be the output of the machine. Slide 3.1 shows such a situation, with output 400.

Variants of Turing machines 3.3 There are numerous reasonable variations of the Turing machine model, and they all turn out to be equivalent. For example: More often than not, there is just a single tape for everything (input, output, work). Some models allow multiple work tapes. The tape can be infinite in both directions (have no beginning). And so on. One of the advantages of the particular model we have chosen here is that it can be easily adapted to cases where infinitely many inputs and outputs are allowed. In such cases, entering the Halt state will only have the effect of sending a (next) output to the output tape, but otherwise the machine would not halt Input Output

Computability 3.4 Definition 3.1. Let f: A  B be a function. (a) We say that a TM M computes f iff, for every string w  A, whenever M receives w as input, M (halts and) outputs string v such that v = f(w). (b) When such a machine M exists, we say that f is computable. There is no need to separately consider functions with more than one (but finitely many) arguments, as tuples of strings can be encoded through (just) strings. For example, the function f(x,y)=x+y can be thought of as a single-argument function that returns string “3” on input string “(2,1)”, returns string “20” on input string “(8,12)”, etc.

Decidability 3.5 Definition 3.2. Let S be a set of strings over a given alphabet A. (a) We say that a TM M decides S iff, for every string w  A*, whenever M receives w as input, M (halts and) outputs: 1 (or “yes”, or “accept”) if w  S; 0 (or “no”, or “reject”) if w  S. (b) When such a machine M exists, we say that S is decidable. As every set S can be thought of as and identified with its characteristic function, decidability is just a special case of computability. The concept of decidability extends to predicates and relations, as the latter can be thought of as sets.

Recursive enumerability 3.6 Definition 3.3. Let S be a set of strings over a given alphabet A. (a) We say that a TM M recognizes S iff, for every string w  A*, whenever M receives w as input: if w  S, then M (halts and) outputs “yes”; if w  S, then M either (halts and) outputs something else (such as “no”), or never halts. (b) When such a machine M exists, we say that S is recursively enumerable. Again, this definition extends from sets to predicates and relations.

Computational problems in the traditional sense 3.7 Computational problems in the traditional sense, as established in the traditional theory of computation, are nothing but functions, sets, predicates or relations (to be computed or decided or recognized). We will follow that tradition, and often say “problem” instead of “function”, “relation”, etc. For example, we may refer to the function f(x)=x 2 as “the problem of finding the square of a given number”, and refer to the set (predicate, relation) {x | x is an even number} as “the problem of telling whether a given number is even”, or “the evennees problem”, etc.

The limitations of the power of TMs 3.8 Alas, not all reasonable problems can be handled by TMs. Examples of undecidable or incomputable problems: 1. The acceptance problem for Turing machines, i.e. the problem of telling whether a given Turing machine accepts (meaning outputting “yes”) a given input. 2. The halting problem for Turing machines, i.e. the problem of telling whether a given Turing machine ever halts on a given input. 3. The problem of finding the Kolmogorov complexity of a given number. 4. The problem of telling whether a given polynomial equation has integral roots. 5. The problem of first-order logical validity. Examples of problems that are not recursively enumerable: 6. The complements of the above problems. 7. The problem of second-order logical validity. 8. The problem of telling whether a given formula of arithmetic is true.

The Church-Turing thesis 3.9 Algorithms = Turing machines Computers = Of course, here “=” should be understood not in the strict sense, but in the sense of equivalence of power. That is, according to the Church-Turing thesis, the problems that have algorithmic solutions also have Turing machine solutions (deciders, recognizers, etc.), and vice versa. Thus, the existence of undecidable or incomputable problems means the existence of problems that have no algorithmic solutions, problems that no computers can ever handle. intuitive conceptmathematical conceptintuitive concept

Definition of mapping reducibility 3.10 ** A ** B f f Definition 3.4. Let A and B be sets of strings over an alphabet . (a) We say that a function f:  *  * is a mapping reduction from A to B iff f is computable and, for every w  *, w  A iff f(w)  B. (b) When such a function f exists, we say that A is mapping reducible to B, and write A  m B. In the literature, mapping reducibility is more often called “many-one reducibility”.

Definition of mapping reducibility 3.10 ** A ** B f f Definition 3.4. Let A and B be sets of strings over an alphabet . (a) We say that a function f:  *  * is a mapping reduction from A to B iff f is computable and, for every w  *, w  A iff f(w)  B. (b) When such a function f exists, we say that A is mapping reducible to B, and write A  m B. In the literature, mapping reducibility is more often called “many-one reducibility”.

Definition of mapping reducibility 3.10 ** A ** B f f Definition 3.4. Let A and B be sets of strings over an alphabet . (a) We say that a function f:  *  * is a mapping reduction from A to B iff f is computable and, for every w  *, w  A iff f(w)  B. (b) When such a function f exists, we say that A is mapping reducible to B, and write A  m B. In the literature, mapping reducibility is more often called “many-one reducibility”.

Using mapping reducibility for proving decidability/undecidability 3.11 Theorem 3.5. If A  m B and B is decidable, then A is decidable. Proof: Let D B be a decider for B and f be a mapping reduction from A to B. We describe a decider D A for A as follows. D A = “On input w: 1. Compute f(w). 2. Run D B on input f(w) and output whatever D B outputs.” Theorem 3.5 remains valid with “recursively enumerable” instead of “decidable”. Thus, the decidability of a problem A can be proven by finding a mapping reduction from A to some problem B which is already known to be decidable. Or, the undecidability of a problem B can be proven by finding a mapping reduction from A to B, where A is a problem already known to be undecidable. ( means “end of proof ”).

A mapping reduction of the acceptance problem to the halting problem 3.12 For every TM M, let M’ be the following TM: M’ = “On input x: 1. Run M on x. 2. If M outputs “yes”, accept. 3. If M outputs anything else, enter an infinite loop.” Thus, If M outputs “yes” on input x, then M’ If M outputs anything else on input x, then M’ If M never halts on input x, then M’ To summarize, M accepts x iff M’ Let then f be the function defined by f(M,w)=(M’,w). And, since we have (M,w)  ACCEPTANCE_PROBLEM iff f(M,w)  HALTING_PROBLEM, f is a mapping reduction of the acceptance problem to the halting problem. accepts x never halts on x never halts on x halts on x Is f computable?Of course.

Definition of Turing reducibility 3.13 An oracle for a set (relation) B is an external device that is capable of reporting whether any given string w is a member of B. An oracle Turing machine (OTM) is a modified Turing machine that has the additional capability of querying an oracle. Example: Construct an OTM O with an oracle for the acceptance problem, such that O decides the nonacceptance problem (the complement of the acceptance problem). O = “On input (M,w), where M is a TM and w is a string: 1. Query the oracle to determine whether M accepts w. 2. If the oracle answers NO, accept; if YES, reject.” Definition 3.6. We say that a problem A is Turing reducible to a problem B, written A  T B, iff there is an OTM M with an oracle for B, such that M decides A (or M computes A, if A is a function rather than a relation).

Using Turing reducibility for proving decidability/undecidability 3.14 Theorem 3.7. If A  T B and B is decidable, then A is decidable. Proof. If B is decidable, then we may replace the oracle for B by an actual procedure that decides B. Thus we may replace the OTM that (using an oracle for B) decides A by an ordinary TM that decides A. Does this proof go through for “recursively enumerable” instead of “decidable”? (see the end of Slide 3.11) No! In fact, one can prove the opposite. For example, as we saw on the previous slide, the nonacceptance problem is Turing reducible to the acceptance problem (generally, any problem is Turing reducible to its complement). And the acceptance problem is obviously recursively enumerable (why?). But the nonacceptance problem is not recursively enumerable (why?). It also follows from the comment at the end of Slide 3.11 that the nonacceptance problem is not mapping reducible to the acceptance problem. Does mapping reducibility always imply Turing reducibility? Yes! Turing reducibility is the weakest form of reducibility.

Turing reductions of the acceptance problem to the halting problem 3.15 Example: Show that the acceptance problem is Turing reducible to the halting problem. Solution 1. When receiving the question “Does M accept w?”, replace M by M’ (as described on Slide 3.12), and ask the oracle whether M’ halts on w. Repeat whatever answer you get from the oracle. Solution 2. When receiving the question “Does M accept w?”, ask the oracle if M halts on input w. If the oracle says “No”, you also say “No”. If the oracle says “Yes”, start running (simulating) machine M on input w until you see it has halted. If its output is “Yes”, you also say “Yes”. Otherwise say “No”. Why could not we just do simulation as described in the last paragraph of Solution 2, without any preliminary use of the oracle? Because there would be no guarantee that the simulation would ever end.

Kolmogorov complexity 3.16 All Turing machines can be listed in the lexicographic order of their descriptions: M 0, M 1, M 2, M 3,..., M i,.... Number i can thus be considered the code of the machine M i. Definition 3.8. Let m be a natural number. The Kolmogorov complexity of m is the smallest number i such that machine M i outputs m on input 0. Note: More often, Kolmogorov complexity is defined not as the above number i itself, but as the size |i| of that number, i.e. the logarithm of i. For our purposes, however, this makes no difference, and we will stick to Definition 3.8 as it is given. Importance: Kolmogorov complexity can be seen as a mathematical counterpart of the intuitive concepts of “randomness” or “amount of information”. The greater the Kolmogorov complexity of a given object, the more random it is and the more information it contains.

Turing reduction of the Kolmogorov complexity problem to the halting problem 3.17 Example: Show that the Kolmogorov complexity problem (i.e. the problem of finding the Kolmogorov complexity of a given number) is Turing reducible to the halting problem. Solution. After receiving the question “What is the Kolmogorov complexity of m?”, initialize variable i to 0, and do the following: 1. Ask the oracle if the machine M i halts on input If the oracle says “No”, increment i to i+1, and go back to Step If the oracle says “Yes”, simulate M i on input 0 until it halts. If you see that the output of M i is m, return i as your output. Otherwise, increment i to i+1 and go back to Step 1. How many times does the above algorithm use the oracle? Unlike the example on Slide 3.15, here the oracle is queried more than once. The number of queries, however, can be shown to be bounded by a certain linear function of m. Theorem 3.9. The Kolmogorov complexity problem is not computable.