Mathematical and computability-theoretic preliminaries

Mathematical and computability-theoretic preliminaries
Episode 2 Mathematical and computability-theoretic preliminaries Sets Sequences Functions Relations Strings Turing machines The traditional concepts of computability, decidability and recursive enumerability The limitations of the power of Turing machines The Church-Turing thesis Mapping reducibilty Turing reducibility Kolmogorov complexity

Set --- collection of distinct objects.
Sets 2.1 Set --- collection of distinct objects. E={2,4,6,8,…}, or E={x | x is a positive integer divisible by 2}, or E={x | x=2k for some positive integer k}, etc. Describing a set: Set-related terminology and notation: aE “a is an element of E”, or “a is in E ” aE “a is not an element of E”, or “a is not in E ” ST “S is a subset of T ” i.e. every element of S is also an element of T ST “the intersection of S and T ” i.e. the set of the objects that are both in S and T ST “the union of S and T ” i.e. the set of the objects that are in either S or T or both  “the empty set”

Sequences, tuples, products
2.2 A sequence is a finite or infinite list. E.g.: 1,1,2,1,2,3,1,2,3,4,1,2,3,4,… is a sequence of natural numbers An n-tuple is a sequence with n elements. E.g.: (5,2) tuple (pair) (3,0,3) tuple (triple) (1,2,1,2) tuple (quadruple) ! {1,2,2} = {1,2} = {2,1}, but (1,2,2)  (1,2)  (2,1) The product of sets S and T is defined by ST = {(s,t) | sS and tT} Generalizes to S1...Sn

Functions 2.3 Function (often also called an operation) f from set A to set B --- assignment of a unique element f(a)B to each aA A B the range of f the domain of f f a b c 1 2 3 4 f: A  B the type of f NAT --- natural numbers: {0,1,2,…} RAT --- rational numbers: {0/1, 5/1, 1/3, 8/5, etc.} If x,y always take values from NAT, what are the types of f, g, h? f(x) = 2x g(x) = x/ h(x,y) = x+y f: g: h: NAT  NAT NAT  RAT NAT  NAT  NAT

subset of An (An means A...A n times).
Relations 2.4 Let A be a set and n be a natural number. An n-ary relation on A is any subset of An (An means A...A n times). When n=0, the relation is said to be nullary; when n=1, it is unary; when n=2, it is binary; when n=3, it is ternary. Example: < is a binary relation on NAT: (3,1) (1,3) (6,20) (12,32) ... (4,4) < (100,39) ... NATNAT

Characteristic functions
2.5 Let A be a set and R be an n-ary relation on it. The characteristic function of R is defined as the function f: An{0,1} such that, for every (w1,…,wn)An, f(w1,…,wn) = 1 if (w1,…,wn)R; 0 if (w1,…,wn)R. We often identify relations with their characteristic functions.

{a,…,z, 0,...,9, !,?,…, $,&,@,,#,...} --- Keyboard alphabet.
Strings 2.6 Alphabet --- a finite set of objects called the symbols of the alphabet. {a,…,z, 0,...,9, !,?,…, --- Keyboard alphabet. {0,...,9} --- Decimal alphabet. Its elements are called decimal digits. {0,1} --- Binary alphabet. Its elements are called bits (binary digits). String over an alphabet  --- a sequence of symbols from . Decimal strings: 2018, 12144, etc. Binary strings (bit strings): 1001, 00000, 011, etc. Finite strings: abracadabra, 西南大学, etc. Infinite strings: , , etc. The empty string is denoted by . The set of all strings over alphabet  is denoted by *. The Concatenation wu of strings w and u is the result of appending u at the end of w. Defined only when w is finite.

Turing machines (TMs) 2.7 A Turing machine (TM) has a finite number of states, two of which, Start and Halt, are special. Has two tapes: the read-only input/output (I/O) tape and the read/write work tape, each equipped with its own scanning head. The tapes are infinite in one (left to right) direction, and are divided into cells. At any time each cell of either tape holds one of the finitely many possible symbols, including the blank symbol -. At the beginning, the machine is in its Start state, both scanning heads are looking at the leftmost cells of their tapes, the work tape is blank, and the I/O tape has ⊥ at its beginning for some string , with such an  considered to be the input. A restriction on  is that it is not allowed to contain ⊥ or ⊤. After the computation starts, the machine goes from one state to another, writes on the work tape, and moves its scanning heads according to its transition function, which precisely prescribes what to do in each particular case, depending on the current state and the contents of the two scanned cells (on the input and work tapes). A restriction is that the symbols ⊥ and ⊤ can never be written on the work tape. If and when the machine enters the Halt state, it halts, and whatever string  is to the left of the work-tape scanning head, will be automatically copied onto the input/output tape in the ⊤-prefixed form ⊤. Such a string  is considered to be the output of (returned by) the machine.

where D1,D2{L,R,S}. The meaning is:
Transition function 2.8 The transition function of a TM, and the whole TM for that matter, can be described using a control diagram (more often called state diagram or transition diagram) like the one shown on Slide 2.9. The circles on such a diagram represent the states of the machine. A labeled edge (“transition”) from one state to another looks like x,y  z,D1,D2 a b where D1,D2{L,R,S}. The meaning is: This transition will be taken (followed) iff the machine is in state a, the cell under the I/O tape scanning head contains symbol x, and the cell under the work tape scanning head contains symbol y. When the transition is taken, the following happens: * The state of the machine changes from a to b. * Symbol y is overwritten with symbol z on the work tape. * The I/O [resp. work] tape scanning head moves one cell to the left if D1=L [resp. D2=L], one cell to the right if D1=R [resp. D2=R], and stays put if D1=S [resp. D2=S]. An exception here is that, if the scanning head is located over the leftmost cell and the prescribed direction is L, things happen as if L was S (the head does not move). When there is no transition to take in a non-Halt state, the machine gets stuck with no output.

(transition function) Control
An example of a TM 2.9 Input/output tape - - - - - - - - - Read-only Work tape - - - - - - - - - Read-write -,-  -,S,L -,0  0,L,R 0,-  0,R,R 0,-  0,R,R -,1  0,L,R 1,-  1,R,R (transition function) Control ⊥,-  -,R,L Start Halt 0,-  0,R,R -,0  1,L,R 1,-  1,R,R 1,-  1,R,R -,1  1,L,R -,-  -,S,L

An example of a TM 2.9 Input Input/output tape ⊥ 1 - - - - - Read-only Work tape - - - - - - - - - Read-write -,-  -,S,L -,0  0,L,R 0,-  0,R,R 0,-  0,R,R -,1  0,L,R 1,-  1,R,R (transition function) Control ⊥,-  -,R,L Start Halt 0,-  0,R,R -,0  1,L,R 1,-  1,R,R 1,-  1,R,R -,1  1,L,R -,-  -,S,L

An example of a TM 2.9 Input Input/output tape ⊥ 1 - - - - - Read-only Work tape 1 - - - - - - - - Read-write -,-  -,S,L -,0  0,L,R 0,-  0,R,R 0,-  0,R,R -,1  0,L,R 1,-  1,R,R (transition function) Control ⊥,-  -,R,L Start Halt 0,-  0,R,R -,0  1,L,R 1,-  1,R,R 1,-  1,R,R -,1  1,L,R -,-  -,S,L

An example of a TM 2.9 Input Input/output tape ⊥ 1 - - - - - Read-only Work tape 1 - - - - - - - Read-write -,-  -,S,L -,0  0,L,R 0,-  0,R,R 0,-  0,R,R -,1  0,L,R 1,-  1,R,R (transition function) Control ⊥,-  -,R,L Start Halt 0,-  0,R,R -,0  1,L,R 1,-  1,R,R 1,-  1,R,R -,1  1,L,R -,-  -,S,L

An example of a TM 2.9 Input Input/output tape ⊥ 1 - - - - - Read-only Work tape 1 - - - - - - Read-write -,-  -,S,L -,0  0,L,R 0,-  0,R,R 0,-  0,R,R -,1  0,L,R 1,-  1,R,R (transition function) Control ⊥,-  -,R,L Start Halt 0,-  0,R,R -,0  1,L,R 1,-  1,R,R 1,-  1,R,R -,1  1,L,R -,-  -,S,L

An example of a TM 2.9 Input Output Input/output tape ⊥ 1 ⊤ 1 1 - Read-only Work tape 1 1 - - - - - - Read-write -,-  -,S,L -,0  0,L,R 0,-  0,R,R 0,-  0,R,R -,1  0,L,R 1,-  1,R,R (transition function) Control ⊥,-  -,R,L Start Halt 0,-  0,R,R -,0  1,L,R 1,-  1,R,R 1,-  1,R,R -,1  1,L,R -,-  -,S,L

Variants of Turing machines
2.10 There are numerous reasonable variations of the Turing machine model, and they all turn out to be equivalent. For example: More often than not, there is just a single tape for everything (input, output, work). Some models allow multiple work tapes. Each tape may have several (rather than one) scanning heads. The tapes can be infinite in both directions (have no beginning). Memory (“tape”) can be 3-dimentional instead of 1-dimentional. And so on.

Definition 2.11 Let f: AB be a function.
Computability 2.11 Definition Let f: AB be a function. (a) We say that a TM M computes f iff, for every string wA, whenever M receives w as an input, M (halts and) outputs string v such that v = f(w). (b) When such a machine M exists, we say that f is computable. There is no need to separately consider functions with more than one arguments, as tuples of strings can be encoded through (just) strings. For example, the function f(x,y)=x+y can be thought of as a single-argument function that returns string “3” on input string “(2,1)”, returns string “20” on input string “(8,12)”, etc.

Definition 2.12 Let S be a set of strings over a given alphabet .
Decidability 2.12 We say that a given TM M accepts [resp. rejects] a given string w iff, whenever M receives w as an input, M (halts and) outputs 1 [resp. 0]. Definition Let S be a set of strings over a given alphabet . (a) We say that a TM M decides S iff, for every input w*: if wS, then M accepts w if wS, then M rejects w (b) When such a machine M exists, we say that S is decidable. in other words, M computes the characteristic function of the “…is a member of S ” relation The concept of decidability extends to relations, as the latter can be thought of as sets: A relation is decidable iff its characteristic function is computable. Thus, decidability is just a special case of computability.

Recursive enumerability
2.13 Definition Let S be a set of strings over a given alphabet . (a) We say that a TM M recognizes S iff, for every input w*: if wS, then M accepts w if wS, then M either rejects w or never halts on w (b) When such a machine M exists, we say that S is recursively enumerable. Again, this definition extends from sets to relations. Synonyms of “recursively enumerable”: recognizable; semidecidable.

Computational problems in the traditional sense
2.14 Computational problems in the traditional sense, as established in the traditional theory of computation, are nothing but functions, sets or relations (to be computed or decided or recognized). For now we will follow that tradition, and often say “problem” instead of “set”, “function”, “relation”, etc. For example, we may refer to the function f(x)=x2 as “the problem of finding the square of a given number”, and refer to the set (relation) {x | x is an even number} as “the problem of telling whether a given number is even”, or “the evenness problem”, etc.

The limitations of the power of TMs
2.15 Not all reasonable problems can be handled by TMs! Examples of undecidable or incomputable problems: 1. The acceptance problem for Turing machines, i.e. the problem of telling whether a given Turing machine accepts a given input. 2. The halting problem for Turing machines, i.e. the problem of telling whether a given Turing machine ever halts on a given input. 3. The problem of finding the Kolmogorov complexity (Slide 2.23) of a given number. 4. The problem of telling whether a given polynomial equation has integral roots. 5. The problem of first-order logical validity. Examples of problems that are not recursively enumerable: 6. The complements of the above problems. 7. The problem of second-order logical validity. 8. The problem of telling whether a given formula of arithmetic is true.

The Church-Turing thesis
2.16 Algorithms = Turing machines Computers = intuitive concept mathematical concept Of course, here “=” should be understood not in the strict sense of identity, but in the sense of equivalence of power. That is, according to the Church-Turing thesis, the problems that have algorithmic solutions also have Turing machine solutions (deciders, recognizers, etc.), and vice versa. Thus, the existence of undecidable or incomputable problems means the existence of problems that have no algorithmic solutions, problems that no computers can ever handle.

Definition of mapping reducibility
2.17 Definition 2.16 Let A and B be sets of strings over an alphabet . (a) We say that a function f: ** is a mapping reduction of A to B iff f is computable and, for every w*, wA iff f(w)B. (b) When such a function f exists, we say that A is mapping reducible to B, and write AmB. In the literature, mapping reducibility is more often called “many-one reducibility”. * * A B f f(w) w f w f(w)

2.17 Definition 2.16 Let A and B be sets of strings over an alphabet . (a) We say that a function f: ** is a mapping reduction of A to B iff f is computable and, for every w*, wA iff f(w)B. (b) When such a function f exists, we say that A is mapping reducible to B, and write AmB. In the literature, mapping reducibility is more often called “many-one reducibility”. * * A B f w f(w) f w f(w)

2.17 Definition 2.16 Let A and B be sets of strings over an alphabet . (a) We say that a function f: ** is a mapping reduction of A to B iff f is computable and, for every w*, wA iff f(w)B. (b) When such a function f exists, we say that A is mapping reducible to B, and write AmB. In the literature, mapping reducibility is more often called “many-one reducibility”. * * A B f f(w) f(w) f

Using mapping reducibility for proving decidability/undecidability
2.18 Theorem If AmB and B is decidable, then A is decidable. Proof: Let DB be a decider for B and f be a mapping reduction of A to B. We describe a decider DA for A as follows. DA= “On input w: 1. Compute f(w). 2. Run DB on input f(w) and output whatever DB outputs.”  ( means “end of proof ”). Thus, the decidability of a problem A can be proven by finding a mapping reduction from A to some problem B which is already known to be decidable. Or, the undecidability of a problem B can be proven by finding a mapping reduction from A to B, where A is a problem already known to be undecidable. Theorem remains valid with “recursively enumerable” instead of “decidable”.

A mapping reduction of the acceptance problem to the halting problem
2.19 For every TM M, let M’ be the following TM: M’ = “On input x: 1. Run M on x. 2. If M outputs 1, accept. 3. If M outputs anything else, enter an infinite loop.” Thus, If M outputs 1 on input x, then M’ If M outputs anything else on input x, then M’ If M never halts on input x, then M’ To summarize, M accepts x iff M’ accepts x never halts on x halts on x Let then f be the function defined by f(M,w)=(M’,w). Is f computable? Of course. And, since we have (M,w)ACCEPTANCE_PROBLEM iff f(M,w)HALTING_PROBLEM, f is a mapping reduction of the acceptance problem to the halting problem.

Definition of Turing reducibility
2.20 An oracle for a set (relation) B is an external (perhaps supernatural) device that is capable of reporting whether any given string w is a member of B. An oracle Turing machine (OTM) is a Turing machine equipped with an oracle for some set B, which the machine can query at any time. Example: Construct an OTM O with an oracle for the acceptance problem which decides the nonacceptance problem (the complement of the acceptance problem). O = “On input (M,w), where M is a TM and w is a string: 1. Query the oracle to determine whether M accepts w. 2. If the oracle answers NO, accept; if YES, reject.” Definition 2.20 We say that a problem A is Turing reducible to a problem (set) B, written ATB, iff there is an OTM with an oracle for B which decides A (or computes A, if A is a function rather than a relation).

Using Turing reducibility for proving decidability/undecidability
2.21 Theorem If ATB and B is decidable, then A is decidable. Proof. If B is decidable, then we may replace the oracle for B by an actual procedure that decides B. Thus we may replace the OTM that (using an oracle for B) decides A by an ordinary TM that decides A.  Does this proof go through for “recursively enumerable” instead of “decidable”? No! In fact, one can prove the opposite. For example, as we saw on the preceding slide, the nonacceptance problem is Turing reducible to the acceptance problem (generally, any problem is Turing reducible to its complement). And the acceptance problem is obviously recursively enumerable (why?). But the nonacceptance problem is not recursively enumerable (why?). It also follows from the comment at the end of Slide 2.17 that the nonacceptance problem is not mapping reducible to the acceptance problem. Does mapping reducibility always imply Turing reducibility? Yes! Turing reducibility is the weakest form of reducibility.

Turing reductions of the acceptance problem to the halting problem
2.22 Example: Show that the acceptance problem is Turing reducible to the halting problem. Solution 1. Upon receiving the question “Does M accept w?” (i.e., the input (M,w)), replace M by M’ (as described on Slide 2.19), and ask the oracle whether M’ halts on w. Accept if the oracle says “yes”, reject if it says “no”.  Solution 2. Upon receiving the question “Does M accept w?”, ask the oracle if M halts on input w. If the oracle says “No”, reject. If the oracle says “Yes”, start running (simulating) machine M on input w until you see it has halted. If its output is 1, accept. Otherwise reject.  Why could not we just do simulation as described in the last paragraph of Solution 2, without any preliminary use of the oracle? Because there would be no guarantee that the simulation would ever end.

Kolmogorov complexity
2.23 All Turing machines can be listed in the lexicographic order of their descriptions: M0, M1, M2, M3, ..., Mi, .... Number i can thus be considered the code of the machine Mi. Definition 2.23 Let m be a natural number. The Kolmogorov complexity of m is the size (logarithm) |i| of the smallest number i such that machine Mi outputs m on input 0. Intuitions: A Turing machine outputting a number m on a fixed input (such as 0) can be seen as a description of m. So, the Kolmogorov complexity k of m is the size of the smallest possible description of m: the reater k, the harder (“complex”) to describe m. Kolmogorov complexity can be seen as a mathematical counterpart of the intuitive concepts of “randomness” or “amount of information”. The greater the Kolmogorov complexity of a given object, the more random it is and the more information it contains.

Turing reduction of the Kolmogorov complexity problem to the halting problem
2.24 Example: Show that the Kolmogorov complexity problem (i.e. the problem of finding the Kolmogorov complexity of a given number) is Turing reducible to the halting problem. Solution. After receiving the question “What is the Kolmogorov complexity of m?”, initialize variable i to 0, and do the following: 1. Ask the oracle if the machine Mi halts on input 0. 2. If the oracle says “No”, increment i to i+1, and go back to Step 1. 3. If the oracle says “Yes”, simulate Mi on input 0 until it halts. If you see that the output of Mi is m, return the size (logarithm) |i| of i as your output. Otherwise, increment i to i+1 and go back to Step 1.  How many times does the above algorithm use the oracle? Unlike the example on Slide 2.21, here the oracle is queried more than once. The number of queries, however, can be shown to be bounded by a certain linear function of m. Theorem 2.24 The Kolmogorov complexity problem is not computable.

Mathematical and computability-theoretic preliminaries

Similar presentations

Presentation on theme: "Mathematical and computability-theoretic preliminaries"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Mathematical and computability-theoretic preliminaries

Similar presentations

Presentation on theme: "Mathematical and computability-theoretic preliminaries"— Presentation transcript:

Similar presentations

About project

Feedback