FORMAL LANGUAGES, AUTOMATA, AND COMPUTABILITY * Read chapter 4 of the book for next time * Lecture9x.ppt

A Turing Machine is represented by a 7-tuple T = (Q, Σ, Γ, , q 0, q accept, q reject ): Q is a finite set of states Γ is the tape alphabet, a superset of Σ;   Γ q 0  Q is the start state Σ is the input alphabet, where   Σ  : Q  Γ → Q  Γ  {L, R} is the transition func q accept  Q is the accept state q reject  Q is the reject state, and q reject  q accept

CONFIGURATION: (1) tape contents, (2) current state, (3) location of read/write head q

A TM recognizes a language iff it accepts all and only those strings in the language. A TM decides a language L iff it accepts all strings in L and rejects all strings not in L. A language L is called Turing-recognizable or recursively enumerable iff some TM recognizes L. A language L is called decidable or recursive iff some TM decides L.

A language is called Turing-recognizable or recursively enumerable (r.e.) if some TM recognizes it. A language is called decidable or recursive if some TM decides it.

Theorem: If A and  A are r.e. then A is decidable. Given: a Turing Machine TM A that recognizes A and a Turing Machine TM R that recognizes  A, we can build a new machine that decides A. How can we prove this? Run TM A and TM R in parallel. (Or more precisely, interleave them.) One of them will eventually recognize the input string. If TM A recognizes it, then accept. If TM R recognizes it, then reject.

A TM that decides { 0 | n ≥ 0 } High-Level Idea. Repeatedly divide the number of zeros in half until it becomes an odd number. If we are left with a single zero, then accept. Otherwise, reject. 2n2n We want to accept iff: the input string consists entirely of zeros, and the number of zeros is a power of 2.

A TM that decides { 0 | n ≥ 0 } 1.Sweep from left to right, cross out every other 0. (Divides number in half.) 2.If in step 1, the tape had only one 0, accept. 3.Else if the tape had an odd number of 0’s, reject. 4.Move the head back to the first input symbol. 5.Go to step 1. PSEUDOCODE: 2n2n

0 → , R  → , R q accept q reject 0 → x, R x → x, R  → , R x → x, R 0 → 0, L x → x, L x → x, R  → , L  → , R 0 → x, R 0 → 0, R  → , R x → x, R { 0 | n ≥ 0 } 2n2n q0q0 q1q1 q2q2 q3q3 q4q4

0 → , R  → , R q accept q reject 0 → x, R x → x, R  → , R x → x, R 0 → 0, L x → x, L x → x, R  → , L  → , R 0 → x, R 0 → 0, R  → , R x → x, R { 0 | n ≥ 0 } 2n2n q0q0 q1q1 q2q2 q3q3 q4q4 q  q  xq 3 00  x0q 4 0  x0xq 3  x0q 2 x  xq 2 0x  q 2 x0x q 2  x0x

C = {a i b j c k | k = i×j, and i, j, k ≥ 1} Example aaabbbbcccccccccccc 33×4 = 124

C = {a i b j c k | k = i×j, and i, j, k ≥ 1} High-Level Idea. For each occurrence of a : { For each occurrence of b : { Delete an occurrence of c. }

C = {a i b j c k | k = i×j, and i, j, k ≥ 1} 1.If the input doesn’t match a * b * c *, reject. 2.Move the head back to the leftmost symbol. 3.Cross off an a, scan to the right until b. Sweep between b ’s and c ’s, crossing out one of each until all b ’s are out. If too few c ’s, reject. 4.Uncross all the b ’s. If there’s another a left, then repeat stage 3. If all a ’s are crossed out, Check if all c ’s are crossed off. If yes, then accept, else reject. PSEUDOCODE:

C = {a i b j c k | k = i×j, and i, j, k ≥ 1} aabbbcccccc xabbbcccccc xayyyzzzccc xabbbzzzccc xxyyyzzzzzz

TURING-MACHINE VARIANTS Turing machines can be extended in various ways, but so long as a new TM only reads and writes a finite number of symbols in each step, an old TM can still simulate it! Example: Turing machines with multiple tapes. Input comes in on one tape, and other tapes are used for scratch work.

MULTITAPE TURING MACHINES  : Q  Γ k → Q  Γ k  {L,R} k FINITE STATE CONTROL

Theorem: Every Multitape Turing Machine can be transformed into a single tape Turing Machine FINITE STATE CONTROL # # #...

Theorem: Every Multitape Turing Machine can be transformed into a single tape Turing Machine FINITE STATE CONTROL # # #...

THE CHURCH-TURING THESIS Anything that can be computed by algorithm (in our intuitive sense of the term “algorithm”) can be computed by a Turing Machine.

We can encode a TM as a string of 0s and 1s 0 n 1 0 m 1 0 k 1 0 s 1 0 t 1 0 r 1 0 u 1… n states m tape symbols (first k are input symbols) start state accept state reject state blank symbol ( (p, a), (q, b, L) ) = 0 p 10 a 10 q 10 b 10 ( (p, a), (q, b, R) ) = 0 p 10 a 10 q 10 b 11

Similarly, we can encode DFAs, NFAs, CFGs, etc. into strings of 0s and 1s A DFA = { (B, w) | B is a DFA that accepts string w } A NFA = { (B, w) | B is an NFA that accepts string w } A CFG = { (G, w) | G is a CFG that generates string w } So we can define the following languages:

A DFA = { (B, w) | B is a DFA that accepts string w } Theorem: A DFA is decidable Proof Idea: Simulate B on w A CFG = { (G, w) | G is a CFG that generates string w } Theorem: A CFG is decidable Proof Idea: Transform G into Chomsky Normal Form. Try all derivations of length 2|w|-1 A NFA = { (B, w) | B is an NFA that accepts string w } Theorem: A NFA is decidable

UNDECIDABLE PROBLEMS

w  L ? acceptreject TM yes no w  Σ* L is decidable (recursive) w  L ? accept reject or no output TM yes no w  Σ* L is semi-decidable (recursively enumerable, Turing-recognizable) Theorem: L is decidable if both L and  L are recursively enumerable

There are languages over {0,1} that are not decidable. If we believe the Church-Turing Thesis, this is major: it means there are things that formal computational models inherently cannot do. We can prove this using a counting argument. We will show there is no function from the set of all Turing Machines onto the set of all languages over {0,1}. (Works for any Σ.) Then we will prove something stronger: There are semi-decidable (r.e.) languages that are NOT decidable.

Turing Machines Languages over {0,1}

Let L be any set and 2 L be the power set of L Theorem: There is no map from L onto 2 L Proof: Assume, for a contradiction, that there is an onto map f : L  2 L Let S = { x  L | x  f(x) } We constructed S so that, for every elem x in L, the set S differs from f(x): S ≠ f(x) because x  S iff x  f(x) Cantor’s Theorem

Theorem: There is no onto function from the positive integers to the real numbers in (0, 1) 12345:12345: … … … … … : Proof:Suppose f is such a function: [ n th digit of r ] = if [ n th digit of f(n) ]  1 0otherwise f(n)  r for all n ( Here, r = )

Let Z + = {1,2,3,4…}. There exists a bijection between Z + and Z +  Z + (1,1) (1,2) (1,3) (1,4) (1,5) … (2,1) (2,2) (2,3) (2,4) (2,5) … (3,1) (3,2) (3,3) (3,4) (3,5) … (4,1) (4,2) (4,3) (4,4) (4,5) … (5,1) (5,2) (5,3) (5,4) (5,5) … (or Q + ) : : : : : ∙. Sidenote

THE MORAL: For any set L, 2 L always has ‘more’ elements than L

Not all languages over {0,1} are decidable, in fact: not all languages over {0,1} are semi-decidable {Turing Machines} {Strings of 0s and 1s} {Sets of strings of 0s and 1s} {Languages over {0,1}} Set L Powerset of L: 2 L {decidable languages over {0,1}} {semi-decidable langs over {0,1}}

A TM = { (M, w) | M is a TM that accepts string w } THE ACCEPTANCE PROBLEM Theorem: A TM is semi-decidable (r.e.) but NOT decidable A TM is r.e. : Define a TM U as follows: On input (M, w), U runs M on w. If M ever accepts, accept. If M ever rejects, reject. Therefore, U accepts (M,w)  M accepts w  (M,w)  A TM Therefore, U recognizes A TM U is a universal TM

A TM = { (M,w) | M is a TM that accepts string w } A TM is undecidable:(proof by contradiction) Assume machine H decides A TM H( (M,w) ) = Acceptif M accepts w Reject if M does not accept w Construct a new TM D as follows: on input M, run H on (M,M) and output the opposite of H D( M ) = Rejectif M accepts M Accept if M does not accept M D D D D D Contradiction!

M1M1 M2M2 M3M3 M4M4 : M1M1 M2M2 M3M3 M4M4 … accept reject OUTPUT OF H accept reject D D accept reject accept ?

Theorem: A TM is r.e. but NOT decidable Theorem:  A TM is not even r.e.! The Halting Problem is Not Decidable

A TM = { (M,w) | M is a TM that accepts string w } A TM is undecidable: Let machine H semi-decides A TM (Such , why?) H( (M,w) ) = Acceptif M accepts w Reject or No outputif M does not accept w Construct a new TM D as follows: on input M, run H on (M,M) and output D( M ) = Rejectif H ( M, M ) Accepts Accept if H ( M, M ) Rejects No output if H ( M, M ) has No output D D D D D D D A constructive proof: H( (D,D) ) = No outputNo Contradictions !

We have shown: Given any presumed machine H for A TM, we can effectively construct a TM D such that (D,D)  A TM but H fails to tell us that. In other words, For any machine H that recognizes A TM we can effectively give an instance where H fails to decide A TM In other words, Given any good candidate for deciding the Halting Problem, we can effectively construct an instance where the machine fails.

HALT TM = { (M,w) | M is a TM that halts on string w } Theorem: HALT TM is undecidable THE HALTING PROBLEM Proof: Assume, for a contradiction, that TM H decides HALT TM We use H to construct a TM D that decides A TM On input (M,w), D runs H on (M,w) If H rejects then reject If H accepts, run M on w until it halts: Accept if M accepts and Reject if M rejects

H (M,w) M w If M doesn’t halt: REJECT If M halts Does M halt on w? D

In many cases, one can show that a language L is undecidable by showing that if it is decidable, then so is A TM We reduce deciding A TM to deciding the language in question A TM ≤ L We just showed: A TM ≤ Halt TM Is Halt TM ≤ A TM ?