CSC 3130: Automata theory and formal languages Andrej Bogdanov The Chinese University of Hong Kong Undecidable Languages Fall 2008

Are Turing Machines computers? A Turing Machine does not take the program as part of its input. computer input data instructions (program) output data... but it can!

The Universal Turing Machine U input x program 〈 M 〉 M on input x The universal TM U takes as inputs a program M and a string x and simulates M on x The program M itself is specified as a TM!

Turing Machines as strings This Turing Machine can be described by the string 〈 M 〉 = ((q0, qacc, qrej), ((q0, q0, ☐ / ☐ R), (q0, qacc, 0/0R), (q0, qrej, 1/1R))) M q0q0 q acc q rej ☐/☐R☐/☐R 0/0R 1/1R

The universal Turing Machine U ((q0, qacc, qrej), ((q0, q0, ☐ / ☐ R), 001 input x program 〈 M 〉 q0 state U on input ( 〈 M 〉, x) will simulate M on input x

Languages about automata A TM can take another TM (or DFA, CFG) as input A DFA = {( 〈 D 〉, w): D is a DFA that accepts input w} A PDA = {( 〈 P 〉, w): P is a PDA that accepts w} A TM = {( 〈 M 〉, w): M is a TM that accepts w} Which of these is decidable? decidable

Languages about automata Idea: Simulate M on input w Simulation is correct: –If M accepts w, we accept –If M rejects w, we reject –If M loops on w, we loop This TM recognizes but does not decide A TM A TM = {( 〈 M 〉, w): M is a TM that accepts w}

Recognizing versus deciding –This kind of TM is called a decider The textbook uses different terminology: The language recognized by a TM is the set of all inputs that make it reach q acc A TM decides language L if it recognizes L and does not loop on any input recognizable = recursively enumerable (r. e.) decidable = recursive

Undecidability Turing’s Theorem: Before we prove this, let’s observe one thing: The language A TM is undecidable. A Turing Machine M can be given its own description 〈 M 〉 as an input!

Proof of undecidability Proof by contradiction: Suppose A TM is decidable. Then there is a TM H that decides A TM : H 〈M〉,w〈M〉,w accept if M accepts w reject if M rejects w or M loops on w 〈M〉,〈M〉〈M〉,〈M〉 accept if M accepts 〈 M 〉 reject if M rejects 〈 M 〉 or M loops on 〈 M 〉 What happens when w = 〈 M 〉 ?

Proof of undecidability H 〈M〉,〈M〉〈M〉,〈M〉 accept if M accepts 〈 M 〉 reject if M rejects 〈 M 〉 or M loops on 〈 M 〉 Let H’ be a TM that does the opposite of H H acc rej acc rej H’

Proof of undecidability H 〈 M 〉, 〈 M 〉 accept if M accepts 〈 M 〉 reject if M rejects 〈 M 〉 or M loops on 〈 M 〉 H’H’ 〈M〉,〈M〉〈M〉,〈M〉 reject if M accepts 〈 M 〉 if M rejects 〈 M 〉 or M loops on 〈 M 〉 accept

Proof of undecidability H’H’ 〈 M 〉, 〈 M 〉 reject if M accepts 〈 M 〉 if M rejects 〈 M 〉 or M loops on 〈 M 〉 accept Let D be the following TM: copy 〈M〉〈M〉 〈 M 〉, 〈 M 〉 H’

Proof of undecidability D 〈M〉〈M〉 reject if M accepts 〈 M 〉 if M rejects 〈 M 〉 or M loops on 〈 M 〉 accept What happens when M = D ? 〈D〉〈D〉 reject if D accepts 〈 D 〉 if D rejects 〈 D 〉 or D loops on 〈 D 〉 If D accepts 〈 D 〉 then D rejects 〈 D 〉 If D rejects 〈 D 〉 then D accepts 〈 D 〉 so D does not exist!

Proof of undecidability: conclusion Proof by contradiction –We assumed A TM was decidable –Then we had Turing Machines H, H’, D –But D does not exist! Conclusion The language A TM is undecidable.

What happened? M1M1 M2M2 M3M3 …  … accrej acc rejacclooprej looprej … … Turing Machine input w We can write an infinite table for every pair (M, w)

What happened? M1M1 M2M2 M3M3 … 〈M1〉〈M1〉〈M2〉〈M2〉 〈M4〉〈M4〉 … acclooprejacc rejaccrejacc looprejlooprej … … Now let’s look only at those w that describe a TM M 〈M3〉〈M3〉

What happened? M1M1 M2M2 … 〈M1〉〈M1〉〈M2〉〈M2〉 〈M4〉〈M4〉 … acclooprejacc rejaccrejacc … … If A TM is decidable, then TM D is in the table 〈M3〉〈M3〉 D rej accrej … …

What happened? M1M1 M2M2 … 〈M1〉〈M1〉〈M2〉〈M2〉 〈M4〉〈M4〉 … acclooprejacc rejaccrejacc … … D does the opposite of the diagonal entries 〈M3〉〈M3〉 D rej accrej D 〈M〉〈M〉 reject if M accepts 〈 M 〉 accept if M rejects or loops on 〈 M 〉

What happened? M1M1 M2M2 … 〈M1〉〈M1〉〈M2〉〈M2〉 〈M4〉〈M4〉 … acclooprejacc rejaccrejacc … … We run into trouble when we look at (D, 〈 D 〉 ) ! 〈M3〉〈M3〉 D rej accrej 〈D〉〈D〉 loop acc ? …

Unrecognizable languages How about languages that are not recognizable? The language A TM is recognizable but not decidable. A TM = {( 〈 M 〉, w): M is a TM that does not accept w} = {( 〈 M 〉, w): M rejects or loops on input w} The language A TM is not recognizable.

Unrecognizable languages General Theorem We know A TM is recognizable, so if A TM were also, then A TM would be decidable Impossible by Turing’s Theorem If L and L are both recognizable, then L is decidable.

Unrecognizable languages Proof idea If L and L are both recognizable, then L is decidable. M rej/loop if w ∉ L accept if w ∈ L w M’ rej/loop if w ∈ L accept if w ∉ L w w accept reject

Unrecognizable languages Proof attempt Let M = TM for L, M’ = TM for L On input w, Simulate M on input w. If it accepts, accept. Simulate M’ on input w. If it accepts, reject. If L and L are both recognizable, then L is decidable. Problem: If M loops on w, we will never get to step 2! ① ②

Bounded simulation Proof Let M = TM for L, M’ = TM for L On input w, Simulate M on input w for k steps. If it accepts, accept. Simulate M’ on input w for k steps. If it accepts, reject. If L and L are both recognizable, then L is decidable. For k = 0 to infinity

Another undecidable language To prove this, we will show that … so by Turing’s Theorem it is undecidable. HALT TM = {( 〈 M 〉, w): M is a TM that halts on input w} If HALT TM can be decided, so can A TM. HALT TM is an undecidable language

Undecidability of halting Suppose H decides HALT TM H reject if M loops on w accept if M halts on w 〈 M 〉, w If HALT TM can be decided, so can A TM. We want to use H to design D for A TM ? reject if M rejects or loops on w accept if M accepts w 〈 M 〉, w

Undecidability of halting H reject if M loops on w accept if M halts on w 〈 M 〉, w D reject if M rejects or loops on w accept if M accepts w 〈 M 〉, w H rej acc simulate M on w rej acc

More undecidable problems L 1 = { 〈 M 〉 : M is a TM that accepts input  } L 2 = { 〈 M 〉 : M is a TM that accepts some input } L 4 = { 〈 M, M’ 〉 : M and M’ accept the same inputs } Which of these are recognizable? L 3 = { 〈 M 〉 : M is a TM that accepts all inputs }

Gödel’s incompleteness theorem Proof due to Turing: Some mathematical statements are true but not provable. If every true statement is provable, then A TM can be decided.

Proof sketch of Gödel’s theorem Suppose every true statement has a proof For every TM M and input w, one of the statements is true, so it must have a proof M accepts wM does not accept w or

Proof sketch of Gödel’s theorem Consider this algorithm for A TM : On input 〈 M 〉, w : For k = 0 to infinity For all possible proofs p of length k If p is a proof of “ M accepts w ”, accept. If p is a proof of “ M does not accept w ”, reject. One of them is true, so it has a proof p Eventually, the algorithm will find this proof

Proof sketch of Godel’s theorem Consider this algorithm for A TM : On input 〈 M 〉, w : For k = 0 to infinity For all possible proofs p of length k If p is a proof of “ M accepts w ”, accept. If p is a proof of “ M does not accept w ”, reject. Recall that TMs can check if a proof p is correct