Decidable and undecidable languages The Chinese University of Hong Kong Fall 2010 CSCI 3130: Formal languages and automata theory Decidable and undecidable languages Andrej Bogdanov http://www.cse.cuhk.edu.hk/~andrejb/csc3130

Problems about automata Does q0 q1 accept input abb? b We can formulate this question as a language: ADFA = {〈D, w〉: D is a DFA that accepts input w} Is ADFA decidable? ((q0,q1)(a,b)((q0,a,q0)(q0,b,q1)(q1,a,q0)(q1,b,q1)(q0)(q1))(abb) D = (Q, S, d, q0, F) w

Problems about automata ADFA = {〈D, w〉: D is a DFA that accepts input w} pseudocode: TM description: On input 〈D, w〉, where D = (Q, S, d, q0, F): On input 〈D, w〉, where D is a DFA, w is a string Set q := q0 Simulate D on input w For i := 1 to length(w): If simulation ends in acc state, accept. Otherwise, reject. q := d(q, wi) If q ∈ F accept, else reject

Problems about automata ADFA = {〈D, w〉: D is a DFA that accepts input w} Turing Machine details: Check input is in correct format. (Transition function is complete, no duplicate transitions) Perform simulation: state input symbol . . . . . . ((q0,q1)(a,b)((q0,a,q0)(q0,b,q1)(q1,a,q0)(q1,b,q1)(q0)(q1))(abb) qacc

Problems about automata decidable ADFA = {〈D, w〉: D is a DFA that accepts input w} Turing Machine details: Check input is in correct format. (Transition function is complete, no duplicate transitions) Perform simulation: Put markers on start state of D and first symol of w Until w marker reaches last symbol: Update both markers If state marker is on accepting state, accept. Else reject.

Acceptance problems about automata decidable ADFA = {〈D, w〉: D is a DFA that accepts input w} ANFA = {〈N, w〉: N is an NFA that accepts w} AREX = {〈R, w〉: R is a regular expression that generates w} Which of these is decidable?

Acceptance problems about automata decidable ADFA = {〈D, w〉: D is a DFA that accepts input w} The following TM decides ADFA: M := On input 〈D, w〉, where D is a DFA and w is a string Simulate D on input w If the simulation ends in acc state of D, accept. If it doesn’t, reject.

Acceptance problems about automata decidable ANFA = {〈N, w〉: N is an NFA that accepts input w} The following TM decides ANFA: N := On input 〈N, w〉, where N is an NFA and w is a string Convert N to a DFA D using the conversion procedure from Lecture 2 Run the TM M for ADFA on input 〈D, w〉 If M accepts, accept. Otherwise, reject.

Acceptance problems about automata decidable AREX = {R is a regular expression that generates w} The following TM decides AREX: P := On input 〈R, w〉, where R is a reg exp and w is a string Convert R to an NFA A using the conversion procedure from Lecture 4 Run the TM N for ANFA on input 〈A, w〉 If N accepts, accept. Otherwise, reject.

Other problems about automata decidable MINDFA = {〈D〉: D is a minimal DFA} The following TM decides MINDFA: R := On input 〈D〉, where D is a DFA Run the distinguishable states algorithm from Lecture 7 on D If every pair of states is distinguishable, accept. Otherwise, reject.

Other problems about automata decidable EQDFA = {〈D1, D2〉: D1, D2 are DFAs and L(D1) = L(D2)} The following TM decides EQDFA: S := On input 〈D1, D2〉, where D1 and D2 are DFAs Run the DFA minimization algorithm from Lecture 7 on D1 to obtain a DFA D1’ Run the DFA minimization algorithm from Lecture 7 on D2 to obtain a DFA D2’ If D1 = D2 accept, otherwise reject.

Other problems about automata decidable EDFA = {〈D〉: D is a DFAs and L(D) is empty} The following TM decides EDFA: T := On input 〈D〉, where D is a DFA Run the TM S for EQDFA on input 〈D, 〉 If S accepts accept, otherwise reject.

Problems about context-free grammars decidable ACFG = {〈G, w〉: G is a CFG that generates w} V := On input 〈G, w〉, where G is a CFG and w is a string Eliminate the nullable and unit productions from G Convert G to Chomsky Normal Form Run the Cocke-Younger-Kasami algorithm on 〈G, w〉 If the CYK algorithm produces a parse tree, accept. Otherwise, reject.

Decidability of context-free languages Every context-free language is decidable. Let L be a context-free language. There is a CFG G for L. The following TM decides L: MG := On input w, Run TM V on input 〈G, w〉. If V accepts accept, otherwise reject.

Are all languages about CFGs decidable? EQCFG = {〈G1, G2〉: G1 and G2 are context-free grammars that generate the same strings} undecidable What’s the difference between EQDFA and EQCFG? To decide EQDFA, we minimized both DFAs But there is no method that, given a CFG or PDA, produces a unique equivalent minimal CFG or PDA

The universal Turing Machine and undecidability

Turing Machines versus computers program output input A computer is a machine that manipulates data according to a list of instructions. How does a Turing Machine take a program as part of its input?

The Universal Turing Machine program 〈M〉 M on input x input x The universal TM U takes as inputs a program M and a string x and simulates M on x The program M itself is specified as a TM!

Turing Machines as strings qa 0/0R A Turing Machine is (Q, , , , q0, qacc, qrej) q 1/1R qr This Turing Machine can be described by the string 〈M〉 = (q,qa,qr)(0,1)(0,1,☐) ((q,q,☐/☐R) (q,qa,0/0R) (q,qr,1/1R)) (q)(qa)(qr)

The universal Turing Machine (q,qa,qr)(0,1)(0,1, 001 program 〈M〉 input w for M U := On input 〈M, w〉, Simulate M on input w If M enters accept state, accept. If M enters reject state, reject.

Acceptance of Turing Machines ATM = {〈M, w〉: M is a TM that accepts w} U := On input 〈M, w〉, Simulate M on input w M accepts w M rejects w M loops on w U accepts 〈M, w〉 U rejects 〈M, w〉 U loops on 〈M, w〉 TM U recognizes but does not decide ATM

Recognizing versus deciding qacc qrej accept reject loop halt The language recognized by a TM is the set of all inputs that make it accept A TM decides language L if it recognizes L and halts (does not loop) on every input

Undecidability Turing’s Theorem: The language ATM is undecidable. Before we show this, let’s observe one thing: The language ATM is undecidable. A Turing Machine M can be given its own description 〈M〉 as an input!

A TM that decides ATM is so potent that it will destroy itself. Turing’s theorem A TM that decides ATM is so potent that it will destroy itself.

Proof of Turing’s Theorem Proof by contradiction: Suppose ATM is decidable. Then there is a TM H that decides ATM: 〈M, 〈M〉〉 accept if M accepts 〈M〉 reject if M rejects 〈M〉 or M loops on 〈M〉 H 〈M, w〉 accept if M accepts w reject if M rejects w or M loops on w What happens when w = 〈M〉?

Proof of undecidability 〈M, 〈M〉〉 H accept if M accepts 〈M〉 reject if M rejects 〈M〉 or M loops on 〈M〉 Let H’ be a TM that does the opposite of H H = (Q, , , , q0, qacc, qrej) H’ H acc rej acc rej H’ = (Q, , , , q0, qrej, qacc)

Proof of undecidability 〈M, 〈M〉〉 H accept if M accepts 〈M〉 reject if M rejects 〈M〉 or M loops on 〈M〉 if M rejects 〈M〉 or M loops on 〈M〉 accept 〈M, 〈M〉〉 H’ reject if M accepts 〈M〉

Proof of undecidability if M rejects 〈M〉 or M loops on 〈M〉 H’ accept 〈M, 〈M〉〉 reject if M accepts 〈M〉 Let D be the following TM: 〈M〉 〈M, 〈M〉〉 copy H’

Proof of undecidability reject if D accepts 〈D〉 if D rejects 〈D〉 or D loops on 〈D〉 if M rejects 〈M〉 or M loops on 〈M〉 accept 〈M〉 D reject if M accepts 〈M〉 What happens when M = D? If D accepts 〈D〉 then D rejects 〈D〉 H never loops, so H’ and D never loop either so D does not exist! If D rejects 〈D〉 then D accepts 〈D〉

Proof of undecidability: conclusion Proof by contradiction We assumed ATM was decidable Then we had Turing Machines H, H’, D But D does not exist! Conclusion The language ATM is undecidable.

What happened? We can write an infinite table for every pair (M, w) all possible inputs w … e 1 00 M1 acc rej rej acc M2 rej acc loop rej … all possible Turing Machines M3 rej loop rej rej … … We can write an infinite table for every pair (M, w)

What happened? Now let’s look only at those w that describe a TM M … acc loop rej acc M2 rej acc rej acc … M3 loop rej loop rej … … Now let’s look only at those w that describe a TM M

What happened? If ATM is decidable, then TM D is in the table … 〈M1〉 acc loop rej acc M2 rej acc rej acc … … … D rej rej acc rej … … If ATM is decidable, then TM D is in the table

What happened? D does the opposite of the diagonal entries D … 〈M1〉 acc loop rej acc M2 rej acc rej acc … … … D rej rej acc rej D does the opposite of the diagonal entries 〈M〉 accept if M rejects or loops on 〈M〉 D reject if M accepts 〈M〉

What happened? ? We run into trouble when we look at (D, 〈D〉)! … 〈M1〉 acc loop rej acc loop M2 rej acc rej acc … acc … … … ? D rej rej acc rej We run into trouble when we look at (D, 〈D〉)!

Unrecognizable languages The language ATM is recognizable but not decidable. How about languages that are not recognizable? ATM = {〈M, w〉: M is a TM that does not accept w} = {〈M, w〉: M rejects or loops on input w} The language ATM is not recognizable.

Unrecognizable languages Theorem We know ATM is recognizable, so if ATM were also, then ATM would be decidable Impossible by Turing’s Theorem If L and L are both recognizable, then L is decidable.

Unrecognizable languages If L and L are both recognizable, then L is decidable. Proof idea w accept reject M rej/loop if w ∉ L accept if w ∈ L w M’ rej/loop if w ∈ L accept if w ∉ L

Unrecognizable languages If L and L are both recognizable, then L is decidable. Problem: If M loops on w, we will never get to step 2! Let M = TM for L, M’ = TM for L On input w, ① Simulate M on input w. If it accepts, accept. ② Simulate M’ on input w. If it accepts, reject.

Bounded simulation If L and L are both recognizable, then L is decidable. Decider for L: Let M = TM for L, M’ = TM for L On input w, For k = 0 to infinity Simulate M on input w for k steps. If it accepts, accept. Simulate M’ on input w for k steps. If it accepts, reject.