1. CSCI 3130: Formal languages and automata theory Andrej Bogdanov http://www.cse.cuhk.edu.hk/~andrejb/csc3130 The Chinese University of Hong Kong Decidable and undecidable languages Fall 2011

2. Problems about automata We can formulate this question as a language: q0q0 q1q1 b b a a Doesaccept input abb ? A DFA = { 〈 D, w 〉 : D is a DFA that accepts input w} ((q0,q1)(a,b)((q0,a,q0)(q0,b,q1)(q1,a,q0)(q1,b,q1)(q0)(q1))(abb) D = (Q, , , q 0, F) w Is A DFA decidable?

3. Problems about automata A DFA = { 〈 D, w 〉 : D is a DFA that accepts input w} pseudocode: On input 〈 D, w 〉, where D = (Q, , , q 0, F) : Set q := q 0 For i := 1 to length(w) : q :=  (q, w i ) If q  ∈  F accept, else reject TM description: On input 〈 D, w 〉, where D is a DFA, w is a string Simulate D on input w If simulation ends in acc state, accept. Otherwise, reject.

4. Problems about automata A DFA = { 〈 D, w 〉 : D is a DFA that accepts input w} Turing Machine details: Check input is in correct format. (Transition function is complete, no duplicate transitions) Perform simulation: ((q0,q1)(a,b)((q0,a,q0)(q0,b,q1)(q1,a,q0)(q1,b,q1)(q0)(q1))(abb).. stateinput symbol.... q acc

5. Problems about automata A DFA = { 〈 D, w 〉 : D is a DFA that accepts input w} Turing Machine details: Check input is in correct format. (Transition function is complete, no duplicate transitions) Perform simulation: Put markers on start state of D and first symbol of w Until marker for w reaches last symbol: Update both markers If state marker is on accepting state, accept. Else reject. decidable

6. Acceptance problems about automata A DFA = { 〈 D, w 〉 : D is a DFA that accepts input w} A NFA = { 〈 N, w 〉 : N is an NFA that accepts w} A REX = { 〈 R, w 〉 : R is a regular expression that generates w} Which of these is decidable? decidable

7. Acceptance problems about automata The following TM decides A DFA : A DFA = { 〈 D, w 〉 : D is a DFA that accepts input w} decidable M := On input 〈 D, w 〉, where D is a DFA and w is a string Simulate D on input w If the simulation ends in acc state of D, accept. If it doesn’t, reject.

8. Acceptance problems about automata The following TM decides A NFA : A NFA = { 〈 N, w 〉 : N is an NFA that accepts input w} decidable N := On input 〈 N, w 〉, where N is an NFA and w is a string Convert N to a DFA D using the conversion procedure from Lecture 2 Run the TM M for A DFA on input 〈 D, w 〉 If M accepts, accept. Otherwise, reject.

9. Acceptance problems about automata The following TM decides A REX : A REX = { 〈 R, w 〉 : R is a regular expression that generates w} decidable P := On input 〈 R, w 〉, where R is a reg exp and w is a string Convert R to an NFA N using the conversion procedure from Lecture 4 Run the TM N for A NFA on input 〈 N, w 〉 If N accepts, accept. Otherwise, reject.

10. Other problems about automata MIN DFA = { 〈 D 〉 : D is a minimal DFA } On input 〈 D 〉, where D is a DFA decidable R := Run the distinguishable states algorithm from Lecture 7 on D If every pair of states is distinguishable, accept. Otherwise, reject. The following TM decides MIN DFA :

11. Other problems about automata EQ DFA = { 〈 D 1, D 2 〉 : D 1, D 2 are DFAs and L(D 1 ) = L(D 2 )} On input 〈 D 1, D 2 〉, where D 1 and D 2 are DFAs decidable S := Run the DFA minimization algorithm from Lecture 7 on D 1 to obtain a DFA D 1 ’ The following TM decides EQ DFA : If D 1 ’ = D 2 ’ accept, otherwise reject. Run the DFA minimization algorithm from Lecture 7 on D 2 to obtain a DFA D 2 ’

12. Other problems about automata E DFA = { 〈 D 〉 : D is a DFAs and L(D) is empty } On input 〈 D 〉, where D is a DFA decidable T := Run the TM S for EQ DFA on input 〈 D, 〉 The following TM decides E DFA : If S accepts accept, otherwise reject.

13. Problems about context-free grammars A CFG = { 〈 G, w 〉 : G is a CFG that generates w} On input 〈 G, w 〉, where G is a CFG and w is a string decidable V := If the CYK algorithm produces a parse tree, accept. Eliminate the nullable and unit productions from G Convert G to Chomsky Normal Form Run the Cocke-Younger-Kasami algorithm on 〈 G, w 〉 Otherwise, reject.

14. Decidability of context-free languages Every context-free language is decidable. Let L be a context-free language. There is a CFG G for L. The following TM decides L : M G := On input w, Run TM V on input 〈 G, w 〉. If V accepts accept, otherwise reject.

15. Are all languages about CFGs decidable? EQ CFG = { 〈 G 1, G 2 〉 : G 1 and G 2 are context-free grammars that generate the same strings } undecidable What’s the difference between EQ DFA and EQ CFG ? To decide EQ DFA, we minimized both DFAs But there is no method that, given a CFG or PDA, produces a unique equivalent minimal CFG or PDA

16. The universal Turing Machine and undecidability

17. Turing Machines versus computers computer data program output A computer is a machine that manipulates data according to a list of instructions. How does a Turing Machine take a program as part of its input?

18. The Universal Turing Machine U input x for M program 〈 M 〉 whatever M does on x The universal TM U takes as inputs a program M and a string x and simulates M on x The program M itself is specified as a TM!

19. Turing Machines as strings This Turing Machine can be described by the string 〈 M 〉 = (q,qa,qr)(0,1)(0,1, ☐ ) ((q,q, ☐ / ☐ R) (q,qa,0/0R) (q,qr,1/1R)) (q)(qa)(qr) M q qaqa qrqr ☐/☐R☐/☐R 0/0R 1/1R A Turing Machine is (Q, , , , q 0, q acc, q rej )

20. The universal Turing Machine U (q,qa,qr)(0,1)(0,1,001 input w for M program 〈 M 〉 U := On input 〈 M, w 〉, Simulate M on input w If M enters accept state, accept. If M enters reject state, reject.

21. Acceptance of Turing Machines A TM = { 〈 M, w 〉 : M is a TM that accepts w} U := On input 〈 M, w 〉, Simulate M on input w M accepts wM rejects wM loops on w U accepts 〈 M, w 〉 U rejects 〈 M, w 〉 U loops on 〈 M, w 〉 TM U recognizes but does not decide A TM

22. Recognizing versus deciding The language recognized by a TM is the set of all inputs that make it accept A TM decides language L if it recognizes L and halts (does not loop) on every input q acc q rej accept rejectloop halt

23. Undecidability Turing’s Theorem: Before we show this, let’s observe one thing: The language A TM is undecidable. A Turing Machine M can be given its own description 〈 M 〉 as an input!

24. Turing’s theorem A TM that decides A TM is so potent that it will destroy itself.

25. Proof of Turing’s Theorem Proof by contradiction: Suppose A TM is decidable. Then there is a TM H that decides A TM : H 〈 M, w 〉 accept if M accepts w reject if M rejects w or M loops on w 〈M, 〈M〉〉〈M, 〈M〉〉 accept if M accepts 〈 M 〉 reject if M rejects 〈 M 〉 or M loops on 〈 M 〉 What happens when w = 〈 M 〉 ?

26. Proof of undecidability H 〈 M, 〈 M 〉〉 accept if M accepts 〈 M 〉 reject if M rejects 〈 M 〉 or M loops on 〈 M 〉 Let H’ be a TM that does the opposite of H H acc rej acc rej H’H’ To go from H to H’, just switch its accept and reject states

27. Proof of undecidability H 〈M, 〈M〉〉〈M, 〈M〉〉 accept if M accepts 〈 M 〉 reject if M rejects 〈 M 〉 or M loops on 〈 M 〉 H’H’ 〈 M, 〈 M 〉〉 reject if M accepts 〈 M 〉 if M rejects 〈 M 〉 or M loops on 〈 M 〉 accept

28. Proof of undecidability H’H’ 〈 M, 〈 M 〉〉 reject if M accepts 〈 M 〉 if M rejects 〈 M 〉 or M loops on 〈 M 〉 accept Let D be the following TM: copy 〈M〉〈M〉 〈 M, 〈 M 〉〉 H’H’

29. Proof of undecidability D 〈M〉〈M〉 reject if M accepts 〈 M 〉 if M rejects 〈 M 〉 or M loops on 〈 M 〉 accept What happens when M = D ? 〈D〉〈D〉 reject if D accepts 〈 D 〉 if D rejects 〈 D 〉 or D loops on 〈 D 〉 If D accepts 〈 D 〉 then D rejects 〈 D 〉 If D rejects 〈 D 〉 then D accepts 〈 D 〉 so D does not exist! H never loops, so D never loops either

30. Proof of undecidability: conclusion Proof by contradiction –We assumed A TM was decidable –Then we built Turing Machines H, H’, D –But D does not exist! Conclusion The language A TM is undecidable.

31. What happened? M1M1 M2M2 M3M3 …  01 00 … accrej acc rejacclooprej looprej … … Turing Machines all possible inputs w We can write an infinite table for every pair (M, w) all possible

32. What happened? M1M1 M2M2 M3M3 … 〈M1〉〈M1〉〈M2〉〈M2〉 〈M4〉〈M4〉 … acclooprejacc rejaccrejacc looprejlooprej … … Now let’s look only at those w that describe a TM M 〈M3〉〈M3〉

33. What happened? M1M1 M2M2 … 〈M1〉〈M1〉〈M2〉〈M2〉 〈M4〉〈M4〉 … acclooprejacc rejaccrejacc … … If A TM is decidable, then TM D is in the table 〈M3〉〈M3〉 D rej accrej … …

34. What happened? M1M1 M2M2 … 〈M1〉〈M1〉〈M2〉〈M2〉 〈M4〉〈M4〉 … acclooprejacc rejaccrejacc … … D does the opposite of the diagonal entries 〈M3〉〈M3〉 D rej accrej D 〈M〉〈M〉 reject if M accepts 〈 M 〉 accept if M rejects or loops on 〈 M 〉

35. What happened? M1M1 M2M2 … 〈M1〉〈M1〉〈M2〉〈M2〉 〈M4〉〈M4〉 … acclooprejacc rejaccrejacc … … We run into trouble when we look at (D, 〈 D 〉 ) ! 〈M3〉〈M3〉 D rej accrej 〈D〉〈D〉 loop acc ? …

36. Unrecognizable languages How about languages that are not recognizable? The language A TM is recognizable but not decidable. A TM = { 〈 M, w 〉 : M is a TM that does not accept w} = { 〈 M, w 〉 : M rejects or loops on input w} The language A TM is not recognizable.

37. Unrecognizable languages Theorem We know A TM is recognizable, so if A TM were also, then A TM would be decidable But Turing’s Theorem says A TM is not decidable If L and L are both recognizable, then L is decidable.

38. Unrecognizable languages Proof idea If L and L are both recognizable, then L is decidable. M rej/loop if w ∉ L accept if w ∈ L w M’M’ rej/loop if w ∈ L accept if w ∉ L w w accept reject

39. Unrecognizable languages Let M = TM for L, M’ = TM for L On input w, Simulate M on input w. If it accepts, accept. Simulate M’ on input w. If it accepts, reject. If L and L are both recognizable, then L is decidable. Problem: If M loops on w, we will never get to step 2! ① ②

40. Bounded simulation Turing Machine that decides L : Let M = TM for L, M’ = TM for L On input w, Do t transitions of M on w. If it accepts, accept. Do t transitions of M’ on w. If it accepts, reject. If L and L are both recognizable, then L is decidable. For t := 0 to infinity