1 CD5560 FABER Formal Languages, Automata and Models of Computation Lecture 12 Mälardalen University 2005

2 Content Introduction Universal Turing Machine Chomsky Hierarchy Decidability Reducibility Uncomputable Functions Rice’s Theorem Decidability Continued

3 Who was Alan Turing? Founder of computer science, mathematician, philosopher, codebreaker, visionary man before his time.

4 Alan Turing 1912 (23 June): Birth, London : Sherborne School 1930: Death of friend Christopher Morcom : Undergraduate at King's College, Cambridge University : Quantum mechanics, probability, logic 1935: Elected fellow of King's College, Cambridge 1936: The Turing machine, computability, universal machine : Princeton University. Ph.D. Logic, algebra, number theory : Return to Cambridge. Introduced to German Enigma cipher machine : The Bombe, machine for Enigma decryption : Breaking of U-boat Enigma, saving battle of the Atlantic

5 Alan Turing : Chief Anglo-American crypto consultant. Electronic work. 1945: National Physical Laboratory, London 1946: Computer and software design leading the world : Programming, neural nets, and artificial intelligence 1948: Manchester University 1949: First serious mathematical use of a computer 1950: The Turing Test for machine intelligence 1951: Elected FRS. Non-linear theory of biological growth 1952: Arrested as a homosexual, loss of security clearance : Unfinished work in biology and physics 1954 (7 June): Death (suicide) by cyanide poisoning, Wilmslow, Cheshire.

6 Hilbert’s Program, 1900 Hilbert’s hope was that mathematics would be reducible to finding proofs (manipulating the strings of symbols) from a fixed system of axioms, axioms that everyone could agree were true. Can all of mathematics be made algorithmic, or will there always be new problems that outstrip any given algorithm, and so require creative acts of mind to solve?

7 TURING MACHINES

8 Turing’s "Machines". These machines are humans who calculate. (Wittgenstein) A man provided with paper, pencil, and rubber, and subject to strict discipline, is in effect a universal machine. (Turing)

Tape Read-Write head Control Unit Standard Turing Machine

Read-Write head No boundaries -- infinite length The head moves Left or Right The Tape

Read-Write head 1. Reads a symbol 2. Writes a symbol 3. Moves Left or Right The head at each time step:

12 Example Time Time Reads 2. Writes 3. Moves Left

13 Head starts at the leftmost position of the input string Blank symbol head Input string The Input String

14 Read Write Move Left Move Right States & Transitions

Time Time 2

16 Determinism Allowed Not Allowed No lambda transitions allowed in TM! Turing Machines are deterministic

17 Formal Definitions for Turing Machines

18 Transition Function

19 Turing Machine Transition function Initial state blank Final states States Input alphabet Tape alphabet

20 Universal Turing Machine

21 A limitation of Turing Machines: Better: re-programmable machines. Turing Machines are “hardwired” they execute only one program

22 Solution:Universal Turing Machine Reprogrammable machine Simulates any other Turing Machine Characteristics:

23 Universal Turing Machine simulates any other Turing Machine Input of Universal Turing Machine Description of transitions of Initial tape contents of

24 Universal Turing Machine Description of Three tapes Tape Contents of Tape 2 State of Tape 3 Tape 1

25 We describe Turing machine as a string of symbols: We encode as a string of symbols Description of Tape 1

26 Alphabet Encoding Symbols: Encoding:

27 State Encoding States: Encoding: Head Move Encoding Move: Encoding:

28 Transition Encoding Transition: Encoding: separator

29 Machine Encoding Transitions: Encoding: separator

30 Tape 1 contents of Universal Turing Machine: encoding of the simulated machine as a binary string of 0’s and 1’s

31 A Turing Machine is described with a binary string of 0’s and 1’s. The set of Turing machines forms a language: Each string of the language is the binary encoding of a Turing Machine. Therefore:

32 Language of Turing Machines L = { , , , …… } (Turing Machine 1) (Turing Machine 2) ……

33 The Chomsky Hierarchy

34 Unrestricted Grammars Productions String of variables and terminals String of variables and terminals

35 Example of unrestricted grammar

36 A language is recursively enumerable if and only if it is generated by an unrestricted grammar. Theorem

37 Context-Sensitive Grammars and Productions String of variables and terminals String of variables and terminals

38 The language is context-sensitive:

39 A language is context sensitive if and only if it is accepted by a Linear-Bounded automaton. Theorem

40 Linear Bounded Automata (LBAs) are the same as Turing Machines with one difference: The input string tape space is the only tape space allowed to use.

41 Left-end marker Input string Right-end marker Working space in tape All computation is done between end markers. Linear Bounded Automaton (LBA)

42 There is a language which is context-sensitive but not recursive. Observation

43 Non-recursively enumerable Recursively-enumerable Recursive Context-sensitive Context-free Regular The Chomsky Language Hierarchy

44 Decidability

45 Consider problems with answer YES or NO. Examples Does Machine have three states ? Is string a binary number? Does DFA accept any input?

46 A problem is decidable if some Turing machine solves (decides) the problem. Decidable problems: Does Machine have three states ? Is string a binary number? Does DFA accept any input?

47 Turing Machine Input problem instance YES NO The Turing machine that solves a problem answers YES or NO for each instance.

48 The machine that decides a problem: If the answer is YES then halts in a yes state If the answer is NO then halts in a no state These states may not be the final states.

49 YES NO Turing Machine that decides a problem YES and NO states are halting states

50 Difference between Recursive Languages (“Acceptera”) and Decidable problems (“Avgöra”) The YES states may not be final states. For decidable problems:

51 Some problems are undecidable: There is no Turing Machine that solves all instances of the problem.

52 A famous undecidable problem: The halting problem

53 The Halting Problem Input: Turing Machine String Question:Does halt on ?

54 Theorem The halting problem is undecidable. Proof Assume to the contrary that the halting problem is decidable.

55 There exists Turing Machine that solves the halting problem YEShalts on doesn’t halt on NO

56 Input: initial tape contents Encoding of String YES NO Construction of

57 Construct machine returns YES then loop forever.If returns NO then halt.If

58 NO Loop forever YES

59 Construct machine Input: If halts on input Then loop forever Else halt (machine )

60 copy

61 Run machine with input itself Input: If halts on input Then loop forever Else halt (machine )

62 on input If halts then loops forever. If doesn’t halt then it halts. : CONTRADICTION !

63 This means that The halting problem is undecidable. END OF PROOF

64 Another proof of the same theorem If the halting problem was decidable then every recursively enumerable language would be recursive.

65 Theorem The halting problem is undecidable. Proof Assume to the contrary that the halting problem is decidable.

66 There exists Turing Machine that solves the halting problem. YEShalts on doesn’t halt on NO

67 Let be a recursively enumerable language. Let be the Turing Machine that accepts. We will prove that is also recursive: We will describe a Turing machine that accepts and halts on any input.

68 halts on ? YES NO Run with input reject accept reject Turing Machine that accepts and halts on any input Halts on final state Halts on non-final state

69 Therefore is recursive. But there are recursively enumerable languages which are not recursive. Contradiction! Since is chosen arbitrarily, we have proven that every recursively enumerable language is also recursive.

70 Therefore, the halting problem is undecidable. END OF PROOF

71 A simple undecidable problem: The membership problem

72 The Membership Problem Input: Turing Machine String Question:Does accept ?

73 Theorem The membership problem is undecidable. Proof Assume to the contrary that the membership problem is decidable.

74 There exists a Turing Machine that solves the membership problem YES accepts NO rejects

75 Let be a recursively enumerable language. Let be the Turing Machine that accepts. We will prove that is also recursive: We will describe a Turing machine that accepts and halts on any input.

76 accepts ? NO YES accept Turing Machine that accepts and halts on any input reject

77 Therefore, is recursive. But there are recursively enumerable languages which are not recursive. Contradiction! Since is chosen arbitrarily, we have proven that every recursively enumerable language is also recursive.

78 Therefore, the membership problem is undecidable. END OF PROOF

79 Reducibility

80 Problem is reduced to problem If we can solve problem then we can solve problem.

81 If is decidable then is decidable. If is undecidable then is undecidable. Problem is reduced to problem

82 Example the halting problem reduced to the state-entry problem.

83 The state-entry problem Inputs: Question: Turing Machine State String Does enter state on input ?

84 Theorem The state-entry problem is undecidable. Proof Reduce the halting problem to the state-entry problem.

85 Suppose we have an algorithm (Turing Machine) that solves the state-entry problem. We will construct an algorithm that solves the halting problem.

86 Algorithm for state-entry problem YES NO enters doesn’t enter Assume we have the state-entry algorithm:

87 Algorithm for Halting problem YES NO halts on doesn’t halt on We want to design the halting algorithm:

88 Modify input machine Add new state From any halting state add transitions to halting states Single halt state

89 halts halts on state if and only if

90 Algorithm for halting problem Inputs:machine and string 2. Run algorithm for state-entry problem with inputs:,, 1. Construct machine with state

91 GenerateState-entry algorithm Halting problem algorithm YES NO YES NO

92 Since the halting problem is undecidable, it must be that the state-entry problem is also undecidable. END OF PROOF We reduced the halting problem to the state-entry problem.

93 Another example The halting problem reduced to the blank-tape halting problem.

94 The blank-tape halting problem Input:Turing Machine Question:Doeshalt when started with a blank tape?

95 Proof Reduce the halting problem to the blank-tape halting problem. Theorem The blank-tape halting problem is undecidable.

96 Suppose we have an algorithm for the blank-tape halting problem. We will construct an algorithm for the halting problem.

97 Algorithm for blank-tape halting problem YES NO halts on blank tape doesn’t halt on blank tape Assume we have the blank-tape halting algorithm

98 Algorithm for halting problem YES NO halts on doesn’t halt on We want to design the halting algorithm:

99 Construct a new machine On blank tape writes Then continues execution like then write step 1step2 if blank tape execute with input

100 halts on input string halts when started with blank tape. if and only if

101 Algorithm for halting problem 1. Construct 2. Run algorithm for blank-tape halting problem with input Inputs: machine and string

102 Generate Blank-tape halting algorithm Halting problem algorithm YES NO YES NO

103 Since the halting problem is undecidable, the blank-tape halting problem is also undecidable. END OF PROOF We reduced the halting problem to the blank-tape halting problem.

104 Summary of Undecidable Problems Halting Problem Does machine halt on input ? Membership problem Does machine accept string ? Is a string member of a recursively enumerable language ? ) (In other words:

105 Does machine halt when starting on blank tape? Blank-tape halting problem State-entry Problem: Does machine enter state on input ?

106 Uncomputable Functions

107 Uncomputable Functions A function is uncomputable if it cannot be computed for all of its domain. Domain Range

108 An uncomputable function: maximum number of moves until any Turing machine with states halts when started with the blank tape. Example

109 Theorem Function is uncomputable. Then the blank-tape halting problem is decidable. Proof Assume to the contrary that is computable.

110 Algorithm for blank-tape halting problem Input: machine 1. Count states of : 2. Compute 3. Simulate for steps starting with empty tape If halts then return YES otherwise return NO

111 Therefore, the blank-tape halting problem must be decidable. However, we know that the blank-tape halting problem is undecidable. Contradiction!

112 Therefore, function is uncomputable. END OF PROOF

113 Rice’s Theorem

114 Non-trivial properties of recursively enumerable languages: any property possessed by some (not all) recursively enumerable languages. Definition

115 Some non-trivial properties of recursively enumerable languages: is empty is finite contains two different strings of the same length

116 Rice’s Theorem Any non-trivial property of a recursively enumerable language is undecidable.

117 We will prove some non-trivial properties without using Rice’s theorem.

118 Theorem For any recursively enumerable language it is undecidable whether it is empty. Proof We will reduce the membership problem to the problem of deciding whether is empty.

119 Membership problem: Does machine accept string ?

120 Algorithm for empty language problem YES NO Assume we have the empty language algorithm: Let be the machine that accepts empty not empty

121 Algorithm for membership problem YES NO accepts rejects We will design the membership algorithm:

122 First construct machine : When enters a final state, compare original input string with. Accept if original input is the same as.

123 is not empty if and only if

124 Algorithm for membership problem Inputs: machine and string 1. Construct 2. Determine if is empty YES: then NO: then

125 construct Check if is empty YES NO YES Membership algorithm

126 Since the membership problem is undecidable, the empty language problem is also undecidable. END OF PROOF We reduced the empty language problem to the membership problem.

127 Decidability …continued…

128 Theorem For a recursively enumerable language it is undecidable to determine whether is finite. Proof We will reduce the halting problem to the finite language problem.

129 Assume we have the finite language algorithm: Algorithm for finite language problem YES NO finite not finite Let be the machine that accepts

130 We will design the halting problem algorithm: Algorithm for Halting problem YES NO halts on doesn’t halt on

131 First construct machine. When enters a halt state, accept any input (infinite language). Initially, simulates on input. Otherwise accept nothing (finite language).

132 halts on is not finite. if and only if

133 Algorithm for halting problem: Inputs: machine and string 1. Construct 2. Determine if is finite YES: then doesn’t halt on NO: then halts on

134 construct Check if is finite YES NO YES Machine for halting problem

135 Since the halting problem is undecidable, the finite language problem is also undecidable. END OF PROOF We reduced the finite language problem to the halting problem.

136 Theorem For a recursively enumerable language it is undecidable whether contains two different strings of same length. Proof We will reduce the halting problem to the two strings of equal length- problem.

137 Assume we have the two-strings algorithm: Let be the machine that accepts Algorithm for two-strings problem YES NO contains doesn’t contain two equal length strings

138 We will design the halting problem algorithm: Algorithm for Halting problem YES NO halts on doesn’t halt on

139 First construct machine. When enters a halt state, accept symbols or. Initially, simulates on input. (two equal length strings)

140 halts on if and only if accepts and (two equal length strings)

141 Algorithm for halting problem Inputs: machine and string 1. Construct 2. Determine if accepts two strings of equal length YES: then halts on NO: then doesn’t halt on

142 construct Check if has two equal length strings YES NO YES NO Machine for halting problem

143 Since the halting problem is undecidable, the two strings of equal length problem is also undecidable. END OF PROOF We reduced the two strings of equal length - problem to the halting problem.

144 Rices sats Om  är en mängd av Turing-accepterbara språk som innehåller något men inte alla sådana språk, så kan ingen TM avgöra för ett godtyckligt Turing-accepterbart språk L om L tillhör  eller ej.

145 Exempel Givet en Turingmaskin M, kan man avgöra om alla strängar som accepteras av M börjar och slutar på samma tecken?

146 Oavgörbart Problemet handlar om en icke-trivial språkegenskap. Det finns TM:er vars accepterade strängar har egenskapen i fråga, och det finns TM:er vars accepterade strängar inte har egenskapen.

147 Formellt:  = { L | TM accepterbara språk vars strängar börjar och slutar på samma tecken. }