1 Turing’s Thesis

2 Turing’s thesis: Any computation carried out by mechanical means can be performed by a Turing Machine (1930)

3 Computer Science Law: A computation is mechanical if and only if it can be performed by a Turing Machine There is no known model of computation more powerful than Turing Machines

4 Definition of Algorithm: An algorithm for function is a Turing Machine which computes

5 When we say: There exists an algorithm Algorithms are Turing Machines We mean: There exists a Turing Machine that executes the algorithm

6 Turing Machines are “hardwired” they execute only one program A limitation of Turing Machines: Real Computers are re-programmable

7 Encoding Turing Machine

8 Alphabet Encoding Symbols: Encoding:

9 State Encoding States: Encoding: Head Move Encoding Move: Encoding:

10 Transition Encoding Transition: Encoding: separator

11 Machine Encoding Transitions: Encoding: separator

12 A Turing Machine is described with a binary string of 0’s and 1’s The set of Turing machines forms a language: each string of the language is the binary encoding of a Turing Machine Therefore:

13 Language of Turing Machines L = { , , , …… } (Turing Machine 1) (Turing Machine 2) ……

14 Countable Sets

15 Infinite sets are either: Countable or Uncountable

16 Countable set: There is a one to one correspondence between elements of the set and Natural numbers Any finite set Any Countably infinite set: or

17 Example: Even integers: The set of even integers is countable Positive integers: Correspondence: corresponds to

18 Example:The set of rational numbers is countable Rational numbers:

19 Naïve Proof Rational numbers: Positive integers: Correspondence: Doesn’t work: we will never count numbers with nominator 2:

20 Better Approach

21

22 Rational Numbers: Correspondence: Positive Integers:

23 We proved: the set of rational numbers is countable by describing an enumeration procedure

24 Definition An enumeration procedure for is a Turing Machine that generates all strings of one by one Let be a set of strings and Each string is generated in finite time

25 Enumeration Machine for Finite time: strings output (on tape)

26 Enumeration Machine Configuration Time 0 Time

27 Time

28 If for a set there is an enumeration procedure, then the set is countable Observation:

29 Example: The set of all strings is countable We will describe an enumeration procedure Proof:

30 Naive procedure: Produce the strings in lexicographic order: Doesn’t work: strings starting with will never be produced

31 Better procedure: 1. Produce all strings of length 1 2. Produce all strings of length 2 3. Produce all strings of length 3 4. Produce all strings of length Proper Order

32 Produce strings in Proper Order: length 2 length 3 length 1

33 Theorem: The set of all Turing Machines is countable Proof: Find an enumeration procedure for the set of Turing Machine strings Any Turing Machine can be encoded with a binary string of 0’s and 1’s

34 1. Generate the next binary string of 0’s and 1’s in proper order 2. Check if the string describes a Turing Machine YES: if YES: print string on output tape NO: if NO: ignore string Enumeration Procedure: Repeat

35 Uncountable Sets

36 A set is uncountable if it is not countable Definition:

37 Theorem: Let be an infinite countable set The powerset of is uncountable

38 Proof: Since is countable, we can write Elements of

39 Elements of the powerset have the form: ……

40 We encode each element of the power set with a binary string of 0’s and 1’s Powerset element Encoding

41 Let’s assume (for contradiction) that the powerset is countable. Then: we can enumerate the elements of the powerset

42 Powerset element Encoding

43 Take the powerset element whose bits are the complements in the diagonal

44 New element: (birary complement of diagonal)

45 The new element must be some of the powerset However, that’s impossible: the i-th bit of must be the complement of itself from definition of Contradiction!!!

46 Since we have a contradiction: The powerset of is uncountable

47 An Application: Languages Example Alphabet : The set of all Strings: infinite and countable

48 Example Alphabet : The set of all Strings: infinite and countable A language is a subset of :

49 Example Alphabet : The set of all Strings: infinite and countable The powerset of contains all languages: uncountable

50 Languages: uncountable Turing machines: countable There are more languages than Turing Machines

51 There are some languages not accepted by Turing Machines (These languages cannot be described by algorithms) Conclusion:

52 Languages Accepted by Turing Machines Languages not accepted by Turing Machines

53 Recursively Enumerable and Recursive Languages

54 Definition: A language is recursively enumerable if some Turing machine accepts it

55 For string : Let be a recursively enumerable language and the Turing Machine that accepts it ifthen halts in a final state ifthen halts in a non-final state or loops forever

56 Definition: A language is recursive if some Turing machine accepts it and halts on any input string In other words: A language is recursive if there is a membership algorithm for it

57 For string : Let be a recursive language and the Turing Machine that accepts it ifthen halts in a final state ifthen halts in a non-final state

58 Facts: 1. There is a specific language which is not recursively enumerable (not accepted by any Turing Machine) 2. There is a specific language which is recursively enumerable but not recursive

59 Recursive Recursively Enumerable Non Recursively Enumerable

60 A Language which is not Recursively Enumerable

61 We want to find a language that is not Recursively Enumerable This language is not accepted by any Turing Machine

62 Consider alphabet Strings:

63 Consider Turing Machines that accept languages over alphabet They are countable:

64 Example language accepted by Alternative representation

65

66 Consider the language consists from the 1’s in the diagonal

67

68 Consider the language consists of the 0’s in the diagonal

69

70 Theorem: Language is not recursively enumerable

71 Proof: is recursively enumerable Assume for contradiction that There must exist some machine that accepts

72 Question:

73 Answer:

74 Question:

75 Answer:

76 Question:

77 Answer:

78 Similarly: for any Because either: or

79 Therefore, the machine cannot exist Therefore, the language is not recursively enumerable End of Proof

80 Observation: There is no algorithm that describes (otherwise would be accepted by some Turing Machine)

81 Recursive Recursively Enumerable Non Recursively Enumerable

82 A Language which is Recursively Enumerable and not Recursive

83 We want to find a language which There is a Turing Machine that accepts the language The machine doesn’t halt on some input Is recursively enumerable But not recursive

84 We will prove that the language Is recursively enumerable but not recursive

85

86 The language Theorem: is recursively enumerable

87 Proof: We will give a Turing Machine that accepts

88 Turing Machine that accepts For any input string Compute, for which Find Turing machine (using an enumeration procedure for Turing Machines) Simulate on input If accepts, then accept End of Proof

89 Observation: Recursively enumerable Not recursively enumerable (Thus, also not recursive)

90 Theorem: The language is not recursive

91 Proof: Assume for contradiction that is recursive Then is recursive: Take the Turing Machine that accepts halts on any input: If accepts then reject If rejects then accept

92 Therefore: is recursive But we know: is not recursively enumerable thus, not recursive CONTRADICTION!!!!

93 Therefore, is not recursive End of Proof

94 Recursive Recursively Enumerable Non Recursively Enumerable

95 The Chomsky Hierarchy (Formal Language Hierarchy)

96 Context-Sensitive Grammars: and: Productions String of variables and terminals String of variables and terminals

97 Non-recursively enumerable Recursively-enumerable Recursive Context-sensitive Context-free Regular The Chomsky Hierarchy

98 Decidability

99 Consider problems with answer YES or NO Examples: Does Machine have three states ? Is string a binary number? Does DFA accept any input?

100 A problem is decidable if some Turing machine decides (solves) the problem Decidable problems: Does Machine have three states ? Is string a binary number? Does DFA accept any input?

101 Turing Machine Input problem instance YES NO The Turing machine that decides (solves) a problem answers YES or NO for each instance of the problem

102 The machine that decides (solves) a problem: If the answer is YES then halts in a yes state If the answer is NO then halts in a no state These states may not be final states

103 YES states NO states Turing Machine that decides a problem YES and NO states are halting states

104 Some problems are undecidable: which means: there is no Turing Machine that solves all instances of the problem A simple undecidable problem: The membership problem

105 The Membership Problem Input:Turing Machine String Question: Does accept ?

106 Theorem: The membership problem is undecidable Proof:Assume for contradiction that the membership problem is decidable (there are and for which we cannot decide whether )

107 Thus, there exists a Turing Machine that solves the membership problem YES NO acceptsrejects

108 Let be a recursively enumerable language Let be the Turing Machine that accepts We will prove that is also recursive: we will describe a Turing machine that accepts and halts on any input

109 accepts ? NO YES accept Turing Machine that accepts and halts on any input reject

110 Therefore,is recursive But there are recursively enumerable languages which are not recursive Contradiction!!!! Since is chosen arbitrarily, every recursively enumerable language is also recursive

111 Therefore, the membership problem is undecidable END OF PROOF

112 Another famous undecidable problem: The halting problem

113 The Halting Problem Input:Turing Machine String Question: Does halt on input ?

114 If the halting problem was decidable then every recursively enumerable language would be recursive

115 Theorem: The halting problem is undecidable Proof:Assume for contradiction that the halting problem is decidable

116 There exists Turing Machine that solves the halting problem YEShalts on doesn’t halt on NO

117 Let be a recursively enumerable language Let be the Turing Machine that accepts We will prove that is also recursive: we will describe a Turing machine that accepts and halts on any input

118 halts on ? YES NO Run with input reject accept reject Turing Machine that accepts and halts on any input Halts on final state Halts on non-final state

119 Thereforeis recursive But there are recursively enumerable languages which are not recursive Contradiction!!!! Since is chosen arbitrarily, every recursively enumerable language is also recursive

120 Therefore, the halting problem is undecidable END OF PROOF