1 Language Recognition (11.4) and Turing Machines (11.5) Longin Jan Latecki Temple University Based on slides by Costas Busch from the courseCostas Busch and …

2 Three Equivalent Representations Finite automata Regular expressions Regular languages Each can describe the others Kleene’s Theorem: For every regular expression, there is a deterministic finite-state automaton that defines the same language, and vice versa.

3 EXAMPLE 1 Consider the language { a m b n | m, n  N}, which is represented by the regular expression a*b*. A regular grammar for this language can be written as follows: S   | aS | B B  b | bB.

4 Regular ExpressionRegular Grammar a*S   | aS (a+b)*S   | aS | bS a* + b*S   | A | B A  a | aA B  b | bB a*bS  b | aS ba*S  bA A   | aA (ab)*S   | abS

5 NFAs  Regular grammars Thus, the language recognized by FSA is a regular language Every NFA can be converted into a corresponding regular grammar and vice versa. Each symbol A of the grammar is associated with a non- terminal node of the NFA s A, in particular, start symbol S is associated with the start state s S. Every transition is associated with a grammar production: T(s A,a) = s B  A  aB. Every production B   is associated with final state s B. See Ex. 3, p. 771, and Ex. 4, p. 772.

6 Kleene’s Theorem Languages Generated by Regular Expressions Languages Recognized by FSA

7 Languages Generated by Regular Expressions Languages Recognized by FSA Languages Generated by Regular Expressions Languages Recognized by FSA We will show:

8 Proof - Part 1 For any regular expression the language is recognized by FSA (= is a regular language) Languages Generated by Regular Expressions Languages Recognized by FSA Proof by induction on the size of

9 Induction Basis Primitive Regular Expressions: NFAs regular languages

10 Inductive Hypothesis Assume for regular expressions and that and are regular languages

11 Inductive Step We will prove: Are regular Languages

12 By definition of regular expressions:

13 By inductive hypothesis we know: and are regular languages Regular languages are closed under: Union Concatenation Star We need to show: This fact is illustrated in Fig. 2 on p. 769.

14 Therefore: Are regular languages And trivially:is a regular language

15 Proof - Part 2 Languages Generated by Regular Expressions Languages Recognized by FSA For any regular language there is a regular expression with Proof by construction of regular expression

16 Since is regular take the NFA that accepts it Single final state

17 From construct the equivalent Generalized Transition Graph in which transition labels are regular expressions Example:

18 Another Example:

19 Reducing the states:

20 Resulting Regular Expression:

21 In General Removing states:

22 The final transition graph: The resulting regular expression:

23 DFA-regular languages Push down automata -Context-free Bounded Turing M’s-Context sensitive Turing machines -Phrase-structure Models of computing

24 Foundations The theory of computation and the practical application it made possible — the computer — was developed by an Englishman called Alan Turing.

25 Alan Turing 1912 (23 June): Birth, Paddington, London : Undergraduate at King's College, Cambridge University : Quantum mechanics, probability, logic 1936: The Turing machine, computability, universal machine : Princeton University. Ph.D. Logic, algebra, number theory : Return to Cambridge. Introduced to German Enigma cipher machine : The Bombe, machine for Enigma decryption : Breaking of U-boat Enigma, saving battle of the Atlantic 1946: Computer and software design leading the world. 1948: Manchester University 1949: First serious mathematical use of a computer 1950: The Turing Test for machine intelligence 1952: Arrested as a homosexual, loss of security clearance 1954 (7 June): Death (suicide) by cyanide poisoning, Wilmslow, Cheshire. —from Andrew Hodges

26 The Decision Problem In 1928 the German mathematician, David Hilbert ( ), asked whether there could be a mechanical way (i.e. by means of a fully specifiable set of instructions) of determining whether some statement in a formal system like arithmetic was provable or not. In 1936 Turing published a paper the aim of which was to show that there was no such method. “On computable numbers, with an application to the Entscheidungs problem.” Proceedings of the London Mathematical Society, 2(42):

27 The Turing Machine In order to argue for this claim, he needed a clear concept of “mechanical procedure.” His idea — which came to be called the Turing machine — was this: (1) A tape of infinite length (2) Finitely many squares of the tape have a single symbol from a finite language. (3) Someone (or something) that can read the squares and write in them. (4) At any time, the machine is in one of a finite number of internal states. (5) The machine has instructions that determine what it does given its internal state and the symbol it encounters on the tape. It can  change its internal state;  change the symbol on the square;  move forward;  move backward;  halt (i.e. stop).

Current state = 1 If current state = 1 and current symbol = 0 then new state = 10 new symbol = 1 move right 0

Current state = 10 If current state = 1 and current symbol = 0 then new state = 10 new symbol = 1 move right

Current state = 10 If current state = 1 and current symbol = 0 then new state = 10 new symbol = 1 move right

31 Functions It is essential to the idea of a Turing machine that it is not a physical machine, but an abstract one — a set of procedures. It makes no difference whether the machine is embodied by a person in a boxcar on a track, or a person with a paper and pencil, or a smart and well-trained flamingo.

32 Turing’s Theorem In the 1936 paper Turing proved that there are “general- purpose” Turing machines that can compute whatever any other Turing machine. This is done by coding the function of the special-purpose machine as instructions of the other machine — that is by “programming” it. This is called Turing’s theorem. These are universal Turing machines, and the idea of a coding for a particular function fed into a universal Turing machine is basically our conception of a computer and a stored program. The concept of the universal Turing machine is just the concept of the computer as we know it.

33 First computers: custom computing machines Eniac: the control is hardwired manually for each problem. Control Input tape (read only) Output tape (write only) Work tape (memory) 1940: VON NEUMANN: DISTINCTION BETWEEN DATA AND INSTRUCTIONS

34 Can Machines Think? In “Computing machinery and intelligence,” written in 1950, Turing asks whether machines can think. He claims that this question is too vague, and proposes, instead, to replace it with a different one. That question is: Can machines pass the “imitation game” (now called the Turing test)? If they can, they are intelligent. Turing is thus the first to have offered a rigorous test for the determination of intelligence quite generally.

35 The Turing Test The game runs as follows. You sit at a computer terminal and have an electronic conversation. You don’t know who is on the other end; it could be a person or a computer responding as it has been programmed to do. If you can’t distinguish between a human being and a computer from your interactions, then the computer is intelligent. Note that this is meant to be a sufficient condition of intelligence only. There may be other ways to be intelligent.

36 Artificial Intelligence

37 The Church-Turning Thesis Turing, and a logician called Alonzo Church ( ), independently developed the idea (not yet proven by widely accepted) that whatever can be computed by a mechanical procedure can be computed by a Turing machine. This is known as the Church-Turing thesis.

38 AI: The Argument We’ve now got the materials to show that AI is possible: P1: Any function that can be computed by a mechanical procedure can be computed by a Turing machine. (Church-Turing thesis) P2: Thinking is nothing more than the computing of functions by mechanical procedures (i.e., thinking is symbol manipulation). (Functionalist-Computationalist thesis) C1: Therefore, thinking can be performed by a Turing machine. P3: Turing machines are multiply realizable. In particular, they can be realized by computers, robots, etc.  It is possible to build a computer, robot, etc. that can think. That is, AI is possible.

39 Turing Machines

40 The Language Hierarchy Regular Languages Context-Free Languages ? ?

41 Regular Languages Context-Free Languages Languages accepted by Turing Machines

42 A Turing Machine Tape Read-Write head Control Unit

43 The Tape Read-Write head No boundaries -- infinite length The head moves Left or Right

Read-Write head The head at each time step: 1. Reads a symbol 2. Writes a symbol 3. Moves Left or Right

Example: Time Time 1 1. Reads 2. Writes 3. Moves Left

Time Time 2 1. Reads 2. Writes 3. Moves Right

47 The Input String Blank symbol head Head starts at the leftmost position of the input string Input string

Blank symbol head Input string Remark: the input string is never empty

49 States & Transitions Read Write Move Left Move Right

50 Example: Time 1 current state

Time Time 2

Time Time 2 Example:

Time Time 2 Example:

54 Determinism Allowed Not Allowed No lambda transitions allowed Turing Machines are deterministic

55 Partial Transition Function Example: No transition for input symbol Allowed:

56 Halting The machine halts if there are no possible transitions to follow

57 Example: No possible transition HALT!!!

58 Final States Allowed Not Allowed Final states have no outgoing transitions In a final state the machine halts

59 Acceptance Accept Input If machine halts in a final state Reject Input If machine halts in a non-final state or If machine enters an infinite loop

60 Turing Machine Example A Turing machine that accepts the language:

61 Time 0

62 Time 1

63 Time 2

64 Time 3

65 Time 4 Halt & Accept

66 Rejection Example Time 0

67 Time 1 No possible Transition Halt & Reject

68 Infinite Loop Example A Turing machine for language

69 Time 0

70 Time 1

71 Time 2

72 Time 2 Time 3 Time 4 Time 5 Infinite loop

73 Because of the infinite loop: The final state cannot be reached The machine never halts The input is not accepted

74 Another Turing Machine Example Turing machine for the language

75 Time 0

76 Time 1

77 Time 2

78 Time 3

79 Time 4

80 Time 5

81 Time 6

82 Time 7

83 Time 8

84 Time 9

85 Time 10

86 Time 11

87 Time 12

88 Halt & Accept Time 13

89 If we modify the machine for the language we can easily construct a machine for the language Observation:

90 Formal Definitions for Turing Machines

91 Transition Function

92 Transition Function

93 Turing Machine: States Input alphabet Tape alphabet Transition function Initial state blank Final states

94 Configuration Instantaneous description:

95 Time 4Time 5 A Move:

96 Time 4Time 5 Time 6Time 7

97 Equivalent notation:

98 Initial configuration: Input string

99 The Accepted Language For any Turing Machine Initial stateFinal state

100 Standard Turing Machine Deterministic Infinite tape in both directions Tape is the input/output file The machine we described is the standard:

101 Computing Functions with Turing Machines

102 A function Domain: Result Region: has:

103 A function may have many parameters: Example: Addition function

104 Integer Domain Unary: Binary: Decimal: We prefer unary representation: easier to manipulate with Turing machines

105 Definition: A function is computable if there is a Turing Machine such that: Initial configurationFinal configuration Domain final stateinitial state For all

106 Initial Configuration Final Configuration A function is computable if there is a Turing Machine such that: In other words: Domain For all

107 Example The function is computable Turing Machine: Input string: unary Output string: unary are integers

108 Start initial state The 0 is the delimiter that separates the two numbers

109 Start Finish final state initial state

110 Finish final state The 0 helps when we use the result for other operations

111 Turing machine for function

112 Execution Example: Time 0 Final Result (2)

113 Time 0

114 Time 1

115 Time 2

116 Time 3

117 Time 4

118 Time 5

119 Time 6

120 Time 7

121 Time 8

122 Time 9

123 Time 10

124 Time 11

125 HALT & accept Time 12

126 Another Example The function is computable Turing Machine: Input string: unary Output string:unary is integer

127 Start Finish final state initial state

128 Turing Machine Pseudocode for Replace every 1 with $ Repeat: Find rightmost $, replace it with 1 Go to right end, insert 1 Until no more $ remain

129 Turing Machine for

130 Example Start Finish

131 Another Example The function is computable if

132 Turing Machine for Input: Output: or if

133 Turing Machine Pseudocode: Match a 1 from with a 1 from Repeat Until all of or is matched If a 1 from is not matched erase tape, write 1 else erase tape, write 0

134 Combining Turing Machines

135 Block Diagram Turing Machine inputoutput

136 Example: if Comparer Adder Eraser