Presentation is loading. Please wait.

Presentation is loading. Please wait.

Formal Languages, Automata and Models of Computation

Similar presentations


Presentation on theme: "Formal Languages, Automata and Models of Computation"— Presentation transcript:

1 Formal Languages, Automata and Models of Computation
CD5560 FABER Formal Languages, Automata and Models of Computation Lecture 13 Mälardalen University 2007

2 Content Introduction Universal Turing Machine Chomsky Hierarchy Decidability Reducibility Uncomputable Functions Rice’s Theorem Interactive Computing & Persistent TM’s (Dina Goldin)

3 http://www.turing.org.uk/turing/ Who was Alan Turing?
Founder of computer science, mathematician, philosopher, codebreaker, visionary man before his time. Jack Copeland and Diane Proudfoot The Alan Turing Home Page Andrew Hodges

4 Alan Turing 1912 (23 June): Birth, London : Sherborne School 1930: Death of friend Christopher Morcom : Undergraduate at King's College, Cambridge University : Quantum mechanics, probability, logic 1935: Elected fellow of King's College, Cambridge 1936: The Turing machine, computability, universal machine : Princeton University. Ph.D. Logic, algebra, number theory : Return to Cambridge. Introduced to German Enigma cipher machine : The Bombe, machine for Enigma decryption : Breaking of U-boat Enigma, saving battle of the Atlantic

5 Alan Turing : Chief Anglo-American crypto consultant. Electronic work. 1945: National Physical Laboratory, London 1946: Computer and software design leading the world : Programming, neural nets, and artificial intelligence 1948: Manchester University 1949: First serious mathematical use of a computer 1950: The Turing Test for machine intelligence 1951: Elected FRS. Non-linear theory of biological growth 1952: Arrested as a homosexual, loss of security clearance : Unfinished work in biology and physics 1954 (7 June): Death (suicide) by cyanide poisoning, Wilmslow, Cheshire.

6 Hilbert’s Program, 1900 Hilbert’s hope was that mathematics would be reducible to finding proofs (manipulating the strings of symbols) from a fixed system of axioms, axioms that everyone could agree were true. Can all of mathematics be made algorithmic, or will there always be new problems that outstrip any given algorithm, and so require creative acts of mind to solve?

7 TURING MACHINES

8 Turing’s "Machines". These machines are humans who calculate
Turing’s "Machines". These machines are humans who calculate. (Wittgenstein) A man provided with paper, pencil, and rubber, and subject to strict discipline, is in effect a universal machine. (Turing)

9 Standard Turing Machine
Tape ...... ...... Read-Write head Control Unit

10 The Tape No boundaries -- infinite length ...... ......
Read-Write head The head moves Left or Right

11 ...... ...... Read-Write head The head at each time step: 1. Reads a symbol 2. Writes a symbol 3. Moves Left or Right

12 Example Time 0 ...... Time 1 ...... 1. Reads 2. Writes 3. Moves Left

13 The Input String Input string Blank symbol ...... ...... head
Head starts at the leftmost position of the input string

14 States & Transitions Write Read Move Left Move Right

15 Time 1 ...... ...... Time 2 ...... ......

16 Determinism Not Allowed Turing Machines are deterministic Allowed
No lambda transitions allowed in standard TM!

17 Formal Definitions for Turing Machines

18 Transition Function

19 Turing Machine Input alphabet Tape alphabet States Final Transition
function Initial state blank

20 Universal Turing Machine

21 A limitation of Turing Machines:
Turing Machines are “hardwired” they execute only one program Better are reprogrammable machines.

22 Solution: Universal Turing Machine Characteristics: Reprogrammable machine Simulates any other Turing Machine

23 Universal Turing Machine
simulates any other Turing Machine Input of Universal Turing Machine Description of transitions of Initial tape contents of

24 Three tapes Description of Universal Turing Machine Tape Contents of
State of

25 We describe Turing machine as a string of symbols:
Tape 1 Description of We describe Turing machine as a string of symbols: We encode as a string of symbols

26 Alphabet Encoding Symbols: Encoding:

27 State Encoding States: Encoding: Head Move Encoding Move: Encoding:

28 Transition Encoding Transition: Encoding: separator

29 Machine Encoding Transitions: Encoding: separator

30 Tape 1 contents of Universal Turing Machine:
encoding of the simulated machine M as a binary string of 0’s and 1’s

31 A Turing Machine is described
with a binary string of 0’s and 1’s. Therefore: The set of Turing machines forms a language: Each string of the language is the binary encoding of a Turing Machine.

32 Language of Turing Machines
, , …… } (Turing Machine 2) ……

33 The Chomsky Hierarchy

34 The Chomsky Language Hierarchy
Recursively-enumerable Recursive Context-sensitive Context-free Regular Non-recursively enumerable

35 Unrestricted Grammars
Productions String of variables and terminals

36 Example of unrestricted grammar

37 Theorem A language L is recursively enumerable
if and only if it is generated by an unrestricted grammar.

38 Context-Sensitive Grammars
Productions String of variables and terminals String of variables and terminals and

39 The language is context-sensitive:

40 Theorem A language L is context sensitive
if and only if it is accepted by a Linear-Bounded automaton.

41 Linear Bounded Automata (LBAs)
are the same as Turing Machines with one difference: The input string tape space is the only tape space allowed to use.

42 Linear Bounded Automaton (LBA)
Input string Working space in tape Left-end marker Right-end marker All computation is done between end markers.

43 Observation There is a language which is recursive
but not context-sensitive.

44 Decidability

45 Consider problems with answer YES or NO.
Examples Does Machine M have three states ? Is string w a binary number? Does DFA M accept any input?

46 A problem is decidable if some Turing machine
solves (decides) the problem. Decidable problems: Does Machine M have three states ? Is string w a binary number? Does DFA M accept any input?

47 The Turing machine that solves a problem
answers YES or NO for each instance. Input problem instance YES Turing Machine NO

48 The machine that decides a problem:
If the answer is YES then halts in a yes state If the answer is NO then halts in a no state These states may not be the final states.

49 YES NO Turing Machine that decides a problem
YES and NO states are halting states

50 Difference between Recursive Languages accept (“Acceptera”) and Decidable problems decide (“Avgöra”) For decidable problems: The YES states may not be final states.

51 Some problems are undecidable:
There is no Turing Machine that solves all instances of the problem.

52 A famous undecidable problem:
The halting problem

53 The Halting Problem Input: Turing Machine M String w
Question: Does M halt on w?

54 Theorem The halting problem is undecidable. Proof Assume to the contrary that the halting problem is decidable.

55 M halts on w M doesn’t halt on w There exists Turing Machine H
that solves the halting problem YES NO M halts on w M doesn’t halt on w

56 YES NO Construction of H Input: initial tape contents encoding of M
string w

57 Construct machine H´ If H returns YES then loop forever. If H returns NO then halt.

58 NO Loop forever YES

59 Construct machine Input: (machine ) If halts on input Then loop forever Else halt

60 copy

61 Run machine with input itself
If halts on input Then loop forever Else halt

62 on input : If halts then loops forever. If doesn’t halt then it halts. CONTRADICTION !

63 The halting problem is undecidable.
This means that The halting problem is undecidable. END OF PROOF

64 Another proof of the same theorem
If the halting problem was decidable then every recursively enumerable language would be recursive.

65 Theorem Proof The halting problem is undecidable.
Assume to the contrary that the halting problem is decidable.

66 M halts on w M doesn’t halt on w There exists Turing Machine H
that solves the halting problem. YES NO M halts on w M doesn’t halt on w

67 Let L be a recursively enumerable language.
Let M be the Turing Machine that accepts L. We will prove that L is also recursive: We will describe a Turing machine that accepts L and halts on any input.

68 Turing Machine that accepts L and halts on any input
reject w NO M halts on w ? YES accept w Halts on final state Run M with input w reject w Halts on non-final state

69 Contradiction! Therefore L is recursive.
Since L is chosen arbitrarily, we have proven that every recursively enumerable language is also recursive. But there are recursively enumerable languages which are not recursive. Contradiction!

70 Therefore, the halting problem is undecidable.
END OF PROOF

71 A simple undecidable problem:
The Membership Problem

72 The Membership Problem
Input: Turing Machine M String w Question: Does M accept w?

73 Theorem Proof The membership problem is undecidable.
Assume to the contrary that the membership problem is decidable.

74 M accepts w M rejects w There exists Turing Machine H
that solves the membership problem YES NO M accepts w M rejects w

75 Let L be a recursively enumerable language.
Let M be the Turing Machine that accepts L. We will prove that L is also recursive: We will describe a Turing machine that accepts L and halts on any input.

76 YES M accepts w? NO Turing Machine that accepts L
and halts on any input YES accept w M accepts w? NO reject w

77 Contradiction! Therefore L is recursive.
Since L is chosen arbitrarily, we have proven that every recursively enumerable language is also recursive. But there are recursively enumerable languages which are not recursive. Contradiction!

78 Therefore, the membership problem
is undecidable. END OF PROOF

79 Reducibility

80 Problem A is reduced to problem B.
If we can solve problem B then we can solve problem A.

81 Problem A is reduced to problem B.
If B is decidable then A is decidable. If A is undecidable then B is undecidable.

82 Example the halting problem reduced to the state-entry problem.

83 The state-entry problem
Input: Turing Machine M State q String w Question: Does M enter state q on input w?

84 Theorem Proof The state-entry problem is undecidable.
Reduce the halting problem to the state-entry problem.

85 Suppose we have an algorithm
(Turing Machine) that solves the state-entry problem. We will construct an algorithm that solves the halting problem.

86 Assume we have the state-entry algorithm:
YES M enters q Algorithm for state-entry problem NO M doesn’t enter q

87 We want to design the halting algorithm:
YES M halts on q Algorithm for Halting problem M doesn’t halt on q NO

88 From any halting state add transitions to q
Modify input machine M Add new state q From any halting state add transitions to q single halt state halting states

89 halts if and only if halts on state

90 Algorithm for halting problem
Input: machine M and string w 1. Construct machine M’ with state q 2. Run algorithm for state-entry problem with inputs: M’, q, w

91 Halting problem algorithm
YES YES State-entry algorithm Generate NO NO

92 We reduced the halting problem
to the state-entry problem. Since the halting problem is undecidable, it must be that the state-entry problem is also undecidable. END OF PROOF

93 Another example The halting problem reduced to the blank-tape halting problem. Input: Turing Machine M Question: Does M halt when started with a blank tape?

94 Theorem Proof The blank-tape halting problem is undecidable.
Reduce the halting problem to the blank-tape halting problem.

95 Suppose we have an algorithm
for the blank-tape halting problem. We will construct an algorithm for the halting problem.

96 M doesn’t halt on blank tape
Assume we have the blank-tape halting algorithm YES M halts on blank tape Algorithm for blank-tape halting problem NO M doesn’t halt on blank tape

97 YES NO M halts on w M doesn’t We want to design the halting algorithm:
Algorithm for halting problem YES NO M halts on w M doesn’t halt on w

98 Construct a new machine
On blank tape writes Then continues execution like step 1 step2 if blank tape execute then write with input

99 halts on input string if and only if halts when started with blank tape.

100 Algorithm for halting problem
Inputs: machine and string 1. Construct 2. Run algorithm for blank-tape halting problem with input

101 YES NO Halting problem algorithm YES Blank-tape Generate halting

102 We reduced the halting problem
to the blank-tape halting problem. Since the halting problem is undecidable, the blank-tape halting problem is also undecidable. END OF PROOF

103 Summary of Undecidable Problems
Halting Problem Does machine M halt on input w? Membership problem Does machine M accept string w? In other words: Is a string w a member of a recursively enumerable language L?)

104 Blank-tape halting problem
Does machine M halt when starting on blank tape? State-entry Problem: Does machine M enter state q on input w?

105 Uncomputable Functions

106 Uncomputable Functions
Domain Range A function is uncomputable if it cannot be computed for all of its domain.

107 Example An uncomputable function: maximum number of moves until
any Turing machine with n states halts when started with the blank tape.

108 Theorem Proof is computable. Function is uncomputable.
Assume to the contrary that is computable. Then the blank-tape halting problem is decidable.

109 Algorithm for blank-tape halting problem
Input: machine 1. Count states of : 2. Compute 3. Simulate for steps starting with empty tape If halts then return YES otherwise return NO

110 Contradiction! Therefore, the blank-tape halting
problem must be decidable. However, we know that the blank-tape halting problem is undecidable. Contradiction!

111 Therefore, function is uncomputable.
END OF PROOF

112 Rice’s Theorem

113 Definition Non-trivial properties of recursively enumerable languages:
any property possessed by some (not all) recursively enumerable languages.

114 Some non-trivial properties of
recursively enumerable languages: L is empty L is finite L contains two different strings of the same length

115 Rice’s Theorem Any non-trivial property of
a recursively enumerable language is undecidable.

116 We will prove some non-trivial properties
without using Rice’s theorem.

117 Theorem Proof For any recursively enumerable language L
it is undecidable whether it is empty. Proof We will reduce the membership problem to the problem of deciding whether L is empty.

118 Membership problem: Does machine M accept string w?

119 Let M be the machine that accepts L
Assume we have the empty language algorithm: YES L(M) empty Algorithm for empty language problem NO L(M) non-empty

120 YES M accepts w NO M rejects w
We will design the membership algorithm: YES M accepts w Algorithm for membership problem NO M rejects w

121 First construct machine
When M enters a final state, compare original input string with w. Accept if original input is the same as w.

122 if and only if is not empty

123 Algorithm for membership problem
Inputs: machine and string 1. Construct 2. Determine if is empty YES: then NO: then

124 Membership algorithm NO YES Check if construct YES is empty NO

125 We reduced the empty language problem
to the membership problem. Since the membership problem is undecidable, the empty language problem is also undecidable. END OF PROOF

126 Decidability …continued…

127 Theorem For a recursively enumerable language L it is undecidable whether L is finite. Proof We will reduce the halting problem to the finite language problem.

128 YES L(M) finite NO L(M) not finite Let M be the machine that accepts L
Assume we have the finite language algorithm: YES L(M) finite Algorithm for finite language problem NO L(M) not finite

129 YES M halts on w NO M doesn’t
We will design the halting problem algorithm: Algorithm for halting problem YES NO M halts on w M doesn’t halt on w

130 First construct machine .
Initially simulates M on input w. When M enters a halt state, accept any input (infinite language). Otherwise accept nothing (finite language).

131 halts on is not finite. if and only if

132 Algorithm for halting problem:
Inputs: machine and string 1. Construct 2. Determine if is finite YES: then doesn’t halt on NO: then halts on

133 Machine for halting problem
construct Check if is finite YES NO

134 We reduced the finite language problem
to the halting problem. Since the halting problem is undecidable, the finite language problem is also undecidable. END OF PROOF

135 Theorem For a recursively enumerable language L it is undecidable whether L contains two different strings of same length. Proof We will reduce the halting problem to the two strings of equal length- problem.

136 L(M) does not contain two equal length strings NO
Let M be the machine that accepts L: Assume we have the equal-strings algorithm: YES L(M) contains Algorithm for two-strings problem L(M) does not contain two equal length strings NO

137 YES NO M halts on w M doesn’t
We will design the halting problem algorithm: Algorithm for halting problem YES NO M halts on w M doesn’t halt on w

138 First construct machine
Initially simulates M on input w. When M enters a halt state, accept symbols a and b. (two equal length strings)

139 halts on if and only if accepts and (two equal length strings)

140 Algorithm for halting problem
Inputs: machine and string 1. Construct 2. Determine if accepts two strings of equal length YES: then halts on NO: then doesn’t halt on

141 YES NO Machine for halting problem YES Check if construct NO has two
equal length strings

142 We reduced the two strings of equal length -
problem to the halting problem. Since the halting problem is undecidable, the two strings of equal length problem is also undecidable. END OF PROOF

143 Rice's theorem: (Henry Gordon Rice ) If S is a non-trivial property of Turing-acceptable languages, then the problem ‘Does L(M) have the property S? is undecidable. Any nontrivial property of the language recognized by a Turing machine is undecidable. (Only trivial properties of programs are algorithmically decidable. Undecidability is not the exception when it comes to recursively enumerable sets, it is the rule. ) If  is a set of Turing-acceptable languages, containing some but not all such languages, no TM can decide for an arbitrary Turing-acceptable language L if L belongs to  or not.

144 Example Given a Turing machine M, is it possible to decide if all strings accepted by M start and end with the same symbol?

145 Undecidable Problem is about non-trivial property of a language. There exist TM’s with given property, and TM’s without.

146 Formally  = { L | TM acceptable languages With strings that start and end with the same symbol. }

147 Interaction: Conjectures, Results, and Myths
Dina Goldin Univ. of Connecticut, Brown University

148 Fundamental Questions Underlying Theory of Computation
What is computation? How is computation modeled?

149 Shared Wisdom (from our undergraduate Theory of Computation courses)
computation: finite transformation of input to output input: finite size (e.g. string or number) closed system: all input available at start, all output generated at end behavior: functions, transformation of input data to output data Church-Turing thesis: Turing Machines capture this (algorithmic) notion of computation Mathematical worldview: All computable problems are function-based.

150 The Mathematical Worldview
“The theory of computability and non-computability [is] usually referred to as the theory of recursive functions... the notion of TM has been made central in the development." Martin Davis, Computability & Unsolvability, 1958 “Of all undergraduate CS subjects, theoretical computer science has changed the least over the decades.” SIGACT News, March 2004 “A TM can do anything that a computer can do.” Michael Sipser, Introduction to the Theory of Computation, 1997

151 The Operating System Conundrum*
Real programs, such as operating systems and word processors, often receive an unbounded amount of input over time, and never "finish" their task. Turing machines do not model such ongoing computation well… [TM entry, Wikipedia] If a computation does not terminate, it’s “useless” – but aren’t OS’s useful?? * Enigma

152 Rethinking Shared Wisdom: (what do computers do?)
computation: finite transformation of input to output input: finite-size (string or number) closed system: all input available at start, all output generated at end behavior: functions, algorithmic transformation of input data to output data Church-Turing thesis: Turing Machines capture this (algorithmic) notion of computation computation: ongoing process which performs a task or delivers a service dynamically generated stream of input tokens (requests, percepts, messages) open system: later inputs depend on earlier outputs and vice versa (I/O entanglement, history dependence) behavior: processes, components, control devices, reactive systems, intelligent agents Wegner’s conjecture: Interaction is more powerful than algorithms

153 Example: Driving home from work
Algorithmic input: a description of the world (a static “map”) Output: a sequence of pairs of #s (time-series data) - for turning the wheel - for pressing gas/break Similar to classic AI search/planning problems.

154 Driving home from work (cont.)
But… in a real-world environment, the output depends on every grain of sand in the road (chaotic behavior). Can we possibly have a map that’s detailed enough? ? Worse yet… the domain is dynamic. The output depends on weather conditions, and on other drivers and pedestrians. We can’t possibly be expected to predict that in advance! Nevertheless the problem is solvable! Google “autonomous vehicle research”

155 Driving home from work (cont.)
The problem is solvable interactively. Interactive input: stream of video camera images, gathered as we are driving Output: the desired time-series data, generated as we are driving similar to control systems, or online computation A paradigm shift in the conceptualization of computational problem solving.

156 Sequential Interaction
Sequential interactive computation: system continuously interacts with its environment by alternately accepting an input string and computing a corresponding output string. Examples: method invocations of an object instance in an OO language a C function with static variables queries/updates to single-user databases recurrent neural networks - control systems - online computation - transducers - dynamic algorithms - embedded systems

157 Sequential Interaction Thesis
Whenever there is an effective method for performing sequential interactive computation, this computation can be performed by a Persistent Turing Machine Universal PTM: simulates any other PTM Need additional input describing the PTM (only once) Example: simulating Answering Machine (simulate AM, will-do), (record hello, ok), (erase, done), (record John, ok), (record Hopkins, ok), (playback, John Hopkins), … Simulation of other sequential interactive systems is analogous.

158 Church-Turing Thesis Revisited
Whenever there is an effective method for obtaining the values of a mathematical function, the function can be computed by a Turing Machine Common Reinterpretation (Strong Church-Turing Thesis) A TM can do (compute) anything that any computer can do!

159 Church-Turing Thesis Revisited
The equivalence of the two is a myth the function-based behavior of algorithms does not capture all forms of computation this myth has been dogmatically accepted by the CS community Turing himself would have denied it in the same paper where he introduced what we now call Turing Machines, he also introduced choice machines, as a distinct model of computation choice machines extend Turing Machines to interaction by allowing a human operator to make choices during the computation.

160 Origins of the Church-Turing Thesis Myth
A TM can do anything that a computer can do. Based on several claims: A problem is solvable if there exists a Turing Machine for computing it. A problem is solvable if it can be specified by an algorithm. Algorithms are what computers do. Each claim is correct in isolation provided we understand the underlying assumptions Together, they induce an incorrect conclusion TMs = solvable problems = algorithms = computation

161 Deconstructing the Turing Thesis Myth (1)
TMs = solvable problems Assumes: All computable problems are function-based. Reasons: Theory of Computation started as a field of mathematics; mathematical principles were adopted for the fundamental notions of computation, identifying computability with the computation of functions, as well as with Turing Machines. The batch-based modus operandi of original computers did not lend itself to other conceptualizations of computation.

162 Deconstructing the Turing Thesis Myth (2)
solvable problems = algorithms Assumes: Algorithmic computation is also function based; i.e., the computational role of an algorithm is to transform input data to output data. Reasons: Original (mathematical) meaning of “algorithms” E.g. Euclid’s greatest common divisor algorithm Original (Knuthian) meaning of “algorithms” “An algorithm has zero or more inputs, i.e., quantities which are given to it initially before the algorithm begins.“ [Knuth’68]

163 Deconstructing the Turing Thesis Myth (3)
algorithms = computation Reasons: The ACM Curriculum (1968): Adopted algorithms as the central concept of CS without explicit agreement on the meaning of this term. Textbooks: When defining algorithms, the assumption of their closed function-based nature was often left implicit, if not forgotten “An algorithm is a recipe, a set of instructions or the specifications of a process for doing something. That something is usually solving a problem of some sort.” [Rice&Rice’69] “An algorithm is a collection of simple instructions for carrying out some task. Commonplace in everyday life, algorithms sometimes are called procedures or recipes...” [Sipser’97]

164 The Shift to Interaction in CS
Algorithmic Interactive Computation = transforming input to output Computation = carrying out a task over time Logic and search in AI Intelligent agents, partially observable environments, learning Procedure-oriented programming Object-oriented programming Closed systems Open systems Compositional behavior Emergent behavior Rule-based reasoning Simulation, control, semi-Markov processes

165 The Interactive Turing Test
From answering questions to holding discussions. Learning from -- and adapting to -- the questioner. “Book intelligence” vs. “street smarts”. “It is hard to draw the line at what is intelligence and what is environmental interaction. In a sense, it does not really matter which is which, as all intelligent systems must be situated in some world or other if they are to be useful entities.” [Brooks]

166 Modeling Interactive Computation: PTMs in Perspective
Many other interactive models Reactive [MP] and embedded systems Dataflow, I/O automata [Lynch], synchronous languages, finite/pushdown automata over infinite words Interaction games [Abramsky], online algorithms [Albers] TM extensions: on-line Turing machines [Fischer], interactive Turing machines [Goldreich]... Concurrency Theory Focuses on communication (between concurrent agents/processes) rather than computation [Milner] Orthogonal to the theory of computation and TMs. What makes PTMs unique? Provably more expressive than TMs. Bridging the gap between concurrency theory (labeled transition systems) and traditional TOC.

167 Future Work: 3 conjectures
Theory of Sequential Interaction conjecture: notions analogous to computational complexity, logic, and recursive functions can be developed for sequential interaction computation Multi-stream interaction From hidden variables to hidden interfaces conjecture: multi-stream interaction is more powerful than sequential interaction [Wegner’97] Formalizing indirect interaction Interaction via persistent, observable changes to the common environment In contrast to direct interaction (via message passing) conjecture: direct interaction does not capture all forms of multi-agent behaviors

168 References http://www.cse.uconn.edu/~dqg/papers/
[Wegner’97] Peter Wegner Why Interaction is more Powerful than Algorithms Communications of the ACM, May 1997 [EGW’04] Eugene Eberbach, Dina Goldin, Peter Wegner Turing's Ideas and Models of Computation book chapter, in Alan Turing: Life and Legacy of a Great Thinker, Springer 2004 [I&C’04] Dina Goldin, Scott Smolka, Paul Attie, Elaine Sonderegger Turing Machines, Transition Systems, and Interaction Information & Computation Journal, 2004 [GW’04] Dina Goldin, Peter Wegner The Church-Turing Thesis: Breaking the Myth presented at CiE 2005, Amsterdam, June to be published in LNCS


Download ppt "Formal Languages, Automata and Models of Computation"

Similar presentations


Ads by Google