Formal Languages, Automata and Models of Computation

Slides:



Advertisements
Similar presentations
CS 345: Chapter 9 Algorithmic Universality and Its Robustness
Advertisements

CSCI 4325 / 6339 Theory of Computation Zhixiang Chen Department of Computer Science University of Texas-Pan American.
Programming Languages Wrap-up. Your Toolkit Object-oriented Imperative Functional Logic.
1 It pays to be Persistent Dina Goldin, U.Mass/Boston (U.Conn) Joint work with Scott Smolka, SUNY at Stony Brook Peter Wegner, Brown University.
1 Undecidability Andreas Klappenecker [based on slides by Prof. Welch]
Sep. 29, 2004Tufts University1 Interaction: Conjectures, Results, and Myths Dina Goldin University of Connecticut.
1 Introduction to Computability Theory Lecture12: Decidable Languages Prof. Amos Israeli.
1 Introduction to Computability Theory Lecture12: Reductions Prof. Amos Israeli.
1 Introduction to Computability Theory Lecture11: Variants of Turing Machines Prof. Amos Israeli.
Introduction to Computability Theory
CPSC 411, Fall 2008: Set 12 1 CPSC 411 Design and Analysis of Algorithms Set 12: Undecidability Prof. Jennifer Welch Fall 2008.
Fall 2003Costas Busch - RPI1 Decidability. Fall 2003Costas Busch - RPI2 Recall: A language is decidable (recursive), if there is a Turing machine (decider)
1 Undecidability Andreas Klappenecker [based on slides by Prof. Welch]
1 The Chomsky Hierarchy. 2 Unrestricted Grammars: Rules have form String of variables and terminals String of variables and terminals.
Fall 2004COMP 3351 The Chomsky Hierarchy. Fall 2004COMP 3352 Non-recursively enumerable Recursively-enumerable Recursive Context-sensitive Context-free.
Turing Machines, Transition Systems, and Interaction
CHAPTER 3 The Church-Turing Thesis
CHAPTER 4 Decidability Contents Decidable Languages
Fall 2004COMP 3351 Reducibility. Fall 2004COMP 3352 Problem is reduced to problem If we can solve problem then we can solve problem.
Linear Bounded Automata LBAs
1 Decidability continued. 2 Undecidable Problems Halting Problem: Does machine halt on input ? State-entry Problem: Does machine enter state halt on input.
1 Foundations of Software Design Fall 2002 Marti Hearst Lecture 29: Computability, Turing Machines, Can Computers Think?
Courtesy Costas Busch - RPI1 Reducibility. Courtesy Costas Busch - RPI2 Problem is reduced to problem If we can solve problem then we can solve problem.
EXPRESS'011 Turing Machines, Transition Systems, and Interaction Dina Goldin, U.Connecticut Scott Smolka, SUNY at Stony Brook Peter Wegner, Brown University.
Turing Machines CS 105: Introduction to Computer Science.
Fall 2006Costas Busch - RPI1 Undecidable Problems (unsolvable problems)
Fall 2003Costas Busch - RPI1 Turing Machines (TMs) Linear Bounded Automata (LBAs)
Prof. Busch - LSU1 Undecidable Problems (unsolvable problems)
Prof. Busch - LSU1 Reductions. Prof. Busch - LSU2 Problem is reduced to problem If we can solve problem then we can solve problem.
1 Reducibility. 2 Problem is reduced to problem If we can solve problem then we can solve problem.
Theory of Computation. Computation Computation is a general term for any type of information processing that can be represented as an algorithm precisely.
1 CD5560 FABER Formal Languages, Automata and Models of Computation Lecture 13 Mälardalen University 2010.
1 CDT314 FABER Formal Languages, Automata and Models of Computation Lecture 12 Mälardalen University 2011.
Review Byron Gao. Overview Theory of computation: central areas: Automata, Computability, Complexity Computability: Is the problem solvable? –solvable.
1 CO Games Development 2 Week 21 Turing Machines & Computability Gareth Bellaby.
Introduction to CS Theory Lecture 15 –Turing Machines Piotr Faliszewski
1 1 CDT314 FABER Formal Languages, Automata and Models of Computation Lecture 15-1 Mälardalen University 2012.
1 CD5560 FABER Formal Languages, Automata and Models of Computation Lecture 13 Mälardalen University 2010.
1 The Halting Problem and Decidability How powerful is a TM? Any program in a high level language can be simulated by a TM. Any algorithmic procedure carried.
CSE 311 Foundations of Computing I Lecture 26 Computability: Turing machines, Undecidability of the Halting Problem Spring
1Computer Sciences Department. Book: INTRODUCTION TO THE THEORY OF COMPUTATION, SECOND EDITION, by: MICHAEL SIPSER Reference 3Computer Sciences Department.
1 Linear Bounded Automata LBAs. 2 Linear Bounded Automata (LBAs) are the same as Turing Machines with one difference: The input string tape space is the.
1 Turing’s Thesis. 2 Turing’s thesis: Any computation carried out by mechanical means can be performed by a Turing Machine (1930)
 2005 SDU Lecture13 Reducibility — A methodology for proving un- decidability.
1Computer Sciences Department. Book: INTRODUCTION TO THE THEORY OF COMPUTATION, SECOND EDITION, by: MICHAEL SIPSER Reference 3Computer Sciences Department.
1 CD5560 FABER Formal Languages, Automata and Models of Computation Lecture 12 Mälardalen University 2005.
1 1 CDT314 FABER Formal Languages, Automata and Models of Computation Lecture 13 School of Innovation, Design and Engineering Mälardalen University 2011.
Fundamentals of Informatics Lecture 12 The Halting Problem Bas Luttik.
1 IDT Open Seminar ALAN TURING AND HIS LEGACY 100 Years Turing celebration Gordana Dodig Crnkovic, Computer Science and Network Department Mälardalen University.
1 Linear Bounded Automata LBAs. 2 Linear Bounded Automata (LBAs) are the same as Turing Machines with one difference: The input string tape space is the.
Overview of the theory of computation Episode 3 0 Turing machines The traditional concepts of computability, decidability and recursive enumerability.
Automata & Formal Languages, Feodor F. Dragan, Kent State University 1 CHAPTER 3 The Church-Turing Thesis Contents Turing Machines definitions, examples,
1 Undecidability Andreas Klappenecker [based on slides by Prof. Welch]
1 Introduction to Turing Machines
1 CD5560 FABER Formal Languages, Automata and Models of Computation Lecture 12 Mälardalen University 2007.
1 The Chomsky Hierarchy. 2 Unrestricted Grammars: Productions String of variables and terminals String of variables and terminals.
Chapter 12 Theory of Computation Introduction to CS 1 st Semester, 2014 Sanghyun Park.
Donghyun (David) Kim Department of Mathematics and Computer Science North Carolina Central University 1 Chapter 5 Reducibility Some slides are in courtesy.
The Church-Turing Thesis Chapter Are We Done? FSM  PDA  Turing machine Is this the end of the line? There are still problems we cannot solve:
Recursively Enumerable and Recursive Languages. Definition: A language is recursively enumerable if some Turing machine accepts it.
Decidability.
Formal Languages, Automata and Models of Computation
Linear Bounded Automata LBAs
Chapter 9 TURING MACHINES.
Busch Complexity Lectures: Undecidable Problems (unsolvable problems)
Jaya Krishna, M.Tech, Assistant Professor
CSCE 411 Design and Analysis of Algorithms
Decidable Languages Costas Busch - LSU.
Formal Languages, Automata and Models of Computation
CO Games Development 2 Week 21 Turing Machines & Computability
Presentation transcript:

Formal Languages, Automata and Models of Computation CDT314 FABER Formal Languages, Automata and Models of Computation Lecture 13 Mälardalen University 2012 1

Content Alan Turing and Hilbert Program Universal Turing Machine Chomsky Hierarchy Decidability Reducibility Uncomputable Functions Rice’s Theorem Church-Turing Thesis Computation beyond Turing Model Interactive Computing, Persistent TM’s (Dina Goldin/Peter Wegner)

TURING MACHINES Based on C Busch, RPI, Models of Computation

Turing’s "Machines". These machines are humans who calculate. (Wittgenstein) A man provided with paper, pencil, and rubber, and subject to strict discipline, is in effect a universal machine. (Turing)

Standard Turing Machine Tape ...... ...... Read-Write head Control Unit

The Tape No boundaries -- infinite length ...... ...... Read-Write head The head moves Left or Right

...... ...... Read-Write head The head at each time step: 1. Reads a symbol 2. Writes a symbol 3. Moves Left or Right

Example Time 0 ...... Time 1 ...... 1. Reads 2. Writes 3. Moves Left

The Input String Input string Blank symbol ...... ...... head Head starts at the leftmost position of the input string

States & Transitions Write Read Move Left Move Right

Time 1 ...... ...... Time 2 ...... ......

Determinism Not Allowed Turing Machines are deterministic Allowed No lambda transitions allowed in standard TM!

Formal Definitions for Turing Machines

Transition Function

Turing Machine Input alphabet Tape alphabet States Final Transition function Initial state blank

Universal Turing Machine

A limitation of Turing Machines: Turing Machines are “hardwired” they execute only one program Better are reprogrammable machines.

Solution: Universal Turing Machine Characteristics: Reprogrammable machine Simulates any other Turing Machine

Universal Turing Machine simulates any other Turing Machine Input of Universal Turing Machine Description of transitions of Initial tape contents of

Three tapes Description of Universal Turing Machine Tape Contents of States of

We encode/describe Turing Machine as a string of symbols. Tape 1 Description of We encode/describe Turing Machine as a string of symbols.

Tape Contents Encoding Alphabet Encoding Symbols: Encoding:

State Encoding States: Encoding: Head Move Encoding Move: Encoding:

Transition Encoding Transition: Encoding: separator

Machine Encoding Transitions: Encoding: separator

Tape 1 contents of Universal Turing Machine: encoding of the simulated machine as a binary string of 0’s and 1’s

A Turing Machine is described with a binary string of 0’s and 1’s. Therefore: The set of Turing machines forms a language: Each string of the language is the binary encoding of a Turing Machine.

Language of Turing Machines 00100100101111, 111010011110010101, …… } (Turing Machine 2) (Turing Machine 3) ……

The Chomsky Hierarchy

The Chomsky Language Hierarchy Recursively-enumerable Recursive Context-sensitive Context-free Regular Non-recursively enumerable

Recursively Enumerable Languages Unrestricted Grammars Productions String of variables and terminals

Example of unrestricted grammar

Theorem A language is recursively enumerable if and only if it is generated by an unrestricted grammar.

Context-Sensitive Grammars and Productions String of variables and terminals

The language is context-sensitive:

A language is context sensitive if and only if Theorem A language is context sensitive if and only if it is accepted by a Linear-Bounded automaton.

Linear Bounded Automata (LBAs) are the same as Turing Machines with one difference: The input string tape space is the only tape space allowed to use.

Left-end marker Input string Right-end Working space in tape All computation is done between end markers. Linear Bounded Automaton (LBA)

Observation There is a language which is context-sensitive but not recursive.

Decidability

Consider problems with answer YES or NO. Examples Does Machine have three states ? Is string a binary number? Does DFA accept any input?

A problem is decidable if some Turing machine solves (decides) the problem. Decidable problems: Does Machine have three states ? Is string a binary number? Does DFA accept any input?

The Turing machine that solves a problem answers YES or NO for each instance. Input problem instance YES Turing Machine NO

The machine that decides a problem: If the answer is YES then halts in a yes state If the answer is NO then halts in a no state These states may not be the final states.

Turing Machine that decides a problem YES NO YES and NO states are halting states

Difference between Recursive Languages (“Acceptera”) and Decidable problems (“Avgöra”) For decidable problems: The YES states may not be final states.

Some problems are undecidable: There is no Turing Machine that solves all instances of the problem.

A famous undecidable problem: The halting problem

The Halting Problem Input: Turing Machine String Question: Does halt on ?

Theorem The halting problem is undecidable. Proof Assume to the contrary that the halting problem is decidable.

There exists Turing Machine that solves the halting problem YES halts on NO doesn’t halt on

Construction of Input: initial tape contents YES NO Encoding of String

Construct machine returns YES then loop forever. If returns NO then halt. If

Loop forever YES NO

Construct machine Input: (machine ) If halts on input Then loop forever Else halt

copy

Run machine with input itself If halts on input Then loop forever Else halt

on input : If halts then loops forever. If doesn’t halt then it halts. CONTRADICTION !

The halting problem is undecidable. This means that The halting problem is undecidable. END OF PROOF

Another proof of the same theorem: If the halting problem was decidable then every recursively enumerable language would be recursive. Recursive vs. Recursively Enumerable Languages http://www.cs.colostate.edu/~massey/Teaching/cs301/RestrictedAccess/Slides/301lecture23.pdf

Theorem The halting problem is undecidable. Proof Assume to the contrary that the halting problem is decidable.

There exists Turing Machine that solves the halting problem. YES halts on NO doesn’t halt on

Let be a recursively enumerable language. Let be the Turing Machine that accepts . We will prove that is also recursive: We will describe a Turing machine that accepts and halts on any input.

NO reject halts on ? YES accept Run with input reject Turing Machine that accepts and halts on any input NO reject halts on ? YES accept Halts on final state Run with input reject Halts on non-final state

Therefore is recursive. Since is chosen arbitrarily, we have proven that every recursively enumerable language is also recursive. But there are recursively enumerable languages which are not recursive. Contradiction!

Therefore, the halting problem is undecidable. END OF PROOF

A simple undecidable problem: The Membership Problem

The Membership Problem Input: Turing Machine String Question: Does accept ?

Theorem The membership problem is undecidable. Proof Assume to the contrary that the membership problem is decidable.

There exists a Turing Machine that solves the membership problem accepts YES NO rejects

Let be a recursively enumerable language. Let be the Turing Machine that accepts . We will prove that is also recursive: We will describe a Turing machine that accepts and halts on any input.

Turing Machine that accepts and halts on any input YES accept accepts ? NO reject

Therefore, is recursive. Since is chosen arbitrarily, we have proven that every recursively enumerable language is also recursive. But there are recursively enumerable languages which are not recursive. Contradiction!

Therefore, the membership problem is undecidable. END OF PROOF

Reducibility

Problem is reduced to problem If we can solve problem then we can solve problem .

Problem is reduced to problem If is decidable then is decidable. If is undecidable then is undecidable.

Example the halting problem reduced to the state-entry problem.

The state-entry problem Inputs: Turing Machine State String Question: Does enter state on input ?

Theorem The state-entry problem is undecidable. Proof Reduce the halting problem to the state-entry problem.

Suppose we have an algorithm that solves the state-entry problem. (Turing Machine) that solves the state-entry problem. We will construct an algorithm that solves the halting problem.

Assume we have the state-entry algorithm: YES enters Algorithm for state-entry problem NO doesn’t enter

We want to design the halting algorithm: YES halts on Algorithm for Halting problem NO doesn’t halt on

Modify input machine Add new state From any halting state add transitions to Single halt state halting states

halts if and only if halts on state

Algorithm for halting problem Inputs: machine and string 1. Construct machine with state 2. Run algorithm for state-entry problem with inputs: ,

Halting problem algorithm YES YES Generate State-entry algorithm NO NO

We reduced the halting problem to the state-entry problem. Since the halting problem is undecidable, it must be that the state-entry problem is also undecidable. END OF PROOF

Another example The halting problem reduced to the blank-tape halting problem.

The blank-tape halting problem Input: Turing Machine Question: Does halt when started with a blank tape?

Theorem The blank-tape halting problem is undecidable. Proof Reduce the halting problem to the blank-tape halting problem.

Suppose we have an algorithm for the blank-tape halting problem. We will construct an algorithm for the halting problem.

blank-tape halting algorithm Assume we have the blank-tape halting algorithm halts on blank tape YES Algorithm for blank-tape halting problem NO doesn’t halt on blank tape

We want to design the halting algorithm: YES halts on Algorithm for halting problem NO doesn’t halt on

Construct a new machine On blank tape writes Then continues execution like step 1 step2 if blank tape execute then write with input

halts on input string if and only if halts when started with blank tape.

Algorithm for halting problem Inputs: machine and string 1. Construct 2. Run algorithm for blank-tape halting problem with input

Halting problem algorithm YES Blank-tape halting algorithm YES Generate NO NO

We reduced the halting problem to the blank-tape halting problem. Since the halting problem is undecidable, the blank-tape halting problem is also undecidable. END OF PROOF

Summary of Undecidable Problems Halting Problem Does machine halt on input ? Membership problem Does machine accept string ? Is a string member of a recursively enumerable language ?) (In other words:

Blank-tape halting problem Does machine halt when starting on blank tape? State-entry Problem: Does machine enter state on input ?

Uncomputable Functions

Uncomputable Functions Domain Range A function is uncomputable if it cannot be computed for all of its domain.

Example An uncomputable function: maximum number of moves until any Turing machine with states halts when started with the blank tape.

Theorem Function is uncomputable. Proof Assume to the contrary that is computable. Then the blank-tape halting problem is decidable.

Algorithm for blank-tape halting problem Input: machine 1. Count states of : 2. Compute 3. Simulate for steps starting with empty tape If halts then return YES otherwise return NO

Therefore, the blank-tape halting problem must be decidable. However, we know that the blank-tape halting problem is undecidable. Contradiction!

Therefore, function is uncomputable. END OF PROOF

Rice’s Theorem

Definition Non-trivial property of recursively enumerable languages: Any property possessed by some (not all) recursively enumerable languages.

Some non-trivial properties of recursively enumerable languages: is empty is finite contains two different strings of the same length

Rice’s Theorem Any non-trivial property of a recursively enumerable language is undecidable.

Rice’s Theorem If  is a set of Turing-acceptable languages that contain some but not all such languages, no TM can decide for an arbitrary Turing-acceptable language L if L belongs to  or not.

Exempel Givet en Turingmaskin M, kan man avgöra om alla strängar som accepteras av M börjar och slutar på samma tecken?

Oavgörbart Problemet handlar om en icke-trivial språkegenskap. Det finns TM:er vars accepterade strängar har egenskapen i fråga, och det finns TM:er vars accepterade strängar inte har egenskapen.

Formellt:  = { L | TM accepterbara språk vars strängar börjar och slutar på samma tecken. }

Now we will prove some non-trivial properties without using Rice’s theorem.

Theorem For any recursively enumerable language it is undecidable whether it is empty. Proof We will reduce the membership problem to the problem of deciding whether is empty.

Membership problem: Does machine accept string ?

Let be the machine that accepts Assume we have the empty language algorithm: YES empty Algorithm for empty language problem NO not empty

We will design the membership algorithm: YES accepts Algorithm for membership problem NO rejects

First construct machine : When enters a final state, compare original input string with . Accept if original input is the same as .

if and only if is not empty

Algorithm for membership problem Inputs: machine and string 1. Construct 2. Determine if is empty YES: then NO: then

Membership algorithm NO YES Check if construct YES is empty NO

We reduced the empty language problem to the membership problem. Since the membership problem is undecidable, the empty language problem is also undecidable. END OF PROOF

Decidability …continued…

Theorem For a recursively enumerable language it is undecidable to determine whether is finite. Proof We will reduce the halting problem to the finite language problem.

Let be the machine that accepts Assume we have the finite language algorithm: Algorithm for finite language problem YES NO finite not finite

We will design the halting problem algorithm: Algorithm for Halting problem YES NO halts on doesn’t halt on

First construct machine . Initially, simulates on input . When enters a halt state, accept any input (infinite language). Otherwise accept nothing (finite language).

halts on is not finite. if and only if

Algorithm for halting problem: Inputs: machine and string 1. Construct 2. Determine if is finite YES: then doesn’t halt on NO: then halts on

Machine for halting problem construct Check if is finite YES NO

We reduced the finite language problem to the halting problem. Since the halting problem is undecidable, the finite language problem is also undecidable. END OF PROOF

Theorem For a recursively enumerable language it is undecidable whether contains two different strings of same length. Proof We will reduce the halting problem to the two strings of equal length- problem.

Let be the machine that accepts Assume we have the two-strings algorithm: Algorithm for two-strings problem YES NO contains doesn’t contain two equal length strings

We will design the halting problem algorithm: Algorithm for Halting problem YES NO halts on doesn’t halt on

First construct machine . Initially, simulates on input . When enters a halt state, accept symbols or . (two equal length strings)

halts on if and only if accepts and (two equal length strings)

Algorithm for halting problem Inputs: machine and string 1. Construct 2. Determine if accepts two strings of equal length YES: then halts on NO: then doesn’t halt on

Machine for halting problem YES YES Check if construct NO NO has two equal length strings

We reduced the two strings of equal length - problem to the halting problem. Since the halting problem is undecidable, the two strings of equal length problem is also undecidable. END OF PROOF

Church-Turing Thesis* *Source: Stanford Encyclopaedia of Philosophy

A Turing machine is an abstract representation of a computing device. It is more like a computer program (software) than a computer (hardware).

were first proposed by Alan Turing, in an attempt to give LCMs [Logical Computing Machines: Turing’s expression for Turing machines] were first proposed by Alan Turing, in an attempt to give a mathematically precise definition of "algorithm" or "mechanical procedure".

The Church-Turing thesis concerns an effective or mechanical method in logic and mathematics.

A method, M, is called ‘effective’ or ‘mechanical’ just in case: M is set out in terms of a finite number of exact instructions (each instruction being expressed by means of a finite number of symbols); M will, if carried out without error, always produce the desired result in a finite number of steps; M can (in practice or in principle) be carried out by a human being unaided by any machinery except for paper and pencil; M demands no insight or ingenuity on the part of the human being carrying it out.

Turing’s thesis: LCMs [logical computing machines; TMs] can do anything that could be described as "rule of thumb" or "purely mechanical". (Turing 1948) He adds: This is sufficiently well established that it is now agreed amongst logicians that "calculable by means of an LCM" is the correct accurate rendering of such phrases.

Turing introduced this thesis in the course of arguing that the Entscheidungsproblem, or decision problem, for the predicate calculus - posed by Hilbert (1928) - is unsolvable.

Church’s account of the Entscheidungsproblem By the Entscheidungsproblem of a system of symbolic logic is here understood the problem to find an effective method by which, given any expression Q in the notation of the system, it can be determined whether or not Q is provable in the system.

Church’s thesis: A function of positive integers is effectively calculable only if recursive.

Misunderstandings of the Turing Thesis Turing did not show that his machines can solve any problem that can be solved "by instructions, explicitly stated rules, or procedures" and nor did he prove that a universal Turing machine "can compute any function that any computer, with any architecture, can compute".

Turing proved that his universal machine can compute any function that any Turing machine can compute; and he put forward, and advanced philosophical arguments in support of, the thesis here called Turing’s thesis.

A thesis concerning the extent of effective methods - procedures that a human being unaided by machinery is capable of carrying out - has no implication concerning the extent of the procedures that machines are capable of carrying out, even machines acting in accordance with ‘explicitly stated rules’.

Among a machine’s repertoire of atomic operations there may be those that no human being unaided by machinery can perform.

Turing introduces his machines as an idealised description of a certain human activity, the tedious one of numerical computation, which until the advent of automatic computing machines was the occupation of many thousands of people in commerce, government, and research establishments.

Turing’s "Machines". These machines are humans who calculate Turing’s "Machines". These machines are humans who calculate. (Wittgenstein) A man provided with paper, pencil, and rubber, and subject to strict discipline, is in effect a universal machine. (Turing)

Computation Beyond Turing Model “Traditionally, the dynamics of computing systems, their unfolding behavior in space and time has been a mere means to the end of computing the function which specifies the algorithmic problem which the system is solving. In much of contemporary computing, the situation is reversed: the purpose of the computing system is to exhibit certain behaviour. (…) We need a theory of the dynamics of informatic processes, of interaction, and information flow, as a basis for answering such fundamental questions as: What is computed? What is a process? What are the analogues to Turing completeness and universality when we are concerned with processes and their behaviours, rather than the functions which they compute?” (Abramsky, 2008) Abramsky, S. (2008) Information, processes and games. In Philosophy of Information; van Benthem, J., Adriaans, P., Eds.; North Holland: Amsterdam, The Netherlands; pp. 483–549.

Computation Beyond Turing Model According to Abramsky, there is the need for second generation models of computation, and in particular there is the need for process models such as Petri nets, Process Algebra, and similar. The first generation models of computation were originating from problems of formalization of mathematics and logic, while processes or agents, interaction, and information flow are genuine product of the modern development of computing. In the second generation models of computation, previous isolated systems with limited interactions with the environment are replaced by processes or agents for which the interactions with each other and with the environment are fundamental.

Computation Beyond Turing Model As a result of interactions among agents and with the environment, complex behavior emerges. The basic building block of this interactive approach is the agent, and the fundamental operation is interaction. The ideal features sought for are computational qualities of computational agents such as biological organisms such as resilience and self-* capabilities. This approach works at both macro-scale (such as processes in operating systems, software agents on the Internet, transactions, etc.) and on micro-scale (program implementation down to hardware).

Interaction: Conjectures, Results, and Myths Additional reading  Interaction: Conjectures, Results, and Myths Dina Goldin Univ. of Connecticut, Brown University http://cs.brown.edu/people/dqg/

Fundamental Questions Underlying Theory of Computation What is computation? How do we model it?

SHARED WISDOM (from our undergraduate Theory of Computation courses) computation: finite transformation of input to output input: finite size (e.g. string or number) closed system: all input available at start, all output generated at end behavior: functions, transformation of input data to output data Church-Turing thesis: Turing Machines capture this (algorithmic) notion of computation Mathematical worldview: All computable problems are function-based.

THE MATHEMATICAL WORLDVIEW “The theory of computability and non-computability [is] usually referred to as the theory of recursive functions... the notion of TM has been made central in the development." Martin Davis, Computability & Unsolvability, 1958 “Of all undergraduate CS subjects, theoretical computer science has changed the least over the decades.” SIGACT News, March 2004 “A TM can do anything that a computer can do.” Michael Sipser, Introduction to the Theory of Computation, 1997

The Operating System Conundrum Real programs, such as operating systems and word processors, often receive an unbounded amount of input over time, and never "finish" their task. Turing machines do not model such ongoing computation well… [TM entry, Wikipedia] If a computation does not terminate, it’s “useless” – but aren’t OS’s useful??

Rethinking Shared Wisdom: (what do computers do?) computation: finite transformation of input to output input: finite-size (string or number) closed system: all input available at start, all output generated at end behavior: functions, algorithmic transformation of input data to output data Church-Turing thesis: Turing Machines capture this (algorithmic) notion of computation computation: ongoing process which performs a task or delivers a service dynamically generated stream of input tokens (requests, percepts, messages) open system: later inputs depend on earlier outputs and vice versa (I/O entanglement, history dependence) behavior: processes, components, control devices, reactive systems, intelligent agents Wegner’s conjecture: Interaction is more powerful than algorithms

Example: Driving home from work Algorithmic input: a description of the world (a static “map”) Output: a sequence of pairs of #s (time-series data) - for turning the wheel - for pressing gas/break Similar to classic AI search/planning problems.

Driving home from work (cont.) But… in a real-world environment, the output depends on every grain of sand in the road (chaotic behavior). Can we possibly have a map that’s detailed enough? ? Worse yet… the domain is dynamic. The output depends on weather conditions, and on other drivers and pedestrians. We can’t possibly be expected to predict that in advance! Nevertheless the problem is solvable! Google “autonomous vehicle research”

Driving home from work (cont.) The problem is solvable interactively. Interactive input: stream of video camera images, gathered as we are driving Output: the desired time-series data, generated as we are driving similar to control systems, or online computation A paradigm shift in the conceptualization of computational problem solving.

Rethinking the mathematical worldview OUTLINE Rethinking the mathematical worldview Persistent Turing Machines (PTMs) PTM expressiveness Sequential Interaction Sequential Interaction Thesis The Myth of the Church-Turing Thesis the origins of the myth Conclusions and future work

Sequential Interaction Sequential interactive computation: System continuously interacts with its environment by alternately accepting an input string and computing a corresponding output string. Examples: method invocations of an object instance in an OO language a C function with static variables queries/updates to single-user databases recurrent neural networks control systems; online computation; transducers; dynamic algorithms; embedded systems, etc.

Sequential Interaction Thesis Whenever there is an effective method for performing sequential interactive computation, this computation can be performed by a Persistent Turing Machine. Universal PTM: simulates any other PTM Need additional input describing the PTM (only once) Example: simulating Answering Machine (simulate AM, will-do), (record hello, ok), (erase, done), (record John, ok), (record Hopkins, ok), (playback, John Hopkins), … Simulation of other sequential interactive systems is analogous.

CHURCH-TURING THESIS REVISITED Whenever there is an effective method for obtaining the values of a mathematical function, the function can be computed by a Turing Machine. Common Reinterpretation (Strong Church-Turing Thesis) A TM can do (compute) anything that a computer can do. The equivalence of the two is a myth! This myth has been dogmatically accepted by the CS community. The function-based behavior of algorithms does not capture all forms of computation. Turing himself would have denied it. In the same paper where he introduced what we now call Turing Machines, he also introduced choice machines, as a distinct model of computation. Choice machines extend Turing Machines to interaction by allowing a human operator to make choices during the computation.

ORIGINS OF THE CHURCH-TURING THESIS MYTH A TM can do anything that a computer can do. Based on several claims: A problem is solvable if there exists a Turing Machine for computing it. A problem is solvable if it can be specified by an algorithm. Algorithms are what computers do. Each claim is correct in isolation, provided we understand the underlying assumptions. Together, they induce an incorrect conclusion TMs = solvable problems = algorithms = computation

Deconstructing the Turing Thesis Myth (1) TMs = solvable problems Assumes: All computable problems are function-based. Reasons: Theory of Computation started as a field of mathematics; mathematical principles were adopted for the fundamental notions of computation, identifying computability with the computation of functions, as well as with Turing Machines. The batch-based operation of original computers did suggest other conceptualizations of computation.

Deconstructing the Turing Thesis Myth (2) solvable problems = algorithms Assumes: Algorithmic computation is also function based; i.e., the computational role of an algorithm is to transform input data to output data. Reasons: Original (mathematical) meaning of “algorithms” E.g. Euclid’s greatest common divisor algorithm Original (Knuthian) meaning of “algorithms” “An algorithm has zero or more inputs, i.e., quantities which are given to it initially before the algorithm begins.“ [Knuth’68]

Deconstructing the Turing Thesis Myth (3) algorithms = computation Reasons: The ACM Curriculum (1968): Adopted algorithms as the central concept of CS without explicit agreement on the meaning of this term.

Deconstructing the Turing Thesis Myth (3) algorithms = computation Reasons: Textbooks: When defining algorithms, the assumption of their closed function-based nature was often left implicit, if not forgotten “An algorithm is a recipe, a set of instructions or the specifications of a process for doing something. That something is usually solving a problem of some sort.” [Rice&Rice’69] “An algorithm is a collection of simple instructions for carrying out some task. Commonplace in everyday life, algorithms sometimes are called procedures or recipes.” [Sipser’97]

Outline Rethinking the mathematical worldview Persistent Turing Machines (PTMs) PTM expressiveness Sequential Interaction The Myth of the Church-Turing Thesis Conclusions and future work

The Shift to Interaction in CS Algorithmic Interactive Computation = transforming input to output Computation = carrying out a task over time Logic and search in AI Intelligent agents, partially observable environments, learning Procedure-oriented programming Object-oriented programming Closed systems Open systems Compositional behavior Emergent behavior Rule-based reasoning Simulation, control, semi-Markov processes

The Interactive Turing Test From answering questions to holding discussions. Learning from -- and adapting to -- the questioner. “Book intelligence” vs. “street smarts”. “It is hard to draw the line at what is intelligence and what is environmental interaction. In a sense, it does not really matter which is which, as all intelligent systems must be situated in some world or other if they are to be useful entities.” [Brooks]

Modeling Interactive Computation: PTMs in Perspective Many other interactive models Reactive and embedded systems Dataflow, I/O automata [Lynch], synchronous languages, finite/pushdown automata over infinite words Interaction games [Abramsky], online algorithms [Albers] TM extensions: on-line Turing machines [Fischer], interactive Turing machines [Goldreich]... Concurrency Theory Focuses on communication (between concurrent agents/processes) rather than computation [Milner] Orthogonal to the theory of computation and TMs. What makes PTMs unique? Provably more expressive than TMs. Bridging the gap between concurrency theory (labeled transition systems) and traditional TOC.

Future Work: 3 conjectures Theory of Sequential Interaction conjecture: notions analogous to computational complexity, logic, and recursive functions can be developed for sequential interaction computation Multi-stream interaction From hidden variables to hidden interfaces conjecture: multi-stream interaction is more powerful than sequential interaction [Wegner’97] Formalizing indirect interaction Interaction via persistent, observable changes to the common environment In contrast to direct interaction (via message passing) conjecture: direct interaction does not capture all forms of multi-agent behaviors (GDC: except in case when environment is represented as an agent in an multi-agent model)

References http://www.cse.uconn.edu/~dqg/papers/ [Wegner’97] Peter Wegner Why Interaction is more Powerful than Algorithms Communications of the ACM, May 1997 [EGW’04] Eugene Eberbach, Dina Goldin, Peter Wegner Turing's Ideas and Models of Computation book chapter, in Alan Turing: Life and Legacy of a Great Thinker, Springer 2004 [I&C’04] Dina Goldin, Scott Smolka, Paul Attie, Elaine Sonderegger Turing Machines, Transition Systems, and Interaction Information & Computation Journal, 2004 [GW’04] Dina Goldin, Peter Wegner The Church-Turing Thesis: Breaking the Myth presented at CiE 2005, Amsterdam, June 2005 to be published in LNCS