Sep. 29, 2004Tufts University1 Interaction: Conjectures, Results, and Myths Dina Goldin University of Connecticut

Sep. 29, 2004Tufts University2 Some Philosophical Questions (to justify the “Ph” in PhD) Fundamental questions underlying Theory of Computation: What is computation? How do we model it?

Sep. 29, 2004Tufts University3 Shared Wisdom (from our undergraduate Theory of Computation courses) computation: finite transformation of input to output input: finite size (e.g. string or number) closed system: all input available at start, all output generated at end behavior: functions, algorithmic transformation of input data to output data Turing thesis: Turing Machines capture this notion of computation Mathematical worldview: All computable problems are function-based.

Sep. 29, 2004Tufts University4 “The theory of computability and non-computability [is] usually referred to as the theory of recursive functions... the notion of TM has been made central in the development." Martin Davis, Computability & Unsolvability, 1958 “Our basic ideas about what a computer is, what is does, and how it does it, have hardly changed for decades.” Paul Dourish, Where the Action is, 2001 “Of all undergraduate CS subjects, theoretical computer science has changed the least over the decades.” SIGACT News, March 2004 “A TM can do anything that a computer can do.” Michael Sipser, Introduction to the Theory of Computation, 1997 The Mathematical Worldview

Sep. 29, 2004Tufts University5 Rethinking Shared Wisdom: (what do computers do? ) computation: finite transformation of input to output input: finite-size (string or number) closed system: all input available at start, all output generated at end behavior: functions, algorithmic transformation of input data to output data Turing thesis: Turing Machines capture this notion of computation computation: ongoing process which performs a task or delivers a service dynamically generated stream of input tokens (requests, percepts, messages) open system: later inputs depend on earlier outputs and vice versa (I/O entanglement, history dependence) behavior: objects, processes, components, control devices, reactive systems, intelligent agents Wegner’s conjecture: Interaction is more powerful than algorithms

Sep. 29, 2004Tufts University6 Example: Driving home from work Algorithmic input: a description of the world (a static “map”) Output: a sequence of pairs of #s (time-series data) - for turning the wheel - for pressing gas/break Similar to classic AI search/planning problems.

Sep. 29, 2004Tufts University7 But… the output depends on every grain of sand in the road (chaotic behavior). Can we possibly have a map that’s detailed enough? Worse yet… the domain is dynamic. The output depends on weather conditions, and on other drivers and pedestrians. We can’t possibly be expected to predict that in advance! Nevertheless the problem is solvable! Google “autonomous vehicle research” Driving home from work (cont.) ?

Sep. 29, 2004Tufts University8 Driving home from work (cont.) The problem is solvable interactively. Interactive input: stream of video camera images, gathered as we are driving Output: a sequence of pairs of #s (time-series data), generated as we are driving similar to control systems, or online computation

Sep. 29, 2004Tufts University9 Rethinking the mathematical worldview Persistent Turing Machines (PTMs) –interactive extension of the Turing Machine model PTM expressiveness Sequential Interaction Thesis Future work Outline

Sep. 29, 2004Tufts University10 Three-tape Turing Machines (N3TM) s - current state w 1 - contents of input tape w 2 - contents of work tape w 3 - contents of output tape n 1, n 2, n 3 - tape head posns N3TM configurations: input work output S Computation = a sequence of transitions between configurations, from initial to halting.

Sep. 29, 2004Tufts University11 Extending N3TM Computations Dynamic stream semantics - Inputs are streams of dynamically generated tokens (strings). - For each input token, there is an N3TM computation generating the corresponding output token. Persistence (memory) - The contents w of the work tape at the beginning of each N3TM computation is the same as at the end of the previous one. in 1 S0S0   ShSh out 1 w1w1 in 1 in 2 S0S0 w1w1  ShSh out 2 w2w2 in 2...

Sep. 29, 2004Tufts University12 Persistent Turing Machine (PTM) Persistent Stream Language (PSL) of a PTM: set of streams PTM: N3TM with persistent stream-based computational semantics

Sep. 29, 2004Tufts University13 f AM (record X, Y) = (ok, XY) f AM (erase, X) = (done,  ) f AM (playback, X) = (X, X) PSL(AM) contains: (record hello, ok), (erase, done), (record Brown, ok), (record University, ok), (playback, Brown University), … but not (record hello, ok), (erase, done), (playback, Brown University), … PTM as a sequential object PTM Example: Answering Machine (AM)

Sep. 29, 2004Tufts University14 PTM Example: Driving Home from Work

Sep. 29, 2004Tufts University15 At each step, output first bit of previous step. –inputs in 1 ; outputs 1 –inputs in 2 ; outputs 1 st bit of in 1 –inputs in 3 ; outputs 1 st bit of in 2 –... PTM with only 3 states –“state” means contents of worktape PSL(Latch) contains: PTM Example: Latch # 1 0 (1*,1) (0*,1) (1*,0) (1*,1) (0*,0) PTM as a Labeled Transition System

Sep. 29, 2004Tufts University16 Rethinking the mathematical worldview Persistent Turing Machines (PTMs) PTM expressiveness –interactive transition systems –Infinite equivalence hierarchy –Amnesic PTMs Sequential Interaction Thesis Future work Outline

Sep. 29, 2004Tufts University17 Interactive Transition Systems over  S is set of states r is initial state (root) m is transition relation Required to be recursively enumerable # 1 0 (1*,1) (0*,1) (1*,0) (1*,1) (0*,0)

Sep. 29, 2004Tufts University18 From PTMs to ITSs  reach(M), m,  ξ(M)   o M i ws sw,',  m  oi wsws,',, iff Reachable memories of a PTM M: Set of words (work-tape contents) w encountered after zero or more macrosteps. where

Sep. 29, 2004Tufts University19 Infinite sequences of input/output token-pairs emanating from a particular ITS state For an ITS T and state s, ISL(T(s)) [and ISL(T)] are defined similarly to PSL(M(s)) [and PSL(M)] Interactive Stream Equivalence T 1 = ISL T 2 if ISL(T 1 ) = ISL(T 2 )

Sep. 29, 2004Tufts University20 ITS Isomorphism Letbe ITSs, i=1,2

Sep. 29, 2004Tufts University21 ITS Bisimulation Letbe ITSs, i=1,2 is a (strong) interactive bisimulation if: Clause 2. with roles of s and t reversed T 1 = bisim T 2 if  an interactive bisim. between them

Sep. 29, 2004Tufts University22 Bisimulation Example a cb b c a a S1S1 S2S2 S 1  bisim S 2

Sep. 29, 2004Tufts University23 PTMs ITSs = ms = iso = bisim = ISL = PSL Equivalence Relations for PTMs vs. ITSs

Sep. 29, 2004Tufts University24 Rethinking the mathematical worldview Persistent Turing Machines (PTMs) PTM expressiveness –interactive transition systems –Infinite equivalence hierarchy –Amnesic PTMs Sequential Interaction Thesis Future work Outline

Sep. 29, 2004Tufts University25 Infinite Equivalence Hierarchy L k (M) = stream prefix language of PTM M set of prefixes of length  k for streams in PSL(M) represents finite observations of M Corresponding notion of equivalence: M 1 = k M 2 : L k (M 1 ) = L k ( M 2 ) Theorem: for any k, there exist PTMs M, M’ such that M = k M’ but M  k+1 M’ Variation: for any k and any PTM M, there exists PTM M’ such that M = k M’ but M  k+1 M’

Sep. 29, 2004Tufts University26 = 1, = 2, = 3 … are equivalence relations that partition the set of PTMs into progressively finer equivalence classes = PSL is Persistent Stream Equivalence; it is at the top of this infinite equivalence hierarchy = 1 is Turing Machine equivalence Infinite Equivalence Hierarchy (cont) =2=2 =1=1... =∞=∞ = PSL

Sep. 29, 2004Tufts University27 Equivalence Hierarchy Gap Proof: construct PTMs M 1 and M 2 where L  (M 1 ) = L  (M 2 ) but PSL (M 1 ) = PSL (M 2 ) = PSL == =2=2 =1=1...

Sep. 29, 2004Tufts University28 ITS for M 1  (  *, 1) n = 0 n = 1n = 2n = 3 (  *, 1) (  *, 0)... (  *,  ) s div (  *,  )... M1 produces output streams of the form 1*0 + M2 is the same, but also produces the stream 1* M 1, M 2 exhibit unbounded non-determinism Theorem: If a PTM M has unbounded nondeterminism, then M diverges.

Sep. 29, 2004Tufts University29 Rethinking the mathematical worldview Persistent Turing Machines (PTMs) PTM expressiveness –interactive transition systems –Infinite equivalence hierarchy –Amnesic PTMs Sequential Interaction Thesis Future work Outline

Sep. 29, 2004Tufts University30 Amnesic PTMs: “half-way” between TMs and PTMs Example: squaring machine (out i = in i 2 ) [Prasse & Rittgen] in 1 S0S0   ShSh out 1 w1w1 in 1 in 2 S0S0   ShSh out 2 w2w2 in 2... Amnesic PTMs extend TMs with stream-based semantics. -- At least as expressive as TMs Unlike PTMs, they lack persistence.

Sep. 29, 2004Tufts University31 It Pays to be Persistent ASLPSL Two ways to show that PTMs are more expressive than Amnesic PTMs (and, by extension, TMs): 1.Collapse of the equivalence hierarchy. 2.Smaller set of stream languages. = =1=1 = PSL ASL = {PSL(M): M is an amnesic PTM} PSL = {PSL(M): M is a PTM}

Sep. 29, 2004Tufts University32 Summary of Results [I&C’04] PTMs ITSs = == =2=2 =1=1 = ms = iso = bisim = ISL = PSL... = ASL

Sep. 29, 2004Tufts University33 Rethinking the mathematical worldview Persistent Turing Machines (PTMs) PTM expressiveness Sequential Interaction Thesis –the origins of the Turing Thesis myth –interaction as a paradigm shift in CS Future work Outline

Sep. 29, 2004Tufts University34 Sequential Interaction Sequential interactive computation system continuously interacts with its environment by alternately accepting an input string and computing a corresponding output string. Examples of sequential interactive computations: -method invocations of an object instance in an OO language -a C function with static variables -queries and updates to a single-user database -control systems -online computation -recurrent neural networks -transducers -dynamic algorithms -embedded systems

Sep. 29, 2004Tufts University35 Sequential Interaction Thesis Whenever there is an effective method for performing sequential interactive computation, this computation can be performed by a Persistent Turing Machine (PTMs can simulate any interactive computing device) analogue of the Turing Thesis for the computation of TMs: Whenever there is an effective method for obtaining the values of a mathematical function, the function can be computed by a Turing Machine

Sep. 29, 2004Tufts University36 Strong Turing Thesis Turing Thesis: Whenever there is an effective method for obtaining the values of a mathematical function, the function can be computed by a Turing Machine Common Reinterpretation (Strong Turing Thesis) A TM can do (compute) anything that a computer can do Strong Turing Thesis is a myth –dogmatically accepted by the computer science community –the function-based behavior of algorithms does not capture all forms of computation Turing himself would have denied it –in the same paper where he introduced what we now call Turing Machines, he also introduced choice machines, a model of computation distinct from Turing Machines and not reducible to it. –choice machines extend Turing Machines to interaction by allowing a human operator to make choices during the computation.

Sep. 29, 2004Tufts University37 Origins of the Turing Thesis Myth A TM can do anything that a computer can do. Based on several claims: 1.A problem is solvable if there exists a Turing Machine for computing it. 2.A problem is solvable if it can be specified by an algorithm. 3.Algorithms are what computers do. Each claim is correct in isolation (provided we understand the underlying assumptions) Together, they induce an incorrect conclusion

Sep. 29, 2004Tufts University38 Deconstructing the Turing Thesis Myth (1) Claim 1: A problem is solvable if there exists a Turing Machine for computing it. Assumes: All computable problems are function-based. Reasons for this assumption: –Adoption of mathematical principles for the fundamental notions of computation, identifying computability with the computation of functions, as well as with Turing Machines. –Theory of Computation was a field of mathematics. –The batch-based modus operandus of original computers did not lend itself to other conceptualizations of computation.

Sep. 29, 2004Tufts University39 Deconstructing the Turing Thesis Myth (2) Claim 2: A problem is solvable if it can be specified by an algorithm. Assumes: - A problem is solvable if there exists a Turing Machine for computing it. - Algorithmic computation is function based (the computational role of an algorithm is to transform input data to output data). Reasons for this assumption: –Original (mathematical) meaning of “algorithms” E.g. Euclid’s greatest common divisor algorithm –Knuth’s definition of algorithms specified this explicitly: “An algorithm has zero or more inputs, i.e., quantities which are given to it initially before the algorithm begins.“ [Knuth’68]

Sep. 29, 2004Tufts University40 Deconstructing the Turing Thesis Myth (3) Claim 3: Algorithms are what computers do. Reasons for this assumption: –The ACM Curriculum (1968): Adopted algorithms as the central concept of CS without explicit agreement on the meaning of this term. –When defining algorithms, most textbooks left implicit the assumption of their closed function-based nature; some explicitly violated it. “An algorithm is a recipe, a set of instructions or the specifications of a process for doing something. That something is usually solving a problem of some sort.” [Rice&Rice’69] “An algorithm is a collection of simple instructions for carrying out some task. Commonplace in everyday life, algorithms sometimes are called procedures or recipes...” [Sipser]

Sep. 29, 2004Tufts University41 The Shift to Interaction in CS Logic and search in AIIntelligent agents Procedure-oriented programming Object-oriented programming Closed systemsOpen systems Compositional behaviorEmergent behavior Rule-based reasoningSimulation, control Computation = transforming input to output Computation = carrying out a task over time Algorithmic Interactive

Sep. 29, 2004Tufts University42 Many other interactive models –Concurrency theory, process algebras –Reactive and embedded systems –Dataflow, I/O automata, synchronous languages, finite/pushdown automata over infinite words –Interaction games, online algorithms What makes PTMs unique? –First model of interaction to bridge the gap between concurrency theory (labeled transition systems) and traditional TOC. –Models of concurrency are orthogonal to traditional models of computation –Other interactive models were used as tools for proving traditional complexity results (e.g. interactive TMs in cryptography) rather than studied for their own sake Modeling Interactive Computation: PTMs in Perspective

Sep. 29, 2004Tufts University43 Rethinking the mathematical worldview Persistent Turing Machines (PTMs) PTM expressiveness Sequential Interaction Thesis Future work –Interactive complexity theory –Multiagent systems –Formalizing indirect interaction Outline

Sep. 29, 2004Tufts University44 Interactive complexity theory conjecture: useful notions of complexity can be developed for sequential interaction computation Multi-stream interaction conjecture: multi-agent interaction is more powerful than sequential interaction [Wegner’97] Formalizing indirect interaction conjecture: direct interaction does not capture all forms of multi-agent behaviors Future Work: 3 conjectures

Sep. 29, 2004Tufts University45 References [Wegner’97] Peter Wegner Why Interaction is more Powerful than Algorithms Communications of the ACM, May 1997 [EGW’04] Eugene Eberbach, Dina Goldin, Peter Wegner Turing's Ideas and Models of Computation book chapter, in Alan Turing: Life and Legacy of a Great Thinker, Springer 2004 [I&C’04] Dina Goldin, Scott Smolka, Paul Attie, Elaine Sonderegger Turing Machines, Transition Systems, and Interaction Information & Computation Journal, 2004 [GW’04] Dina Goldin, Peter Wegner The Origins of the Turing Thesis Myth Brown University Technical Report