CS 345: Chapter 9 Algorithmic Universality and Its Robustness The Simplest Machines that Get It Done

Search For Simplicity The beginning of this chapter focuses on reducing an algorithm to its simplest components. Part of the purpose of this is to find a model of computation that is general in nature. Note that is driven partly by the nature of the mathematician.

Also, by removing the dependency on a particular machine, language, technology, an algorithm can be examined in its essential form. A common model of computation in theoretical computer science was developed by Alan Mathison Turing (1912 - 1954) Turing developed the concept of a Universal – Algorithm Machine. Today, we call this model a Turing Machine (TM).

Turing Machines Although a Turing Machine (TM) has an elegant mathematical definition, we can visualize one as a physical entity. Tape for input and output Read/Write head Finite-State Machine for control Example: EVEN-EVEN, PALINDROME

Turing Machine TM = (Q, , , , q0, , F) Q = a finite set of internal states  = a finite set of symbols: the input alphabet.  = a finite set of symbols: the tape alphabet. : Q Q  { L, R }, a transition function q0  Q is the initial state. is a blank symbol and  F  Q is a set of halt states. It is assumed that     {}

EVEN-EVEN Determines if a string of a’s and b’s have an even number of both. Uses 4 states S1: read even a’s and even b’s S2: read even a’s and odd b’s S3: read odd a’s and even b’s S4: read odd a’s and odd b’s If you HALT in S1, there are an even number of a’s and b’s

EVEN-EVEN a S1 S3 a b b b b a S2 S4 a

PALINDROME Uses 8 states Read first letter of input, remember it, then erase it. Go to other end of input, check that it is same letter. If same letter, erase it, go to beginning of input and repeat process. If not, then crash

PALINDROME 8 a/a R b/b R a/a L b/b L #/# L a/# L 2 3 4 #/# R #/# R 1 b/# R #/# R #/# L b/# L #/# R 5 6 7 a/a R b/b R a/a L b/b L

Programming A Turing Machine We can think of a TM as a primitive type of computer that executes a fixed program. The program is stored within the TM’s state-transition diagram. The input is coded as a linear set of strings created from the input alphabet and stored on the tape. 1

A TM can be used to solve a decision problem by having one halt state for “yes” and one for “no”. For a non-decision problem, a TM can use special marks to indicate its output or erase its tape leaving only its output. A TM has very limited actions and is very tedious to program for even trivial problems. However, we can ask, “exactly what can be solved by a TM?” 2

The Church-Turing Thesis Which algorithmic problems can be solved by a TM? Any algorithmic problem for which we can find an algorithm that can be programmed in some programming language, any language, running on some computer, any computer, even one that has not been built yet, but can be built, and even one that will require unbounded amounts of time and memory space for ever-larger inputs is solvable by a Turing Machine. A TM is capable of solving any effectively solvable algorithmic problem. 3

The Church-Turing Thesis is not a theorem. The Church-Turing Thesis equates the mathematically precise notion of solvable by a Turing Machine with the informal, intuitive notion of solvable effectively, which refers to real computers and languages. Although it sounds like wild speculation, it is a deep and far-reaching statement put forward by two of the most respected pioneers of theoretical computer science.

Computability is Robust models of a universal computer Alonzo Church: Lambda Calculus Alan Turing: Turing Machines Emil Post: Production Systems Stephen Kleene: Recursive Functions A consequence of the Church-Turing Thesis is that all these formalisms have been shown to be equivalent 4

Another meaning for robustness is that the class of computable, effectively solvable, or decidable algorithmic problems remains the same even when we change the computer model or the programming language. This means that all programming languages that have the same capabilities of a TM are equivalent in terms of the problems they can solve. This robustness applies even to variants of a TM

Variants of a Turing Machine Movement: Left, Right, No movement One-way infinite tape. Multiple tapes: multiple read/write heads or movement in multiple dimensions Transducers and Acceptors The equivalence of these different variations is shown by demonstration how one type of machine can simulate another.

Simulation as Reduction To say one model of computation can simulate another is to say there is a reduction from one model to another. An interesting concept is that one TM can simulate another. That is, a TM can take for input an encoded TM and its input and run the encoded TM on the given input. This is similar to what an interpreter does.

A Turing Machine can be encoded using strings of 0 and 1 A Turing Machine can be encoded using strings of 0 and 1. In this manner, a TM can be associated with a number. This number is called its Gödel number and the process of encoding a TM as a number is called the Gödelization of a TM. The TM that takes for input the encoding of any other TM together with its input and simulates the second TM running on its input is called the Universal Turing Machine.

The UTM was designed by Turing in 1936 and is not only the foundation of all Computer Theory, but it was also the conceptual archetype of the early computers.

TM’s and P vs. NP A TM is very inefficient for even trivial tasks. However, a TM is only polynomially less efficient than even the most sophisticated computers Therefore, we can define a problem to belong to P if it can be solved by a deterministic TM in polynomial time

A problem belongs to NP if it can be solved in polynomial time by a nondeterministic Turing Machine Turing Machine for Lower Bound Proofs Turing Machine with one-way tape or Finite Automata Finite Automata with a Push Down Stack or PDA Turing Machine with a bounded tape (not infinite) or LBA

Languages and Machines The study of formal languages looks at the relationship between types of languages and the type of machine that can recognize or accept that language. For example, the type of language recognized by a FA is called a regular language

As part of this language, a FA can recognize when there are an even number of a’s and b’s in a string, but it cannot tell if there are the same number of a’s and b’s A FA cannot count because it has no memory only states and there is a finite number of states. The matching of machine’s, languages, and grammars is known as the Chomsky Hierarchy after linguist Noam Chomsky

Chomsky Hierarchy

One area where this study of formal languages and machines is important is for compiler construction Another area is Natural Language Processing. FA are used to represent many processes. Note, figure 9.14 on page 247. If you know what a DFD is, this figure might look similar. A FA is used by the scanner of a compiler to recognize tokens.

A simple FA for recognizing an identifier in Java Letter or Digit Letter Underscore Letter or Digit A simple FA for recognizing an identifier in Java