2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.1 MAT 7003 : Mathematical Foundations (for Software Engineering) J Paul Gibson, A207 Computability

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.2 Computable Functions and Computing Machines (Computers) Alan Turing: 1937 he published a theory of computable functions in: On Computable Numbers, with an Application to the Entscheidungsproblem He reformulated Kurt Gödel's 1931 results on the limits of proof and computation, replacing Gödel's universal arithmetic-based formal language with Turing machines He went on to prove that there was no solution to the Entscheidungsproblem by first showing that the halting problem for Turing machines is undecidable: it is not possible to decide, in general, algorithmically whether a given Turing machine will ever halt. (This proof depended on the notion of a Universal machine) His proof was published subsequent to Alonzo Church's equivalent proof in respect to his lambda calculus (1936) In 1999, Time Magazine named Turing as one of the 100 Most Important People of the 20th Century for his role in the creation of the modern computer.

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.3 Computable Functions and Computing Machines (Computers) A Turing machine that is able to simulate any other Turing machine is called a Universal Turing machine (UTM, or simply a universal machine). A more mathematically-oriented definition with a similar "universal" nature was introduced by Alonzo Church (and his student Stephen Kleene) at roughly the same time. Since then, many computational models – including some very simple models - have been shown to be computationally equivalent to the Turing machine; such models are said to be Turing complete. Computability is the study of the limits of these machines

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.4 Computable Functions and Computing Machines (Computers) Informally the Church–Turing thesis states: that if an algorithm (a procedure that terminates) exists then there is an equivalent Turing machine, recursively-definable /recursively enumerable function, or applicable λ-function, for that algorithm. Or Every effectively calculable function is a computable function, where a function is effectively calculable if its values can be found by some purely mechanical process Because all the different attempts at formalizing the concept of "effective calculability/computability" have yielded equivalent results, it is now universally accepted that the Church–Turing thesis is correct.

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.5 Computable Functions and Computing Machines – Typographical/Term Rewrite Systems (TRSs) A TRS is a formal system based on the ability to generate a set of strings following a simple set of syntactic rules. Each rule is calculable --- the generation of a new string from an old string by application of a rule always terminates A TRS may produce an infinite number of strings TRSs can be as powerful as any computing machine (Turing equivalent)

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.6 Typographical Re-write Systems (TRS) TRSs are simple to implement (simulate) using other computational models Using TRSs we introduce the following concepts: proof, theorem, decision procedure, meta-analysis, structural induction, necessary and sufficient, isomorphism, meaning and consistency Don’t worry … they are very simple to understand ….

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.7 Case Study The MUI TRS ( thanks to Douglas Hofstadter) Alphabet = {M,I,U} Strings: any sequence of characters found in the alphabet Axiom: MI Generation Rules: for all strings such that x and y are strings of MUI or ‘ ‘ : 1) xI can generate xIU 2) Mx can generate Mxx 3) xIIIy can generate xUy 4) xUUy can generate xy A theorem of a TRS is any string which can be generated from the axioms (or any other theorem) A proof of a theorem corresponds to the set of rules which have been followed to generate that theorem

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.8 Case Study The MUI TRS (proof procedure) Alphabet = {M,I,U} Strings: any sequence of characters found in the alphabet Axiom: MI Generation Rules: for all strings such that x is a string of MUI or x =‘’ : 1) xI can generate xIU 2) Mx can generate Mxx 3) xIIIy can generate xUy 4) xUUy can generate xy Question: can you prove the theorem MUIIU? Question: can we automate the process of testing for theoremhood of a given string in a finite period of time? Input string True or False machine Such a machine would be a decision procedure of MUI

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.9 Case Study The MUI TRS (decision tree) Alphabet = {M,I,U} Strings: any sequence of characters found in the alphabet Axiom: MI Generation Rules: for all strings such that x is a string of MUI or x =‘’ : 1) xI can generate xIU 2) Mx can generate Mxx 3) xIIIy can generate xUy 4) xUUy can generate xy Is this a decision procedure for the MUI machine? … Construct a tree of strings, starting with the axiom at the root. Any application rule constitutes a branch of the tree. To decide if a given string is a theorem it is sufficient to keep extending the tree until the string is found. Task: construct the top (1st 3 layers) of such a tree

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.10 Case Study The MUI TRS (meta-reasoning) Alphabet = {M,I,U} Strings: any sequence of characters found in the alphabet Axiom: MI Generation Rules: for all strings such that x is a string of MUI or x =‘’ : 1) xI can generate xIU 2) Mx can generate Mxx 3) xIIIy can generate xUy 4) xUUy can generate xy Question: is IIIIUUUIIIUUUI a theorem of the system? Question: can you prove your answer is correct? Note: only through meta-reasoning can we do this !!

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.11 Case Study The MUI TRS (more meta-reasoning) Alphabet = {M,I,U} Strings: any sequence of characters found in the alphabet Axiom: MI Generation Rules: for all strings such that x is a string of MUI or x =‘’ : 1) xI can generate xIU 2) Mx can generate Mxx 3) xIIIy can generate xUy 4) xUUy can generate xy The meta-property that all theorems start with an M is called a necessary but not sufficient property of theorem-hood. Question: before we move on … is MU a theorem of MUI? Now we move onto a more practical TRS...

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.12 Case Study The pq- TRS Alphabet = {p,q,-} Axiom: for any such x such that x is a possibly empty sequence of ‘-’s, xp-qx- is an axiom Generation Rules: for any x,y,z which are possibly empty sequences of ‘-’s, if xpyqz is a theorem then xpy-qz- is a theorem Question: is there a decision procedure for this formal system? Hint: all re-write rules lengthen the string so …?

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.13 Case Study The pq- TRS Alphabet = {p,q,-} Axiom: for any such x such that x is a possibly empty sequence of ‘-’s, xp-qx- is an axiom Generation Rules: for any x,y,z which are possibly empty sequences of ‘-’s, if xpyqz is a theorem then xpy-qz- is a theorem Why is the pq- TRS practical? Because it provides us with a formal model of a mathematical property: the addition of integers p---q----- is a theorem and “2+3=5” is true --p-q-- is a non-theorem and “2+1=2” is false

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.14 Case Study The pq- TRS interpretation If we interpret p as plus q as equals and a sequence of n ‘-’s as the integer n then we have a means of checking x+y=z for all non-negative integers x,y and z We say that pq- is consistent (under the given interpretation) because all theorems are true after interpretation We say that pq- is complete if all true statements (in the domain of interpretation) can be generated as theorems in the system. We say that the interpretation is isomorphic to the system if the system is both complete and consistent

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.15 Case Study The pq- TRS extension The pq- system is isomorphic to a very limited domain of interpretation (but maybe that is all that is required!) Normally, to widen a domain we can add an axiom add a generating rule For example, what happens if we add the axiom: xp-qx. Using this, we can generate many new theorems! Question: with this new axiom what about completeness and consistency? Answer: the new, extended system is not consistent with our interpretation.

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.16 Case Study The extended pq- TRS reinterpreted After extension, --p--q--- is now a theorem but 2+1=2 is not true To solve this problem we can re-interpret for consistency --- interpet q as “ >= “ However, we have now lost completeness --- “2+5 >= 4” is true (in our domain of interpretation) but --p-----q---- is a non-theorem Note: this is a big problem of mathematics (c.f Church) --- it is not possible to have a complete, decidable system of mathematical properties which is consistent if all the theorems that can be checked are consistent then there are some things which we would like to be able to prove as theorems which the system is not strong enough for us to do

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.17 Case Study A tq- TRS Question: can you define a TRS for modelling the multiplication of two integers can you show that it is complete and consistent Interpretation: t as times q as equals sequences of ‘-’s as integers

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.18 BACK TO THE SOFTWARE PROCESS … Imagine you were asked to implement a function, f say, to calculate the ith prime number. Thus, given the primes to be 2,3,5,7,11,13,17,19,…, f(1) =2, f(2) = 3, f(3) =5, … I assume you could all code this directly in C++ (Java, Prolog …) How many of you could prove your code was correct? Where would you even start? First: formalise requirements ‘automagically’ Second: transform requirements into design and prove transformation to be correct Third: keep correctly transforming design until it is directly implementable Fourth: implement it ‘automagically’

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.19 BACK TO THE SOFTWARE PROCESS … First: formalise requirements ‘automagically’ 1) formally define primes 2) formally define lists (and lengths and orderings) 3) formally define the list of ordered primes Second: transform requirements into design and prove transformation to be correct 4) design an algorithm to check that the length of the list ( l say) up to your result (r, say) is such that f(l) = r. Third: nothing to do?? Fourth: implement it ‘automagically’ Where/ how you do this is part of the decision making process.

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.20 A TRS for formally defining if a number is prime Note: easier to do in other formal languages/methods because the necessary concepts (like integers and lists are part of the language) But, with the TRS we define just what we need and use it only where needed. In software process it is this targetting (with the minimum force necessary) which is best … Question: can you write a TRS for deciding if a given number is prime? Hint: if not, try to break the problem down into bits For the lists model/properties we should (but don’t have to) move up a level of abstraction! We introduce Abstract Data Types…. IMHO the most powerful and universally applicable software process formal methods tool.

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.21 From TRSs to Abstract Data Types (ADTs) ADTs are a very powerful specification technique which exist in many forms (languages). These languages are often given operational semantics in a way similar to TRSs (in fact, they are pretty much equivalent) Most ADTs have the following parts --- A type which is made up from sorts Sorts which are made up of equivalent sets Equivalent sets which are made up of expressions For example, the integer type could be made up of sorts integer and boolean 1 equivalence set of the integer sort could be {3, 1+2, 2+1, 1+1+1} 1 equivalence set of the boolean sort could be {3=3, 1=1, not(false)}

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.22 Case Study 4: A simple ADT specification TYPE integer SORTS integer, boolean OPNS 0:-> integer succ: integer -> integer eq: integer, integer -> boolean +: integer, integer -> integer EQNS forall x,y: integer 0 eq 0 = true; succ(x) eq succ(y) = x eq y; 0 eq succ(x) = false; succ(x) eq 0 = false; 0 + x = x; succ(x) + y = x + (succ(y)); ENDTYPE

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.23 Case Study 4: A simple ADT specification TYPE integer SORTS integer, boolean OPNS 0:-> integer succ: integer -> integer eq: integer, integer -> boolean +: integer, integer -> integer EQNS forall x,y: integer 0 eq 0 = true; succ(x) eq succ(y) = x eq y; 0 eq succ(x) = false; succ(x) eq 0 = false; 0 + x = x; succ(x) + y = x + (succ(y)); ENDTYPE Question: how do we show, for example = 3, 3+2 = 4+1, 2+2 != 3+2

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.24 Case Study 4: A simple ADT specification TYPE integer SORTS integer, boolean OPNS 0:-> integer succ: integer -> integer eq: integer, integer -> boolean +: integer, integer -> integer EQNS forall x,y: integer 0 eq 0 = true; succ(x) eq succ(y) = x eq y; 0 eq succ(x) = false; succ(x) eq 0 = false; 0 + x = x; succ(x) + y = x + (succ(y)); ENDTYPE Note: this model is complete and consistent with respect to the modelling of the addition of integers (like the TRS pq-) Question: extend this model to include multiplication

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.25 Case Study 4: An equivalent ADT specification TYPE integer SORTS integer, boolean OPNS 0:-> integer succ: integer -> integer eq: integer, integer -> boolean +: integer, integer -> integer EQNS forall x,y: integer 0 eq 0 = true; succ(x) eq succ(y) = x eq y; 0 eq succ(x) = false; succ(x) eq 0 = false; 0 + x = x; succ(x) + y = x + (succ(y)); x+y = y+x; ENDTYPE Consider changing the original specification to make explicit the fact that x+y = y +x, for all integer values of x and y: Note: this does not change the meaning of the specification but it may affect the implementation of the evaluation of expressions

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.26 Case Study 4: Evaluation termination If expressions are evaluated as left to right re-writes (as they often are) then evaluation may not terminate: 3 +4 = 4+3 may be re-written as 4+3 = 3+4 which may be re-written as 3+4 = 4+3 … Consequently, there are 3 important properties of ADT specifications: completeness consistency evaluation termination/convergence

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.27 Case Study 4: Incompleteness, inconsistency and termination Not having enough equations can make a specification incomplete. For example, the integer ADT specification would be incomplete without the equation: 0 eq 0 = true Having too many equations can make a specification inconsistent. For example, the integer ADT specification is inconsistent if we add the equation: x + succ(0) = x but adding the equation: x + succ(0) = succ(x) would not introduce inconsistency (just redundancy) Changing the equations may affect termination: 0 + x = x to x + 0 = x would introduce non-termination to the original ADT specification

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.28 Case Study A Set ADT specification TYPE Set SORTS Int, Bool OPNS empty:-> Set str: Set, int -> Set add: Set, int -> Set contains: Set, int -> Bool EQNS forall s:Set, x:Int contains(empty, int) = false; x eq y => contains(str(s,x), y) = contains(s,y); not (x eq y) => contains(str(s,x), y) = contains(s,y); contains(s,x) => add(s,x) = s; not(contains(s,x)) => add(s,x) = str(s,x) ENDTYPE Notes: use of str and add preconditions completeness? consistency? Question: add operations for -- remove union equality

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.29 Case Study 6: Set verification We would like to verify the following properties: e  (S-e) = true e  S1  S2  e  S1  e  S2 Proof technique: structural induction on the ADT specification Question: try it yourselves to see how it goes... Invariant Property: verify also that a set never contains any repeated elements

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.30 Back to the Primes Proof Question: write an ADT specification of a list of integers include a means of verifying that it is ordered include a function for returning the length All that is left to do is plug the two parts together and we have a formal specification (and implementation) of our prime problem requirements. We can look at this problem within another model of computation: automata

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.31 Computable Functions and Computing Machines: Finite Automata Finite automata are computing devices that accept/recognize regular languages and are used to model operations of many systems we find in practice. A classic example is of a vending machine. For example, consider a simple vending machine that accepts only nickels and dimes and requires a payment of 15 cents: QUESTION: how many different paths to a terminating/accepting state?

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.32 Computable Functions and Computing Machines: Finite Automata Definition: deterministic finite automaton (DFA) The set Q in the above definition is simply a set with a finite number of elements. Its elements can, however, be interpreted as a state that the system (automaton) is in. The transition function is also called a next state function meaning that the automaton moves into the state (q, a) if it receives the input symbol a while in state q. Note that δ is a function. Thus for each state q of Q and for each symbol a of ∑, δ (q, a) must be specified. If the finite automaton is in an accepting state when the input ceases to come, the sequence of input symbols given to the finite automaton is "accepted"

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.33 Computable Functions and Computing Machines: Finite Automata DFAs are often represented by digraphs called (state) transition diagram. The vertices (denoted by single circles) of a transition diagram represent the states of the DFA The arcs labeled with an input symbol correspond to the transitions. An arc ( p, q ) from vertex p to vertex q with label σ represents the transition δ(p, σ ) = q. The accepting states are indicated by double circles. Transition functions can also be represented by transition tables.

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.34 Computable Functions and Computing Machines: Finite Automata Examples of finite automaton

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.35 Computable Functions and Computing Machines: Finite Automata Examples of finite automaton TO DO: Draw the DiGraph corresponding to this transition table

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.36 Computable Functions and Computing Machines: Finite Automata Examples of finite automaton TO DO: Draw the transition table corresponding to this digraph

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.37 Computable Functions and Computing Machines: Finite Automata A finite automaton as a machine A finite automaton can also be thought of as the device consisting of a tape and a control circuit which satisfy the following conditions: The tape has the left end and extends to the right without an end. The tape is divide into squares in each of which a symbol can be written prior to the start of the operation of the automaton. The tape has a read only head. The head is always at the leftmost square at the beginning of the operation. The head moves to the right one square every time it reads a symbol. It never moves to the left. When it sees no symbol, it stops and the automaton terminates its operation. There is a finite control which determines the state of the automaton and also controls the movement of the head.

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.38 Computable Functions and Computing Machines: Finite Automata A finite automaton as a machine

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.39 Computable Functions and Computing Machines: Finite Automata The Chomsky hierarchy: linking machines to languages Type-0 grammars (unrestricted grammars) include all formal grammars. They generate exactly all languages that can be recognized by a Turing machine. Type-1 grammars (context-sensitive grammars) generate the context-sensitive languages. The languages described by these grammars are exactly all languages that can be recognized by a linear bounded automaton (a nondeterministic Turing machine whose tape is bounded by a constant times the length of the input.) Type-2 grammars (context-free grammars) generate the context-free languages. These languages are exactly all languages that can be recognized by a non- deterministic pushdown automaton. Context-free languages are the theoretical basis for the syntax of most programming languages. Type-3 grammars (regular grammars) generate the regular languages. These languages are exactly all languages that can be decided by a finite state automaton. Regular languages are commonly used to define search patterns and the lexical structure of programming languages.

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.40 Computable Functions and Computing Machines: Finite Automata The differences arise out of the different types of production rules that are allowed:

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.41 Computable Functions and Computing Machines: Finite Automata Finite Automata and Regular Languages - some definitions

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.42 Computable Functions and Computing Machines: Finite Automata Finite Automata and Regular Languages - some definitions

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.43 Computable Functions and Computing Machines: Finite Automata Finite Automata and Regular Languages - some definitions

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.44 Computable Functions and Computing Machines: Finite Automata Finite Automata and Regular Languages - some definitions

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.45 Computable Functions and Computing Machines: Finite Automata Finite Automata and Regular Languages - some definitions

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.46 Computable Functions and Computing Machines: Finite Automata Finite Automata and Regular Languages

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.47 Computable Functions and Computing Machines: Finite Automata Finite Automata, Regular Languages and Regular Expressions We use the following operations to construct regular expressions: Boolean or - A vertical bar (or +) separates alternatives. For example, gray|grey can match "gray" or "grey". Grouping - Parentheses are used to define the scope and precedence of the operators For example, gray|grey and gr(a|e)y are equivalent patterns which both describe the set of "gray" and "grey". Quantification - A quantifier after a token (such as a character) or group specifies how often that preceding element is allowed to occur. The most common quantifiers are the question mark ?, the asterisk * (derived from the Kleene star), and the plus sign +. ?The question mark indicates there is zero or one of the preceding element. For example, colou?r matches both "color" and "colour". *The asterisk indicates there are zero or more of the preceding element. For example, ab*c matches "ac", "abc", "abbc", "abbbc", and so on. +The plus sign indicates that there is one or more of the preceding element. For example, ab+c matches "abc", "abbc", "abbbc", and so on, but not "ac".

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.48 Computable Functions and Computing Machines: Finite Automata Finite Automata, Regular Languages and Regular Expressions Kleene’s Theorem

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.49 Computable Functions and Computing Machines: Finite Automata Finite Automata, Regular Languages and Regular Expressions QUESTIONS: TO DO 1 Find the shortest string that is not in the language represented by the regular expression a*(ab)*b*. 2 For the two regular expressions given below: (a) find a string corresponding to r2 but not to r1 and (b) find a string corresponding to both r1 and r2. r1 = a* + b* r2 = ab* + ba* + b*a + (a*b)* 3 Let r1 and r2 be arbitrary regular expressions over some alphabet. Find a simple (the shortest and with the smallest nesting of * and +) regular expression which is equal to each of the following regular expressions (a) (r1 + r2 + r1r2 + r2r1)* (b) (r1(r1 + r2)*)+

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.50 Computable Functions and Computing Machines: Finite Automata Finite Automata, Regular Languages and Regular Expressions QUESTION: Can we build a regular expression to check that a string is a palindrome? ANSWER: NO … can you think about why this is the case?

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.51 Computable Functions and Computing Machines: Finite Automata Nondeterministic Finite Automata NFAs are quite similar to DFAs. The only difference is in the transition function. NFAs do not necessarily go to a unique next state. An NFA may not go to any state from the current state on reading an input symbol or it may select one of several states nondeterministically (e.g. by throwing a die) as its next state.

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.52 Computable Functions and Computing Machines: Finite Automata Nondeterministic Finite Automata

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.53 Computable Functions and Computing Machines: Finite Automata Nondeterministic Finite Automata: Example

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.54 Computable Functions and Computing Machines: Finite Automata Nondeterministic Finite Automata: Theorem NOTE: L is the language accepted by the machine/automata QUESTION: Can you convert this NDFA into an equivalent DFA?

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.55 Computable Functions and Computing Machines: Finite Automata Nondeterministic Finite Automata: Theorem We have seen this (intuitively) with palindromes, but the pumping lemma allows us to prove it formally NOTE: The language of palindromes is not regular (type 3) but it is context-free (type 2).

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.56 Computable Functions and Computing Machines: Finite Automata Nondeterministic PushDown Automata A (non-deterministic) pushdown automaton is like an NFA, except it has a stack (pushdown store) for recording a potentially unbounded amount of information, in a last-in-first-out (LIFO) fashion. Theorem. A language is context-free (Chomsky type 2) iff some NPDA accepts it.

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.57 Computable Functions and Computing Machines: Finite Automata The workings of an NPDA In each step, the NPDA pops the top symbol off the stack; based on (1) this symbol, (2) the input symbol currently reading, and (3) its current state, it can: 1. push a sequence of symbols (possibly ) onto the stack 2. move its read head one cell to the right, and 3. enter a new state according to the transition rule of the machine. We allow ε -transition: an NPDA can pop and push without reading the next input symbol or moving its read head. Note: an NPDA can only access the top of stack symbol in each step.

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.58 Computable Functions and Computing Machines: Finite Automata Example: primes An example of a language that is not context-free is: The language can easily be shown to be neither regular nor context free by applying the respective pumping lemmas for each of the language classes to L.

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.59 Computable Functions and Computing Machines: finite automata A linear bounded automaton is a restricted form of a nondeterministic Turing machine. Linear bounded automata are acceptors for the class of context- sensitive languages (type 1 in Chomsky hierarchy). L can be shown to be a context-sensitive language by constructing a linear bounded automaton which accepts L. We will see what this looks like after we consider Turing machines

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.60 Turing Machines: consider a typical/standard definition A Turing machine is a kind of state machine. At any time the machine is in any one of a finite number of states. Instructions for a Turing machine consist in specified conditions under which the machine will transition between one state and another. A Turing machine has an infinite one-dimensional tape divided into cells. Traditionally we think of the tape as being horizontal with the cells arranged in a left-right orientation. The tape has one end, at the left say, and stretches infinitely far to the right. Each cell is able to contain one symbol, either ‘0’ or ‘1’. The machine has a read-write head, which at any time scanning a single cell on the tape. This read-write head can move left and right along the tape to scan successive cells. The action of a Turing machine is determined completely by (1) the current state of the machine (2) the symbol in the cell currently being scanned by the head and (3) a table of transition rules, which serve as the “program” for the machine.

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.61 Turing Machines: consider a typical/standard definition

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.62 Turing Machines: consider a typical/standard definition In modern terms: the tape serves as the memory of the machine the read-write head is the memory bus through which data is accessed (and updated) by the machine. Note: the machine's tape is infinite in length, corresponding to an assumption that the memory of the machine is infinite. that a function will be Turing-computable if there exists a set of instructions that will result in the machine computing the function regardless of the amount of time it takes. One can think of this as assuming the availability of infinite time to complete the computation. If a function is not Turing-computable it is because Turing machines lack the computational machinery to carry it out, not because of a lack of spatio-temporal resources.

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.63 Turing Machines: consider a typical/standard definition Describing Turing Machines Every Turing machine has the same machinery. What makes one Turing machine perform one task and another a different task is the table of transition rules that make up the machine's program, and a specified initial state for the machine. We will assume throughout that a machine starts in the lowest numbered of its states. We can describe a Turing machine, therefore, by specifying only the 4-tuples that make up its program. Note: there are « equivalent » alternative definitions, eg using 5-tuples where the machine can write on the tape and move the head in a single step

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.64 Turing Machines: consider a typical/standard definition Example In order to speak about a Turing machine that does something useful, we will have to provide an interpretation of the symbols recorded on the tape. For example, if we want to design a machine which will perform some mathematical function, addition say, then we will need to describe how to interpret the ones and zeros appearing on the tape as numbers.

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.65 Turing Machines: another typical/standard definition Example: A Turing machine for checking if input string is a palindrome TO DO: check you understand how this works

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.66 Turing Machines: programming problem TO DO: design a TM to calculate the remainder function % X%Y = Z ∃ N: NAT. X = Y*N + Z && Z<Y, where X and Y are positive integers and Z is a non-negative integer Eg 18%5 = 3 since 18=5*3 + 3 && 3<5 20%4 = 0 since 20 = 5*4 +0 Your input string will be written in unary notation with the tape head pointing at the 1st character in the input string: %111 X represents the function call 8%3 The result (in this case 11) should be output to the immediate right of the tape head: 11X

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.67 Computable Functions and Computing Machines: finite automata A linear bounded automaton: a restriction on TMs An LBA differs from a Turing machine in that while the tape is initially considered to have unbounded length, only a finite contiguous portion of the tape, whose length is a linear function of the length of the initial input, can be accessed by the read/write head. This limitation makes an LBA a more accurate model of computers that actually exist than a Turing machine in some respects. A linear bounded automaton recognizes context-sensitive languages A Turing machine recognizes all formal languages (unrestricted grammars) … these are also known as recursively enumerable languages. Recursively enumerable sets are also known as semi-recursive/partially decidable sets because it is always possible to confirm in finite time – using a Turing Machine - that a given element is a member of the set, but not necessarily that it isn’t.

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.68 Computable Functions and Computing Machines: finite automata Recursively enumerable vs recursive A recursively enumerable language is a formal language for which there exists a Turing machine (or other computable function) that will halt and accept when presented with any string in the language as input but may either halt and reject or loop forever when presented with a string not in the language. Contrast this to recursive languages, which require that the Turing machine halts in all cases. Recursive languages are between type 0 and type 1 in the original Chomsky hierarchy

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.69 Computable Functions and Computing Machines: finite automata Recursively enumerable vs recursive

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.70 Computable Functions and Computing Machines: finite automata Universal Turing Machine Every Turing machine computes a certain fixed partial computable function from the input strings over its alphabet. In that sense it behaves like a computer with a fixed program. However, we can encode the action table of any Turing machine in a string. Thus we can construct a Turing machine that expects on its tape a string describing an action table followed by a string describing the input tape, and computes the tape that the encoded Turing machine would have computed. Turing described such a construction in complete detail in his 1936 paper ASIDE: There are some very small universal Turing Machines c.f. Woods and Neary QUESTION:can you guess how small (number of states and transitions)

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.71 Computable Functions and Computing Machines: finite automata Universal Machines In order to show that any other machine is universal (ie it is equivalent in power to a Universal Turing Machine, we usually follow a proof by construction/proof by simulation: To show an A (some class of machines) is as powerful as a B (some class of machines) we need to show that for any B, there is some equivalent A. Proof-by-construction: Given any b  B, construct an a  A that recognizes the same language as b. Proof-by-simulation: Show that there is some A that can simulate any B. Note: the existence of a Universal machine in B guarantees the existence of a universal machine in A. Also, we can show that a machine is no more powerful than a Turing machine by showing that a TM can simulate its behaviour.

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.72 Computable Functions and Computing Machines: finite automata Universal Machines: Register Machines It has been proven that Register Machines are universal – recognise recursively enumerable languages. These machines are interesting because they correspond to a very low-level programming language. There are many different sub-classes - from primitive to computer-like, eg: Counter machine -- the most primitive and reduced model. Lacks indirect addressing. Pointer machine -- a blend of counter machine and RAM models. Random access machine (RAM) -- a counter machine with indirect addressing and, usually, an augmented instruction set. Random access stored program machine model (RASP) -- a RAM with instructions in its registers analogous to the Universal Turing machine; thus it is an example of the von Neumann architecture. But unlike a computer the model is idealized with effectively-infinite registers. The instruction set is much reduced in the number of instructions. The most primitive machine models are useful if one wishes to show (through simulation) that another machine is universal. The most powerful are useful in showing that something (set/function) is semi-computable/decidable

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.73 Computable Functions and Computing Machines: finite automata Universal Machines: A Simple Register Machine NOTE: You dont need to know how this works – you just need to see the simplicity of the definition.

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.74 Computable Functions and Computing Machines: finite automata Universal Machines: Register Machine Extensions

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.75 Computable Functions and Computing Machines: finite automata Universal Machines: lambda-calculus The Lambda Calculus was developed by Alonzo Church in the 1930s and published in 1941 as ‘The Calculi Of Lambda Conversion’. It became important, along with Turing machines, in the development of computation theory, and is the theoretical basis of all functional programming languages, such as Lisp, Haskell and ML.

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.76 Computable Functions and Computing Machines: finite automata Universal Machines: lambda-calculus The calculus can be called the smallest universal programming language.The calculus consists of a single transformation rule (variable substitution) and a single function denition scheme.

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.77 Computable Functions and Computing Machines: finite automata Universal Machines: lambda-calculus The lambda calculus can be used to implement: Natural numbers and arithmetic Boolean logic Conditional execution Recursion The lambda calculus is equivalent to a Turing machine

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.78 Computable Functions and Computing Machines: finite automata The halting problem Given a description of a program, decide whether the program finishes running or will run forever. This is equivalent to the problem of deciding, given a program and an input, whether the program will eventually halt when run with that input, or will run forever. Alan Turing proved in 1936 that a general algorithm to solve the halting problem for all possible program-input pairs cannot exist. The halting problem is undecidable over Turing machines. The halting problem is famous because it was one of the first problems proven algorithmically undecidable. (First for the lambda calculus and then for Turing machines)

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.79 Computable Functions and Computing Machines: finite automata The halting problem: sketch of proof The proof shows there is no total computable function that decides whether an arbitrary program i halts on arbitrary input x; that is, the following function h is not computable: Here program i refers to the i th program in an enumeration of all the programs of a fixed Turing-complete model of computation. The proof proceeds by directly establishing that every total computable function with two arguments differs from the required function h. To this end, given any total computable binary function f, the following partial function g is also computable by some program e:

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.80 Computable Functions and Computing Machines: finite automata The halting problem: sketch of proof Because g is partial computable, there must be a program e that computes g, by the assumption that the model of computation is Turing-complete. This program is one of all the programs on which the halting function h is defined. The next step of the proof shows that h(e,e) will not have the same value as f(e,e). It follows from the definition of g that exactly one of the following two cases must hold: g(e) = f(e,e) = 0. In this case h(e,e) = 1, because program e halts on input e. g(e) is undefined and f(e,e) ≠ 0. In this case h(e,e) = 0, because program e does not halt on input e. In either case, f cannot be the same function as h. Because f was an arbitrary total computable function with two arguments, all such functions must differ from h. This proof is typically referred to as a diagonalization proof.

2012: J Paul GibsonTSP: Mathematical FoundationsMAT7003/L7-Computability.81 Computable Functions and Computing Machines: finite automata The halting problem: solvable on finite machines (in theory) The halting problem is, in theory if not in practice, decidable for linear bounded automata (LBAs), or deterministic machines with finite memory. A machine with finite memory has a finite number of states, and thus any deterministic program on it must eventually either halt or repeat a previous state: "...any finite-state machine, if left completely to itself, will fall eventually into a perfectly periodic repetitive pattern. The duration of this repeating pattern cannot exceed the number of internal states of the machine..."(Minsky 1967, p. 24) Minsky warns us, however, that machines such as computers with e.g. a million small parts, each with two states, will have on the order of 2^1,000,000 possible states!!!!!