1. Finite State Automata / Machines and Turing Machines

2. Models of Computation  These are fundamental models of what can be achieved using a computer, independent of any specific hardware platform or programming language.  They provide an introduction to the formal models that students will encounter if they study computing at university.

3. Models of Computation  We will study the following two models: ◦ Finite State Automata / Machines: A model of logic systems that is based on the concept of states, with input symbols or actions triggering transitions between these states. ◦ Turing Machines: A general model of computation based on states. Data is stored on a tape, which can be read and written by a head. Changes to the data on the tape and the state of the machine are made using a transition function.  There are many other models such as Pushdown Automata and Markov Algorithms, some of which can be shown to be equivalent to each other in their capabilities.

4. Models of Computation  Finite state automata (FA) are used to recognise strings. Possible uses include: o Recognising and accepting the strings that can be used in a formal language, such as the valid keywords in a programming language (lexical analysis). o Searching for strings within a block of text, such as in a text document. o Searching for pages containing a certain string on the world wide web using a search engine.  It has been proven that the class of languages that can be recognised by FA are the same languages that can be expressed using regular expressions.

5. Models of Computation  A finite state automaton (FA) consists of: ◦ An input alphabet, which is the set of symbols that the automaton will accept as input. ◦ A finite number of states. ◦ A set of transitions between these states, which are triggered by the input symbols. ◦ One state is the start state: the initial state of a machine when a computation starts. ◦ Some states are designated as accepting / final states: the automaton accepts its input if it finishes in one of these states.  A sequence of symbols input into an automaton is known as a string.  The set of all strings accepted by an automaton is known as its language.

6. Models of Computation State01 S0S0 S0S0 S1S1 S1S1 S0S0 S2S2 S2S2 S0S0 S2S2  States: { S 0, S 1, S 2 }  Input Alphabet: { 0, 1 }  Start State: S 0 (indicated by incoming arrow)  Accepting States: { S 2 } (indicated by double circle)  Transitions: Are in the transition table to the right. Finite state automata are often pictured as state transition diagrams. The state transition diagram below shows an FA which accepts the set of binary numbers that end in two ones. Transition Table

7. Models of Computation  Input String: 0110 State01 S0S0 S0S0 S1S1 S1S1 S0S0 S2S2 S2S2 S0S0 S2S2 Transition Table State Transition Diagram StateInputNew State S0S0 0S0S0 S0S0 1S1S1 S1S1 1S2S2 S2S2 0S0S0 StateInputNew State S0S0 0S0S0 S0S0 0S0S0 S0S0 1S1S1 S1S1 1S2S2 Definition of FA String Rejected  Input String: 0011 String Accepted  

8. Models of Computation  Q1) Is input string aba accepted? Stateab S0S0 S1S1 S2S2 S1S1 S1S1 S1S1 S2S2 S2S2 S3S3 S3S3 S2S2 S3S3 Transition Table State Transition Diagram Definition of FA  Q2) Is input string baab accepted?  Q3) What is the purpose? View Solutions

9. Models of Computation  The finite state machines that we have looked at so far are all concerned with accepting or rejecting input strings, these are known as finite state automata.  Finite state machines can also produce output.  Some tasks that a finite state machine with output can carry out that a finite state automaton cannot are: ◦ Processing strings to produce output. e.g. encrypting, counting substrings, performing some binary calculations, number base conversion. ◦ Controlling electronic devices. ◦ Mimicking neural networks.  Additionally, some tasks that can be achieved by a finite state automaton can be achieved more efficiently by a finite state machine which produces output. For example, less states may be required.

10. Models of Computation  A finite state machine (FSM) is an finite state automaton (FA) which can produce output.  In addition to the properties of an FA an FSM also has: ◦ An output alphabet (the symbols or actions that can be output). ◦ An association between outputs and either transitions (Mealy machines) or states (Moore machines).  Unlike an FSA, an FSM does not have accepting states.

11. Models of Computation  A Moore machine is a type of finite state machine that associates output with states – the output depends only on the current state.  Each state is labelled with the name of the state and the output that should be produced in that state. o e.g. the label ‘S 0, 1’ means that the state is named S 0 and in this state, a 1 should be output.

12. Models of Computation  A Moore machine to output a unary count of how often substring ‘ab’ appears in a string. Notes: (1)States labelled with name and associated output. (2)ℇ = the empty string Input Symbol Original State New State Output bS0S0 S0S0 ℇ aS0S0 S1S1 ℇ bS1S1 S2S2 1 aS2S2 S1S1 ℇ aS1S1 S1S1 ℇ bS1S1 S2S2 1 Input string ‘babaab’ produces output ‘11’ which is the unary representation of 2 as it contains ‘ab’ twice:

13. Models of Computation  A Mealy machine is a type of finite state machine that associates output with transitions – the output depends on the current state and the input symbol.  Each transition is labelled with the input symbol that triggers the transition and the output that should be produced. o e.g. the label ‘1 | x’ on a transition means that when the symbol 1 is input, x should be output.

14. Models of Computation  A Mealy machine to output a unary count of how often pairs of letters appear in a string. Notes: (1)Transitions labelled with input and output symbols (2)ℇ = the empty string Input Symbol Original State New State Output bS0S0 S2S2 ℇ bS2S2 S2S2 1 bS2S2 S2S2 1 aS2S2 S1S1 ℇ aS1S1 S1S1 1 bS1S1 S2S2 ℇ Input string ‘bbbaab’ produces output ‘111’ which is the unary representation of 3 as it contains ‘bb’ twice and ‘aa’ once.

15. Models of Computation  Moore and Mealy machines are equivalent. You can make a Moore machine to perform the same function as any Mealy Machine and vice-versa.  The ‘best’ choice of machine type depends upon the scenario.

16. Models of Computation  Because an FA / FSM only has a finite number of states (say N states), it can only “remember” or “count” N things.  There are therefore some strings that an FA / FSM cannot recognise. e,g. strings consisting of an equal number of 0s and 1s, strings with balanced parentheses such as ((()())).  So, a more complex theoretical model is required, that can better incorporate the concept of “memory”. One such model is the Turing machine.

17. Models of Computation Click here for some additional exercises on FA/FSM or here to start Turing Machines.here

18. Models of Computation  Informally, a Turing machine (TM) can be thought of as a finite state machine that has memory, in the form of one or more infinitely long tapes that can be read from and written to. 1010111 ........ SESE Tape for memory Head at current position Current state Finite state machine controls operation

19. Models of Computation  We shall only consider a Turing machine with one tape that is infinitely long in one direction.  Some variations of this model exist: ◦ With a tape that is infinitely long in both directions. ◦ With more than one tape, and one head per tape. ◦ With a multidimensional tape. ◦ Non-deterministic.  We shall not study these variations as it has been proven that they are no more powerful that the standard model.  Computer scientists use them as: ◦ A TM produced using one of these variations may be simpler than the single-tape equivalent. ◦ Some proofs can be produced more efficiently using one the variations.

20. Models of Computation A Turing machine (TM) consists of:  A finite number of states.  A start state which is the state that the TM begins a computation in. Also known as the initial state.  A set of terminal states. If the TM enters one of these states during a computation, it is said to have accepted its input and will immediately stop. Also know as accepting, stop, halting or final states.  A tape alphabet, the symbols which can appear on the tape.  An input alphabet, which is a proper subset of the tape alphabet, and which the initial input must be written in.  A blank symbol, which is in the tape alphabet but not the input alphabet, and is used to mark an empty cell on the tape.  A transition function which specifies the action that a TM will take based upon its current state and the symbol read from the tape.

21. Models of Computation The steps involved in a TM computation are:  At the start: ◦ The data to be used (the input) is written onto the tape. ◦ The TM starts in an initial state with its head positioned at the leftmost tape square.  Repeatedly: ◦ The symbol stored at the square on the tape that the head is positioned at is read. ◦ A transition is made using the transition function. This will be based upon the symbol read and the current state of the TM. For each transition the TM will:  Change state (although this may be to the state the TM is already in).  Write a symbol to the tape on the square at which the head is positioned, overwriting the current contents.  Move the head one square left or right, based on the transition function.  The computation finishes if no transition is defined, if the head moves past the left-hand end of the tape or if a terminal state is reached (latter case = input accepted).  The data present on the tape at the end is the TM’s output.

22. Models of Computation  The TM on the next slide computes the correct parity bit to append to a binary number to achieve even parity.  Key design concepts : ◦ The TM needs to distinguish between whether there are an odd or even number of 1s in the binary number on the tape, so two states will be required for this – one indicating that an even number of 1s have been read so far, the other indicating an odd number. ◦ An additional state will be needed to write the correct parity bit, once the end of the binary number is reached. ◦ The TM should not change the bits already on the tape, so these will need to be rewritten to it after each transition.

23. Models of Computation  Input alphabet = { 0, 1 }  Blank symbol =   Tape alphabet = { 0, 1,  } δ (S E, 0) = (S E, 0, >) δ (S E, 1) = (S O, 1, >) δ (S E,  ) = (S W, 0, >) δ (S O, 0)= (S O, 0, >) δ (S O, 1) = (S E, 1, >) δ (S O,  ) = (S W, 1, >)  States = { S E, S O, S W }  Initial state = S E  Terminal states = { S W }  Transition function viewed as both an FSM and a mathematical function: Function notation: δ (Current State, Input Symbol) = (New State, Output Symbol, Head Direction) Diagram transition arrow notation: Input Symbol, Output Symbol, Head Direction

24. Models of Computation 1010  SESE 1010  SOSO 1010  SOSO 1010  SESE 1010  SESE 10100  SWSW Transition function reminder Processing input 1010: 1) 2) 3) 4) 5) 6) Correct parity bit (0) added.

25. Models of Computation 1101  SESE Transition function reminder Q9) Trace the execution of the TM with input 1101 1) View Solution

26. Models of Computation Q10) Design a TM to replace 0s in a string with as and 1s with bs, halting when the end of the string is reached. e.g. 010011 is transformed into ababb.  Define the alphabets.  Design the transition function as an FSM.  Rewrite the transition function as a mathematical function. View Solution

27. Models of Computation  We will develop a Turing machine to copy a binary number an arbitrary distance along a tape.  The number will be separated from the position that it is to be copied to by # symbols. should become: 10011### .. 10011###10011 

28. Models of Computation Design concepts:  Need to mark off symbols that have already been copied - can replace 0 with a and 1 with b as in previous exercise.  Will need to convert the as and bs back to 0s and 1s, as machine is copying the number so cannot destroy original.

29. Models of Computation Five states are required:  S S – The starting state, from which the digit to copy will be identified. The digit will then be replaced (0  a, 1  b) and the machine will enter either state S 0 if a 0 is being copied or S 1 if a 1 is being copied. When the end of the original number string is met, the terminal state S T should be entered.  S 0 – Copying a zero. A zero is being copied, the tape head should move right until a blank square if found, then output the zero and enter state S R.  S 1 – Copying a one, similar to state S 0.  S R – Returning to the left after copying a bit. Keep moving left until get to an a or b. Return this to its initial value (0 or 1) then enter state S S to copy the next digit.  S T – Terminal state. No transitions will need to be defined for this state.

30. Models of Computation δ (S S, 0) = (S 0, a, >)Need to copy 0 so replace with a and enter state S 0 δ (S S, 1) = (S 1, b, >)Need to copy 1 so replace with b and enter state S 1 δ (S S, #) = (S T, #, >)End of number reached, enter terminal state S T δ (S 0, x) = (S 0, x, >)for x  { 0,1,#} Move right until reach a blank square δ (S 0,  ) = (S R, 0, <)then write a 0 and enter state S R δ (S 1, x) = (S 1, x, >)for x  { 0,1,#} Move right until reach a blank square δ (S 1,  ) = (S R, 1, <)then write a 1 and enter state S R δ (S R, x) = (S R, x, <) for x  { 0,1,#} Return left without changing tape data δ (S R, a) = (S S, 0, >)until reach a or b, then change back to δ (S R, b) = (S S, 1, >)0/1 and enter state S S to copy next bit.  Input alphabet = { 0, 1, # }  Blank symbol =   Tape alphabet = { 0, 1, #, , a, b }  States = { S S, S 0, S 1, S R, S T }  Initial state = S S  Terminal states = { S T } Q11) Trace the execution of the TM on the input 10# View Solution

31. Models of Computation  We have seen that TMs can do simple tasks such as calculating parity bits and manipulating strings.  TMs can also be designed that do arithmetic.  But is the TM really a general model of a computer, that is as powerful as any computer than could be built? YES!

32. Models of Computation  A complex TM may have hundreds or thousands of states and a very complicated transition function.  As with writing programs in a high level programming language, the key to developing a complex TM is to combine simpler TMs together.  It can be proven that if there are two TMs which perform two jobs, they can be combined together to produce a single TM.  One TM can also effectively be used as a subroutine by another TM.

33. Models of Computation  Informally, the Church-Turing Thesis (also know as Church’s Hypothesis) states that: “Everything that can be computed can be computed by a Turing machine.”  The thesis cannot be proven, but is believed on the basis of the amount of evidence that supports it.  Computer scientists have devised several alternative models of computation, but all have proven to be equivalent to the Turing machine.

34. Models of Computation  The Church-Turing Thesis can also be viewed as offering a definition of what an algorithm is.  Since anything that can be computed can be computed by a TM, if there is a method to carry out a task (an algorithm) then there will be a TM that performs the same computation.  The Church-Turing Thesis could be invalidated if a quantum computer could be built.

35. Models of Computation  A Turing machine can be ‘built’ to carry out any task that is computable but we don’t build a new computer each time that we want to carry out a new task – we simply write a new program.  Is there a similar concept for Turing machines? Yes – the Universal Turing Machine (UTM)

36. Models of Computation In his 1936 paper*, Turing wrote: “It is possible to invent a single machine which can be used to compute any computable sequence. If this machine U is supplied with a tape on the beginning of which is written the S.D (the standard description of an action table) of some computing machine M, then U will compute the same sequence as M.” * On computable numbers, with an application the Entscheidungsproblem

37. Models of Computation  The essence of Turing’s UTM concept is that a description of a Turing machine (states, transition function etc.) could be written on the tape that also contains the data to be processed.  The UTM would process the description of the machine, applying it to the data on the tape.  The UTM is considered by many to be a precursor of the Von Neumann architecture, i.e. that instructions and data can be stored in the same memory.  The UTM can also be viewed as an interpreter.

38. Models of Computation  Formal models of computing are important because they provide a model of what can be computed, irrespective of what hardware or software is available now or may be available in the future.  The Church-Turing thesis states that anything that is computable can be computed by a Turing machine, so no computer can be more powerful that a Turing machine.  Turing machines can also be used to prove that some things cannot be computed – we will look at this in the session on complexity and computability.

39. Models of Computation End of Presentation

40. Models of Computation Design finite state automata to:  Q4) Accept any binary number that contains three or more 0s.  Q5) Accept any binary number that begins with 10.  Q6) Accept any binary number that is divisible by two (i.e. ends in 0). View Solutions

41. Models of Computation  Q7) Design a Moore machine to control a car interior light.  The light is controlled by a switch and sensors in the two front doors. o The switch can be in one of three positions: O, P or D. In position O the light is off, in position P the light is permanently on, and in position D the light is controlled by the doors. o If the switch is in position D then the light comes on if either left (L) or right (R) front doors are open. Question 7 continues on the next slide

42. Models of Computation  One way to design the machine is to deal with the switch first, then add in the operation of the doors.  Here is a machine that has three states, one for each switch position.  The machine currently leaves the light off in position D.  Now, extend the machine design to turn the light on and off correctly when door L or R is opened. View Solution

43. Models of Computation  Q8) Modify the design of the Mealy machine in the previous example so that it counts how often sequences of three of the same letter appear within a string. View Solution

44. Models of Computation  Q1) Is input string aba accepted? Stateab S0S0 S1S1 S2S2 S1S1 S1S1 S1S1 S2S2 S2S2 S3S3 S3S3 S2S2 S3S3 Transition Table State Transition Diagram StateInputNew State S0S0 aS1S1 S1S1 bS1S1 S1S1 aS1S1 StateInputNew State S0S0 bS2S2 S2S2 aS2S2 S2S2 aS2S2 S2S2 bS3S3 Definition of FA  Q2) Is input string baab accepted?  Q3) What is the purpose? Accepts strings beginning and ending in b (of minimum length 2).   Return to Slides Stateab S0S0 S1S1 S2S2 S1S1 S1S1 S1S1 S2S2 S2S2 S3S3 S3S3 S2S2 S3S3 InputNew State S0S0 aS1S1 S1S1 bS1S1 S1S1 aS1S1 StateInputNew State S0S0 bS2S2 S2S2 aS2S2 S2S2 aS2S2 S2S2 bS3S3  Q1) Is input string aba accepted?

45. Models of Computation  Q4) Accept any binary number that contains three or more 0s. Next Solution Key Ideas: (1)Each time a 0 is read, advance towards accepting state, until 3 read. (2)If a 1 is read, stay in the current state. (3)When reach accepting state, continue to accept further input.

46. Models of Computation  Q5) Accept any binary number that begins with 10. Key Ideas: (1)Must advance towards accepting state when read 1 then when read 0 straight after it. (2)If read a 0 or 1 when don’t want one, then should enter a state from which cannot exit – a ‘trapped’ state. (3)When in accepting state (S 3 ) or ‘trapped’ state (S 2 ) must continue to accept further input. Next Solution

47. Models of Computation  Q6) Accept any binary number that is divisible by two (i.e. ends in 0). Next Exercise Key Ideas: (1)Must enter the accepting state when a 0 is read. (2)Once in the accepting state, should stay in it if another 0 is read. (3)Should exit the accepting state if a 1 is read.

48. Models of Computation  Q7) Here is a complete design for the Moore machine: Next Exercise

49. Models of Computation  Q8) Here is a complete design for the Mealy machine: Start Turing Machines

50. Models of Computation 1101  SESE 1101  SOSO 1101  SESE 1101  SESE 1101  SOSO 11011  SWSW Transition function reminder Q9) Trace the execution of the TM with input 1101. 1) 2) 3) 4) 5) 6) Return to Slides

51. Models of Computation Q10) Design concepts:  The symbol to output to the tape depends only on the input symbol that is read, not anything that has happened previously, so the bulk of the processing can be done with one state which can also be the start state.  The machine needs to process a string and starts at its left hand end, so all tape head moves should be to the right.  The machine should stop when it reads the first blank symbol as this will indicate that the head has passed the end of the string. Continue With Solution

52. Models of Computation  Input alphabet = { 0, 1 }  Blank symbol =   Tape alphabet = { 0, 1, a, b,  } δ (S 0, 0) = (S 0, a, >) δ (S 0, 1) = (S 0, b, >) δ (S 0,  ) = (S 1, , >)  States = { S 0, S 1 }  Initial state = S 0  Terminal states = { S 1 }  Transition function viewed as both an FSM and a mathematical function: Return to Slides

53. Models of Computation 10# .. S 1) b0# .. S1S1 2) b0# .. S1S1 3) b0# .. S1S1 4) b0#1 .. SRSR 5) b0#1 .. SRSR 6) b0#1 .. SRSR 10#1  S 8) 1a#1 .. S0S0 9) 1a#1 .. S0S0 10) 1a#1 .. S0S0 11) 1a#10 .. SRSR 12) 1a#10 .. SRSR 1a#10  SRSR 14) 10#10 .. S 15) 10#10 .. STST 16) 7)13) Return to Slides Q11) Results of trace for input 10# :