1. Finite-State Machines 15-211 Fundamental Data Structures and Algorithms Peter Lee March 11, 2003

3. Announcements  Homework #4 is due next week!  Monday, March 17, 11:59pm  Reading:  Handout  Midterm Exam has been graded  Get your exam at recitation

4. Models of Computation

5. Models of computation  Understanding computation – 1.What problems can be solved by computers? 2.What problems cannot? 3.What problems can be solved “quickly”? 4.Are there fundamental differences among computers with respect to the kinds of problems they can solve? (Ignoring space and time.)

6. Models of computation  What details of these machines do we need to understand in order to understand computation?

7. Models of computation  What details about these computers can be ignored in answering these questions?  In other words:  What can abstracted away?  abstrahere = to drag away

8. Time, code, space

9. Computability theory 1.What problems can be solved by computers? 2.Are there well-stated mathematical problems that cannot be solved by computers?

10. Computability theory  The Halting Problem:  Can a program be written that, given another program and its input, determines if that program will terminate when executed?  Can the Java compiler be enhanced to check whether programs will always terminate?

11. Computability theory  Note:  We can prove termination (or show a counterexample) for most programs we write.  The question is whether this can be done, in an automatic way, for all programs.

12. Computability theory 1.Which problems can be solved by computers? 2.Are there well-stated mathematical problems that cannot be solved by computers?

13. Computability theory  The Halting Problem is undecidable.  Turing, Church (1936)  This means that it is not possible to write a program to solve the Halting Problem.  Proofs of such statements are made on simplified mathematical models of computation

14. Other questions  Performance of algorithms  Model is surrogate for “computer time”  Which operations get counted?  What are their relative costs?  Does it matter? Why do we use big-O?  Correctness of algorithms and code  How can we be sure the programs does what it is intended to do?  If it fails, what is broken?  Computer?  Program?  Compiler?  OS?

15. Examples of general computational models Examples of general models of computation  CPU + Memory  Random access machine  CPU + “Tape”  Turing machine (1936)  CPU + Many counters  Minsky machine (1960s)  Functions of functions  Lambda calculus  String matching systems  Post production systems  Function programs  General recursive functions General purpose computers  laptop  cell phone  palm device  calculator ?

16.  Abstractly Infinite tape Finite control  Concretely The Turing Machine

17.  Abstractly Infinite 2D array Finite control  Concretely The Random Access Machine

18. The general models  These general models are essentially equivalent.  Each model can solve the same problems as any other model.  This is shown by simulating one model with another.  Example: simulating the Turing machine tape with a random access memory.

19. The general models  Anything that is felt to be effectively computable can be described within one of the formal models.  Church’s Thesis (1930s)

20. Simpler models But:  There are some simpler models that cannot compute all computable functions M 1 < M 2 < … < Computable  One class of simple models: Finite State Machines Finite control

21. FSMs

22. The FSM model  A finite-state machine is a computing machine that  takes a string as input, and  outputs a yes/no answer  When an FSM outputs “yes” for a particular input string, we say that it “accepts” or “recognizes” it Input String FSM {Yes, No}

23. The FSM model  Inputs:  Strings built from a fixed alphabet Alphabet: {a,b,c} Strings: aabbacabcaa aba b aaac  Machine: A directed graph  Nodes: states of the machine  Edges: transitions from one state to another  Determined by the next input character 1 0 a b

24. The FSM model  Special states  Start state  Final or accepting state Input String FSM {Yes, No} 1 2 0 aa bba,b

25. FSM Example  Input alphabet{a, b}  States{q0, q1, q2}  Start stateq0  Final states{q2}  Transitions{ (q0,a)  q1, … } 1 2 0 aa bba,b Which strings of as and bs are accepted?

26. Finite State Machines (FSMs)  Key limitation: memory size is fixed  Independent of input size.  No infinite tape or random-access memory Input String FSM {Yes, No}

27. What is not finite state?  There are simple string languages that cannot be recognized using FSMs  Examples (w is any string)  ww  ww R  Hint: How much memory is needed to recognize ww?  Hint: How can ww R be recognized?  Other computational models  finite state < [other models] < computable

28. Why study FSMs  A common algorithm design technique  Tokenization (“lexical analysis”)  Programs, web pages, text, …  Control systems  Elevators, boilers, soda machines, …  Computational model  Linear time stream processing  Simple and elegant

29. History  Origin: Neuronal models  McCulloch and Pitts (1943)  Reworked into finite state machines  Kleene (1954)  Related to Markov chains

30. The soda machine – concretely

31. The soda machine – abstractly  Input alphabet  $0.25  Dispense  Return-coin  States  Ready-for-coins  25  50  Empty ……

32. FSM Example  Which strings of as and bs are accepted?  Input alphabet{a, b}  States{q0, q1, q2}  Start stateq0  Final states{q2} 1 2 0 a a b bb

33. Deterministic FSMs (aka, DFAs)   Input alphabet  S State set  q0  S Initial state  F  S Final states  :S    S Transition function M = (, S, q0, F, )

34. Transition functions  Given a state and the next input symbol, give us the next state. :S    S This is normally extended to ’:S  *  S

35. Transition functions  Inductively: ’(q, ) = q ’(q, aw) = ’((q, a), w) the empty string input symbol remainder of input string

36. Formal languages  The language accepted by M: L(M) = {w | ’(q0,w)  F} This is the set of all strings accepted by M

37. Formal languages  What is the language accepted by this DFA? 1 2 0 a a b bb

38. Enhancing the model  Nondeterminism  Allow the machine to be in more than one state at once  Equivalently, allow more than one transition on any given input symbol  Enable the machine to “guess” which transition is the correct one

39. Making the NFA model precise   Input alphabet  S State set  q0  S Initial state  F  S Final states  :S    (S) Transition function M = (, S, q0, F, )

40. The language of the NFA  Extend :S    P(S) to ’:S  *  P(S) ’(q, ) = {q} ’(q, aw) =  p(q, a) ’(p, w)  Language accepted by M  L(M) = {w | ’(q0,w)  F  }

41. The language of the NFA  Enhance :S    P(S) to ’:S  *  P(S) to ’’:P(S)  *  P(S) ’’({q1,…qn}, w) =  i ’(qi, w)

42. Formal languages  What is the language accepted by this NFA? 1 2 0 a b a, b

43. Quiz Break

44. Red-green quiz AB CD x1x1 x2x2 x3x3 Consider this marble toy. 1) Model it as a FSM: marble in A is 0 input marble in B is 1 input sequence of marbles is accepted if the last marble comes out of D 2) What is the language accepted by this toy?

45. A Big Question  Are there languages L that can be accepted by an NFA but no DFA?

46. Another extension  Add  transitions  The “” represents “empty”, so a  transition means a transition without consuming any input.

47. Big Questions 1. Are there languages L that can be accepted by NFAs but not DFAs? 2. Are there languages L that can be accepted by NFAs but not NFAs?

48. Big Questions 1. Are there languages L that can be accepted by NFAs but not DFAs? No! 2. Are there languages L that can be accepted by NFAs but not NFAs? No!

49. Proving DFA  NFA  DFA  NFA   NFA  DFA 

50. Proving DFA  NFA  DFA  NFA  Obvious  NFA  DFA 

51. The Idea  An NFA can be in more than one state at a time  Define a DFA whose states correspond to combinations of the NFA states  In the NFA, let N = |S|  How many states will the corresponding DFA have?

52. Doing it  In the NFA  Suppose S = {0,1,2}  In the DFA, construct  S = {[],[0],[1],[2],[01],[02],[12],[012]}

53. Doing it: the subset construction  Both machines have the same input alphabet  In the NFA  Suppose S = {0,1,2}  3 states  In the DFA, construct  S = {[],[0],[1],[2],[01],[02],[12],[012]}  8 states  Modeling all possible combinations of the 3 NFA states

54. Doing it  In the NFA  Suppose S = {0,1,2}  In the DFA, construct  S = {[],[0],[1],[2],[01],[02],[12],[012]}  In the NFA  Suppose ({0}, a) = {0,1}  In the DFA, construct  ([0], a) = [01]

55. Doing it  In the NFA  Suppose S = {0,1,2}  In the DFA, construct  S = {[],[0],[1],[2],[01],[02],[12],[012]}  In the NFA  Suppose ({0}, a) = {0,1}  Suppose ({0,1}, b) = {0,2}  In the DFA, construct  ([0], a) = [01]  ([01], b) = [02]

56. Doing it  In the DFA, construct  Initial state [0]  In the DFA, construct  Final states [2], [02], [12], [012]

57. Completing the proof  Show that the two machines accept the same language Next time

58. Big Question, revisited  Are there languages L that can be accepted by NFAs but not DFAs? No! We sketched the proof of this by showing how, given an NFA, a DFA can be constructed that accepts the same languages

59. Big Questions, revisited 1. Are there languages L that can be accepted by NFAs but not DFAs? No! We sketched the proof of this by showing how, given an NFA, a DFA can be constructed that accepts the same languages 2. Are there languages L that can be accepted by NFAs but not NFAs? No! Similar proof technique…

60. Summary  Introduction to some big concepts  models of computation  finite state machines  deterministic and nondeterministic automata  formal languages  Next time:  what can we do with FSMs?