1. January 7, 2015CS21 Lecture 21 CS21 Decidability and Tractability Lecture 2 January 7, 2015

2. CS21 Lecture 22 Outline Finite Automata Nondeterministic Finite Automata Closure under regular operations NFA, FA equivalence

3. January 7, 2015CS21 Lecture 23 What is computation? the set of strings that lead to “accept” is the language recognized by this machine if every other string leads to “reject”, then this language is decided by the machine machine input accept reject loop forever

4. January 7, 2015CS21 Lecture 24 Terminology finite alphabet Σ : a set of symbols language L  Σ * : subset of strings over Σ a machine takes an input string and either –accepts, rejects, or –loops forever a machine recognizes the set of strings that lead to accept a machine decides a language L if it accepts x  L and rejects x  L

5. January 7, 2015CS21 Lecture 25 What goes inside the box? We want the simplest mathematical formalization of computation possible. Strategy: –endow box with a feature of computation –try to characterize the languages decided –identify language we “know” real computers can decide that machine cannot –add new feature to overcome limits machine input accept reject loop forever

6. January 7, 2015CS21 Lecture 26 Finite Automata simple model of computation reads input from left to right, one symbol at a time maintains state: information about what seen so far (“memory”) –finite automaton has finite # of states: cannot remember more things for longer inputs 2 ways to describe: by diagram, or formally

7. January 7, 2015CS21 Lecture 27 FA diagrams read input one symbol at a time; follow arrows; accept if end in accept state states (single) start state (several) accept states 1 alphabet Σ = {0,1} 0 0,1 transition for each symbol

8. January 7, 2015CS21 Lecture 28 FA operation Example of FA operation: 1 0 0,1 input:0101 not accepted

9. January 7, 2015CS21 Lecture 29 FA operation Example of FA operation: 1 0 0,1 input:101 accepted What language does this FA recognize? L = {x : x  {0,1}*, x 1 = 1}

10. January 7, 2015CS21 Lecture 210 Example FA What language does this FA recognize? L = {x : x  {0,1}*, x has even # of 1s} illustrates fundamental feature/limitation of FA: –“tiny” memory –in this example only “remembers” 1 bit of info. 1 0 0 1 oddeven

11. January 7, 2015CS21 Lecture 211 Try: AC CBCC AA BBBBBBB CCBC Example FA Σ = {A,B,C} 51015 203025 35 rej A C B C B A A C B A,B,C C B AC B A B A,C B C A “35 cents” {Q,N,D} A,B,C

12. January 7, 2015CS21 Lecture 212 FA formal definition A finite automaton is a 5-tuple (Q, Σ, δ, q 0, F) –Q is a finite set called the states –Σ is a finite set called the alphabet –δ:Q x Σ → Q is a function called the transition function –q 0 is an element of Q called the start state –F is a subset of Q called the accept states

13. January 7, 2015CS21 Lecture 213 FA formal definition Specification of this FA in formal terms: –Q = {even, odd} –Σ = {0,1} –q 0 = even –F = {even} 1 0 0 1 oddeven function δ: δ(even, 0) = even δ(even, 1) = odd δ(odd, 0) = odd δ(odd, 1) = even

14. January 7, 2015CS21 Lecture 214 Formal description of FA operation finite automaton M = (Q, Σ, δ, q 0, F) accepts a string w = w 1 w 2 w 3 …w n  Σ* if  sequence r 0,r 1,r 2,…,r n of states for which –r 0 = q 0 –δ(r i, w i+1 ) = r i+1 for i = 0,1,2, …, n-1 –r n  F

15. January 7, 2015CS21 Lecture 215 What now? We have a model of computation (Maybe this is it. Maybe everything we can do with real computers we can do with FA…) try to characterize the languages FAs can recognize –investigate closure under certain operations show that some languages not of this type

16. January 7, 2015CS21 Lecture 216 Characterizing FA languages We will show that the set of languages recognized by FA is closed under: –union “C = (A  B)” –concatenation “C = (A  B)” –star “ C = A* ” Meaning: if A and B are languages recognized by a FA, then C is a language recognized by a FA

17. January 7, 2015CS21 Lecture 217 Characterizing FA languages union “C = (A  B)” (A  B) = {x : x  A or x  B or both} concatenation “C = (A  B)” (A  B) = {xy : x  A and y  B} star “ C = A* ” (note: ε always in A*) A* = {x 1 x 2 x 3 …x k : k ≥ 0 and each x i  A}

18. January 7, 2015CS21 Lecture 218 Concatenation attempt (A  B) = {xy : x  A and y  B} AB What label do we put on the new transitions?

19. January 7, 2015CS21 Lecture 219 Concatenation attempt Need it to happen “for free”: label with ε (?) allows construct with multiple transitions with the same label (!?) 1 1 ε = 11 AB

20. January 7, 2015CS21 Lecture 220 Nondeterministic FA We will make life easier by describing an additional feature (nondeterminism) that helps us to “program” FAs We will prove that FAs with this new feature can be simulated by ordinary FA –same spirit as programming constructs like procedures The concept of nondeterminism has a significant role in TCS and this course.

21. January 7, 2015CS21 Lecture 221 NFA diagrams states (single) start state (several) accept states transitions: may have several with a given label (or none) may be labeled with ε At each step, several choices for next state

22. January 7, 2015CS21 Lecture 222 NFA operation Example of NFA operation: 0,ε11 0,1 alphabet Σ = {0,1} input:010 not accepted

23. January 7, 2015CS21 Lecture 223 NFA operation Example of NFA operation: 0,ε11 0,1 alphabet Σ = {0,1} input:110 accepted

24. January 7, 2015CS21 Lecture 224 NFA operation One way to think of NFA operation: string x = x 1 x 2 x 3 …x n accepted if and only if –there exists a way of inserting ε’s into x x 1 εεx 2 x 3 …εx n –so that there exists a path of transitions from the start state to an accept state

25. January 7, 2015CS21 Lecture 225 NFA formal definition A nondeterministic FA is a 5-tuple (Q, Σ, δ, q 0, F) –Q is a finite set called the states –Σ is a finite set called the alphabet –δ:Q x (Σ  {ε}) →  (Q) is a function called the transition function –q 0 is an element of Q called the start state –F is a subset of Q called the accept states transitions labeled with alphabet symbols or ε “powerset of Q”: the set of all subsets of Q

26. January 7, 2015CS21 Lecture 226 NFA formal definition Specification of this NFA in formal terms: –Q = {s 1, s 2, s 3, s 4 } –Σ = {0,1} –q 0 = s 1 –F = {s 4 } 0,ε11 0,1 s1s1 s2s2 s3s3 s4s4 δ(s 1, 0) = {s 1 } δ(s 1, 1) = {s 1, s 2 } δ(s 1, ε) = { } δ(s 2, 0) = {s 3 } δ(s 2, 1) = { } δ(s 2, ε) = {s 3 } δ(s 3, 0) = { } δ(s 3, 1) = {s 4 } δ(s 3, ε) = { } δ(s 4, 0) = {s 4 } δ(s 4, 1) = {s 4 } δ(s 4, ε) = { }

27. January 7, 2015CS21 Lecture 227 Formal description of NFA operation NFA M = (Q, Σ, δ, q 0, F) accepts a string w = w 1 w 2 w 3 …w n  Σ* if w can be written (by inserting ε’s) as: y = y 1 y 2 y 3 …y m  (Σ  {ε})* and  sequence r 0,r 1,…,r m of states for which –r 0 = q 0 –r i+1  δ(r i, y i+1 ) for i = 0,1,2, …, m-1 –r m  F

28. January 7, 2015CS21 Lecture 228 Closures Recall: want to show the set of languages recognized by NFA is closed under: –union “C = (A  B)” –concatenation “C = (A  B)” –star “ C = A* ”

29. January 7, 2015CS21 Lecture 229 Closure under union C = (A  B) = {x : x  A or x  B} A B A B ε ε C

30. January 7, 2015CS21 Lecture 230 Closure under concatenation C = (A  B) = {xy : x  A and y  B} AB AB ε ε C

31. January 7, 2015CS21 Lecture 231 Closure under star C = A* = {x 1 x 2 x 3 …x k : k ≥ 0 and each x i  A} A A ε ε ε C

32. January 7, 2015CS21 Lecture 232 NFA, FA equivalence Theorem: a language L is recognized by a FA if and only if L is recognized by a NFA. Must prove two directions: (  ) L is recognized by a FA implies L is recognized by a NFA. (  ) L is recognized by a NFA implies L is recognized by a FA. (usually one is easy, the other more difficult)

33. January 7, 2015CS21 Lecture 233 NFA, FA equivalence (  ) L is recognized by a FA implies L is recognized by a NFA Proof: a finite automaton is a nondeterministic finite automaton that happens to have no ε-transitions, and for which each state has exactly one outgoing transition for each symbol.

34. January 7, 2015CS21 Lecture 234 NFA, FA equivalence (  ) L is recognized by a NFA implies L is recognized by a FA. Proof: we will build a FA that simulates the NFA (and thus recognizes the same language). –alphabet will be the same –what are the states of the FA?

35. January 7, 2015CS21 Lecture 235 NFA, FA equivalence –given NFAM = (Q, Σ, δ, q 0, F) –construct FAM’ = (Q’, Σ’, δ’, q 0 ’, F’) –same alphabet: Σ’ = Σ –states are subsets of M’s states: Q’ =  (Q) –if we are in state R  Q’ and we read symbol a  Σ’, what is the new state? 0,ε11 0,1

36. January 7, 2015CS21 Lecture 236 NFA, FA equivalence –given NFAM = (Q, Σ, δ, q 0, F) –construct FAM’ = (Q’, Σ’, δ’, q 0 ’, F’) Helpful def’n: E(S) = {q  Q : q reachable from S by traveling along 0 or more ε-transitions} –new transition fn: δ’(R, a) =  r  R E(δ(r, a)) = “all nodes reachable from R by following an a-transition, and then 0 or more ε-transitions” 0,ε11 0,1

37. January 7, 2015CS21 Lecture 237 NFA, FA equivalence –given NFAM = (Q, Σ, δ, q 0, F) –construct FAM’ = (Q’, Σ’, δ’, q 0 ’, F’) –new start state: q 0 ’ = E({q 0 }) –new accept states: F’ = {R  Q’ : R contains an accept state of M) 0,ε11 0,1

38. January 7, 2015CS21 Lecture 238 NFA, FA equivalence We have proved (  ) by construction. Formally we should also prove that the construction works, by induction on the number of steps of the computation. –at each step, the state of the FA M’ is exactly the set of reachable states of the NFA M…