1. Basic Concepts and Formal Language theory Unit –I By T.H. Gurav 8 October 2015

2. Elements of Formal Language Symbols Alphabets Strings Languages 8 October 2015

3. Alphabet An alphabet is a finite, non-empty set of symbols We use the symbol ∑ (sigma) to denote an alphabet Examples: Binary: ∑ = {0,1} All lower case letters: ∑ = {a,b,c,..z} Alphanumeric: ∑ = {a-z, A-Z, 0-9} DNA molecule letters: ∑ = {a,c,g,t} … 8 October 2015

4. Strings A string or word is a finite sequence of symbols chosen from ∑ Empty string is  (or “epsilon”) Length of a string w, denoted by “|w|”, is equal to the number of (non-  ) characters in the string E.g., x = 010100 |x| = 6 x = 01  0  1  00  |x| = ? xy = concatentation of two strings x and y 8 October 2015

5. Kleen Closure ( ∑* ) Given an alphabet ∑, we wish to define a language in which any string of letters from ∑ is a word/string, even null string. This language we shall call the closure of the alphabet. Positive Closure (∑ + ) 8 October 2015

6. Languages L is a said to be a language over alphabet ∑, only if L  ∑*  this is because ∑* is the set of all strings (of all possible length including 0) over the given alphabet ∑ Examples: 1. Let L be the language of all strings consisting of n 0’s followed by n 1’s: L = { ,01,0011,000111,…} 2. Let L be the language of all strings of with equal number of 0’s and 1’s: L = { ,01,10,0011,1100,0101,1010,1001,…} Definition:Ø denotes the Empty language Let L = {  }; Is L=Ø? NO 8 October 2015

7. Definition 1. A set is a group of objects ． The objects in a set are called the elements, or members, of the set. Example 1 The set of positive integers less than 100 can be denoted as {1,2,3,…99} Definition 2. Two sets are equal if and only if they have the same elements. Sets 8 October 2015

8. Cartesian Product The Cartesian product of two sets is defined as: A  B = {(a, b) | a  A  b  B} Example: A = {x, y}, B = {a, b, c} A  B = {(x, a), (x, b), (x, c), (y, a), (y, b), (y, c)} 8 October 2015

9. Set Operations Union: A  B = {x | x  A OR x  B} Example: A = {a, b}, B = {b, c, d} A  B = {a, b, c, d} Intersection: A  B = {x | x  A AND x  B} Example: A = {a, b}, B = {b, c, d} A  B = {b} 8 October 2015

10. Set Operations Two sets are called disjoint if their intersection is empty, that is, they share no elements: A  B =  The difference between two sets A and B contains exactly those elements of A that are not in B: A-B = {x | x  A  x  B} Example: A = {a, b}, B = {b, c, d}, A-B = {a} 8 October 2015

11. Set Operations The complement of a set A contains exactly those elements under consideration that are not in A: -A = U-A Example: U = N, B = {250, 251, 252, …} -B = {0, 1, 2, …, 248, 249} 8 October 2015

12. x y z A function x Not a function Functions and Relations Let A and B be the sets. A function f from A to B is an assignment of exactly one element of B to each element of A. We write f(a)=b if b is the unique element of B assigned by the function f to the element a of A. If f is a function from A to B, we write f : A  B. 8 October 2015

13. Relations Let A and B be the sets. A relation R from A to B is a subset of A X B From the defination, relation R is a set of pairs. If (a,b) ɛ R, we say a has a relation R with b, denoted as aRb. Functions can be consider as relations, but relations are more general than functions. 8 October 2015

14. R is an equivalence relation if for any pair (x,y) of R xRx for all x (reflexivity) If xRy then yRx (symmetry) If xRy and yRz, then xRz. (transitivity) We usually use to denote equivalence relation. Example : if A= {a,b} then Reflexive relation = {(a,a),(b,b)} Transitive relation = {(a,b),(b,a),(a,a)} Symmetric relation = {(a,b),(b,a)} Properties of Relation 8 October 2015

15. Transitive Closure (R + ) Reflexive and Transitive Closure(R*) of a relation R Eg: R ={(1,2),(2,3),(3,4),(5,4)} Solution : for R + we will compute R 2, R 3,R 4,…. R 2 = RoR = {(1,2),(2,3),(3,4),(5,4)} o {(1,2),(2,3),(3,4),(5,4)} = {(1,3),(2,4)} R 3 = R 2 oR = {(1,3),(2,4)}o{(1,2),(2,3),(3,4),(5,4)} = {(1,4)} R 4 = R 3 oR = {(1,4)}o{(1,2),(2,3),(3,4),(5,4)} =Ф R 5 = Ф 8 October 2015

16. R + = R U R 2 U R 3 {(1,2),(2,3),(3,4),(5,4)} {(1,3),(2,4)} {(1,4)} R + = {(1,2),(2,3),(3,4),(5,4)} U {(1,3),(2,4)} U {(1,4)} R + = {(1,2),(1,3),(1,4),(2,3),(2,4),(3,4),(5,4)} R* = R + U { (1,1),(2,2),(3,3),(4,4),(5,5)} = {(1,2),(1,3),(1,4),(2,3),(2,4),(3,4),(5,4)} U { (1,1),(2,2),(3,3),(4,4),(5,5)} ={(1,2),(1,3),(1,4),(2,3),(2,4),(3,4),(5,4),(1,1),(2,2),(3,3),(4,4),(5,5)} 8 October 2015

17. Example 1 v7 v1 v2 v5 v4 v9 v8 v3 e1 e3 e4 e5 e7 e6 e2 e8 e9 e10 A directed graph (digraph) Example 2 v7 v1 v2 v5 v4 v9 v8 v3 e1 e3 e4 e5 e7 e6 e2 e8 e9 e10 An undirected graph Graphs A Graph is a construct of two sets, denoted as G=(V,E) where V={v1,v2,v3…,vn} is a set of vertices and E={e1,e2,…em} is a set of edges. Each eage is pair from V 8 October 2015

18. Induction If we have a propositional function P(n), and we want to prove that P(n) is true for any natural number n, we do the following: 1. Show that P(0) is true. (basis step) 2. Show that if P(n) then P(n + 1) for any n  N. (inductive step) 3. Then P(n) must be true for any n  N. (conclusion) 8 October 2015

19. Induction 1 + 2 + … + n = n (n + 1)/2 1. Show that P(0) is true. (basis step) For n = 0 we get 0 = 0. True. 8 October 2015

20. Induction 2. Show that if P(n) then P(n + 1) for any n  N. (inductive step) 1 + 2 + … + n = n (n + 1)/2 1 + 2 + … + n + (n + 1) = n (n + 1)/2 + (n + 1) = (n + 1) (n/2 + 1) = (n + 1) (n + 2)/2 = (n + 1) ((n + 1) + 1)/2 8 October 2015

21. Induction 3. Then P(n) must be true for any n  N. (conclusion) 1 + 2 + … + n = n (n + 1)/2 is true for all n  N. End of proof. 8 October 2015

22. Introduction to Automata Theory 8 October 2015

23. Theory of computation Theory of Computation aims at studying “How and how efficiently problems can be solved on a model of computation, using an algorithm”.model of computationalgorithm The field is divided into three major branches: automata theory, automata theory Computability theory and Computability theory computational complexity theory. computational complexity theory 8 October 2015

24. Automata Automaton = an abstract computing device(Note: A “device” need not even be a physical hardware!)Automata theory is theory of models. Working of every process can be represented by means of model. 8 October 2015

25. Finite Automata states An automaton with a set of states, and its “control” moves from state to state in response to external “inputs” is called a finite automaton. FA, provides the simplest model of a computing device. It has a central processor of finite capacity 8 October 2015

26. Components of Finite Automata 8 October 2015 left-end x 1 x 2 x 3 x 4 x 5 ……. x n input: x.... Accept / Final state current state initial state Finite control (program) Movable tape head (Read Only) …. no right-end

27. Components of Finite Automata 8 October 2015 The components of FA model are Input tape, Read control and finite control. The input tape is divided into number of cells. Each cell can hold one i/p symbol. The read head reads one symbol at a time and moves ahead. Finite control acts like a CPU. Depending on the current state and input symbol read from the input tape it changes state.

28. Finite Automaton – Formal Definition An automaton is represented formally by a 5-tuple5-tuple (Q,Σ,δ,q 0,F), where: Q is a finite set of states. Σ is a finite set of symbols, called the alphabet of thesymbolsalphabet automaton. q 0 is the start state, that is, the state of the automaton before any input has been processed, where q 0 ∈ Q. F is a set of states of Q (i.e. F ⊆ Q) called accept states. δ is the transition function, that is, δ: Q × Σ → Q. For elements q, r  Q; a  Σ; the formula δ(q,a) = {r} specifies the next state (r) by M after reading symbol ‘a’ while in state q. 8 October 2015

29. How does an FA work? At the beginning, an FA is in the start state (initial state) its tape head points at the first cell For each move, FA reads the symbol under its tape head changes its state (according to the transition function) to the next state determined by the symbol read from the tape and its current state move its tape head to the right one cell 8 October 2015

30. When does an FA stop working? When it reads all symbols on the tape Then, it gives an answer if the input is in the specific language: Answer “YES” if its last state is a final state Answer “NO” if its last state is not a final state 8 October 2015

31. Alternative ways of FA representation 1. Transition Diagram Current State Input Symbol Next State 8 October 2015

32. Example- The circles indicates the states If accepting state is marked with double circle The arrows pointing from a state q indicates how to move on reading a character when current state is q 0,1 Startq0 q1 8 October 2015

33. Transition Table stateFor all Tape Symbols (a) qiqi q j Where: q i is the current state q j is the next state a is the current symbol

34. Acceptability of a string by FA A string ‘s’ is accepted by a FA if  (q0,x)=q for some qЄF. Eg : Q = { q0,q1,q2,q3} ∑ = {0,1} F = {q0} Transition function is given as : States i/p 0 1 q0 q2 q1 q1 q3 q0 q2 q0 q3 q3 q1 q2

35. Solution : Give entire sequence of states for the i/p string 110101  (q0,110101) =  (q1,10101) =  (q0,0101) =  (q2,101) =  (q3,01) =  (q1,1) =  (q0,^)=q0 Hence q0 1 q1 1 q0 0 q2 1 q3 0 q1 1 q0 Thus string is accepted. 8 October 2015

36. Extended Transition function to Strings It is slightly cumbersome to use a transition function for every i/p symbol. This problem can be resolved using a transition function for the whole string. This extended transition function is defined as δ :Q X ∑ *  Q 8 October 2015

37. Extended Transition function to Strings Language of DFA : DFA defines a language which contains all the strings that reach to the final state of DFA starting with the start state. If δ is transition function then δ is extended transition function to strings. δ gives resultant o/p state ‘q’ from state q0 by i/p of string w. δ is defined as : 1. δ(q0,ε)=q0; 2. δ(q,w)= δ(δ ( q,w 1 ),w n-1 ) where w=w 1 w 2 ….w n

38. Determinism and Non-determinism A Finite Automaton may be ‘Deterministic’ or ‘Non- Deterministic’:  Deterministic Automaton means that the machine cannot be in more state than one at any point of time.  Non-Deterministic Automaton means that the machine can be in two or more states at any point of time. 8 October 2015

39. DFA The FA is called as Deterministic FA,if there is only one path for a specific i/p from current state to next state. Definition: DFA is mathematically represented as a 5- uple(Q,∑,δ,q0, F) The transition function is a function in Q × ∑ → Q 1. a finite set of states (often denoted Q) 2. a finite set of symbols (alphabet) 3. a transition function that takes as argument a state and a symbol and returns a state (often denoted ) 4. a start state often denoted q0 5. a set of final or accepting states (often denoted F) We have q0 Є Q and F Є Q

40. Example : Password When does an automata accepts a word ? It reads a word and accepts it if it stops in an accepting state. Only word then is accepted q5 is stuck or dead state

41. NFA Similar to DFA Nondeterministic move On reading an input symbol, the automaton can choose to make a transition to one of selected states. Without reading any symbol, the automaton can choose to make a transition to one of selected states or not. 0 0 

42. NFA definition A nondeterministic finite automaton M is a five- tuple M = (Q, , , q 0, F), where: 1. Q is a finite set of states of M 2.  is the finite input alphabet of M 3.  : Q    2 Q (power set of Q), is the state transition function mapping a state-symbol pair to a subset of Q 4. q 0 is the start state of M 5. F  Q is the set of accepting states or final states of M

43. DFA vs NFA DFANFA 1For Every symbol of the alphabet, there is only one state transition in DFA 1We do not need to specify how does the NFA react according to some symbol. 2DFA can not use Empty String transition 2NFA can use Empty String transition. 3DFA can be understood as one machine. 3NFA can be understood as multiple little machines computing at the same time. 4DFA will reject the string if it end at other than accepting state. 4If all of the branches of NFA dies or rejects the string, we can say that NFA reject the string.

44. NFA to DFA conversion Subset Construction Subset Construction method : Given an NFA with states Q, inputs Σ, transition function δ N, state state q 0, and final states F, construct equivalent DFA with:  states 2 Q (Set of subsets of Q).  Inputs Σ.  Start state {q 0 }.  Final states = all those with a member of F.

45. Eg: Construct a DFA equivalent to N=({q0,q1},{0,1},δ,q0,{q0}) δ is given in following table: Solution : the states in DFA are subsets of {q0,q1} ie Ф,{q0},{q1},{q0,q1} Initial state : {q0} Final states : {q0} and {q0,q1}... As q0 present in these sets State01 q0 q1 q0,q1

46. δ can be constructed as bellow : Draw Transition diagram : State01 q0 q1 q0,q1 {q0,q1}

47. NFA with  -transitions NFA that allows the transition of an empty string from a state Jumping to a state is possible even without input Situation: strings from a lang with 1 st any no. of a’s then any no. of b’s and then any no. of c’s.

48. NFA with  -transitions A nondeterministic finite automaton with  -transitions (or  -NFA) is a five-tuple M = (Q, , , q 0, F), where: 1. Q is a finite set of states of M 2.  is the finite input alphabet of M 3.  : Q  (  +  )  power set of Q, is the state transition function mapping a state-symbol pair to a subset of Q 4. q 0 is the start state of M 5. F  Q is the set of accepting states or final states of M

49. Epsilon closure In an NFA M, let q  Q ECLOSE(q) represents all states r that can be reached from q using only  -transitions including itself. Recursive definition for ECLOSE o If  (q,  ) is empty, ECLOSE(q) = {q} o Else, Let  (q,  ) = {r 1, r 2,…, r n }. ECLOSE(q) =  ECLOSE(r i )  {q}

50. Example : find  -closure for the following NFA with .  Solution : 1.  -closure (q0) = {q0,q1,a2} means self state + reachable states. 2.  -closure(q1) = {q1,q2} 3.  -closure(q2)={q2} q0 q2 q1 ac b

51. Steps to convert  -NFA to NFA If  -NFA(Q, , , q 0, F) is given then we need to find NFA (Q, ,  ’’, q 0, F’). If  (0,1) then find  ’’(q,0) and  ’’(q,1) for all q ε Q. Steps to find  ’’(q,0) 1. Find  -closure of q. Outcome is set of states. Let that set is P 2. Find  (pi,0) for all pi ε P i>=1 3. Take union of all states obtained from step 2. 4. Find  -closure of all states obtained in step 3. Same steps are repeated for  ’’(q,1) for i/p alphabet 1.

52. Final states NFA M’ = (Q, ,  ’’, q 0, F’) such that F’ is described as follows F’ = F plus all states q such that ECLOSE(q) contains a state in F

53. Converting  -NFA into DFA Step1 : first convert  NFA into NFA Step 2: then convert NFA into DFA by sub- state construction method.

54. Minimization of FA The minimization of FA means reducing no. of states in FA. We need to find equivalent states and then represent them by one representative state. Find equivalent classes ( ie groups of equivalent states) Step 1: Divide FA’s states into two groups a. accepting group b. rejecting group Step 2: Find out if states in groups go to same group on given i/p or create new group. Step 3: Repeat step 2 till no new groups are created.

55. Finite Automaton Some Applications Software for designing and checking the behavior of digital circuits Lexical analyzer of a typical compiler Software for scanning large bodies of text (e.g., web pages) for pattern finding Software for verifying systems of all types that have a finite number of states (e.g., stock market transaction, communication/network protocol) 8 October 2015

56. Mealy and Moore Machines Theory of Computation

57. Specialized FA DFA used to recognize a language. Need: But computers often have to do more useful function of performing calculations and conveying results. Solution: Need m/c that have output capability.

58. Melay and Moore models Moore Machines : output depend only on the present state. Mealy Machines : output depends on the present state and the present input.

59. Primitive State Diagram, Moore state/ output input A/0 C/0D/0 B/1 etc. off on off

60. Definition of a Moore Machine It is a six tuple (Q,∑,Δ,δ,λ,q0) where 1. Q = A finite set of states 2. ∑ = Alphabet of input letters 3. Δ = Alphabet of output letters 4. δ = Transition function ∑ x Q into Q A unique one for each letter and each state 5. λ = Output Table. Maps Q into Δ 6. q0 = is the start state.

61. Primitive State Diagram, Mealy state input/output A CD B etc. off/1 on/0 off /0

62. TG for Mealy m/c Q Σ 01 Stateo/pStateo/p

63. Definition of a Mealy Machine It is a six tuple (Q,∑,Δ,δ,λ,q0) where all the symbols have the same meaning as Moore machine except λ. λ is the output function mapping ∑x Q into Δ

64. Interconversion Mealy to Moore Moore to mealy 8 October 2015

65. Thanking you… 8 October 2015