1 Chapter 1 Introduction to the Theory of Computation

2 Theory of Computation  Can we mathematically model the computation?  Automata Theory  Regular languages: DFA, NFA, Regular expressions  Context-free languages: Context-free grammar, pushdown automata  Turing machines: the most powerful model of computation  Can the computer solve all problems?  Computability Theory  Halting problem  How efficient the solution is?  Complexity Theory  Time and space complexity (Big O notation)

3 1.1 Mathematical Preliminaries Sets (1)  A set is a collection of objects  Order and repetition is irrelevant  X = {1, 2, 3}  X = {n | 1  n  3, n  N }  Y = { } =   Y is an empty set  The power set of X is the set of all subsets of X   (X) = 2 X = { Y | Y  X}  |  (X)| = 2 |X|  Example X = {1, 2, 3}  2 X =  (X)={{},{1},{2},{3},{1,2},{1,3},{2,3}, {1,2,3}}  {1,2}  Xis a proper subset of X  {1,2,3}  X is NOT a proper subset of X  {1,2,3} is a superset of {1, 2}

4 Sets (2)  The  symbol signifies membership (belong to):  x  X indicates that x is a member or element of the set X  x  X indicates that x is not a member or element of the set X  Assume X = {1, 2, 3}, then  1  X, but 4  X  Two sets are equal if they contain the same members (elements)  If X = {1, 2, 3}, Y = {1, 2, 3}, and Z = {2, 3, 4}  Then X = Y, but X  Z, and so Y  Z

5 Sets (3)  Assume X = {1,2,3} and Y = {2,4}  Union  X  Y = {z | z  X or z  Y} = {1,2,3,4}  Intersection  X  Y = {z | z  X and z  Y} = {2}  Difference  X - Y = {z | z  X and z  Y} = {1,3}

6 Sets (4)  Complement  Let X be a subset of Y  The complement of X ( X ) with respect to Y is the set of elements in Y but not in X  X = { z | z  Y and z  X } = Y - X  Assume X = {1,2,3} is a subset of Y = {1,2,3,4}  X = {4}  DeMorgan’s Laws  (X  Y) = X  Y  (X  Y) = X  Y

7 Inductive Proofs   Prove a statement S(X) about a family of objects X (e.g., integers, trees) in three parts:  Basis Step:  Prove for one or several small values of X directly  Assumption step :  Assume S(Y) for Y smaller than X  Prove S(X) using that assumption

8 Inductive Proofs (EX1)  Prove by induction that  S: ……+ n = n (n + 1) / 2  Basis step:  n = 1  1 ( 1 + 1) / 2 = 1  Assumption step:  Assume that S is true for all series of length < n  Prove that S is true for a series of length n?  The series of length n is  …+n-1+n  By the assumption 1+2+…+n-1 = (n-1)((n-1)+1)/2  Add n to the summation proves S.

9 Inductive Proofs (EX2)   Example  A complete binary tree with n leaves has 2n - 1 nodes

10 Inductive Proofs (EX2)   Basis:  If T has 1 leaf, it is a one-node tree. 1 = 2*1 - 1, so OK  Assumption:  Assume that the statement is true for all trees that have leaves less than n  Proof: Proof that the statement is true for trees that have n leaves  T has two subtrees, U and V  U has u leaves and V has v leaves  According to the assumption: U has (2u – 1) of nodes and V has (2v – 1) of nodes  T must be a root plus two subtrees U and V. The number of leaves in T equal number of leaves in U + number of leaves in V = u + v  The number of nodes in T = (2u – 1) + (2v – 1) + 1 = 2 ( u + v) - 1 

11 If-And-Only-If Proofs   Often, a statement we need to prove is of the form  “X if and only if Y”   We are then required to do two things:  Prove the if-part: Assume Y and prove X  Prove the only-if-part: Assume X, prove Y   Remember:  The if and only-if parts are converses of each other   One part, say “if X then Y” says nothing about whether Y is true when X is false.   An equivalent form to “if X then Y” is “if not Y then not X”; the latter is the contra-positive of the former.

12 Equivalence of Sets   Many important facts in language theory are of the form that two sets of strings, described in two different ways, are really the same set   To prove sets S and T are the same, prove:  x is in S if and only if x is in T. That is:  Assume x is in S; prove x is in T.  Assume x is in T ; prove x is in S.

13Functions  Let f be a function defined on a set A and taking values to a set B. Set A is called the domain of f; set B is called the range of f.  The arity of the functions  The number and type of input arguments  the number and type of output  Examples:  The arity of the Multiplication: Num × Num  Num  The arity of the SquareRoot : Num  Num

14 1.2:Strings and Languages (1)  Alphabet: Finite, nonempty set of symbols  Examples:   = {0, 1}: binary alphabet   = {a, b, c, …, z}: the set of all lower case letters  The set of all ASCII characters  String: Finite sequence of symbols from an alphabet   Examples:  where  = {0, 1}  abracadabra where  = {a, b, c, …, z}

15 1.2:Strings and Languages (2)  Empty String: The string with zero occurrences of symbols from  and is denoted  or  Empty String: The string with zero occurrences of symbols from  and is denoted  or  Length of String: Number of symbols in the string  The length of a string w is usually written |w|  |1010| = 4  |  | = 0  |uv| = |u| + |v|  Reverse : w R  If w = abc, w R = cba

16 1.2:Strings and Languages (3)  Concatenation: if x and y are strings, then xy is the string obtained by placing a copy of y immediately after a copy of x  x = a 1 a 2 …a i, y = b 1 b 2 …b j  xy = a 1 a 2 …a i b 1 b 2 …b j  Example: x = 01101, y = 110, xy =  x  =  x = x

17 1.2:Strings and Languages (4)  Power of an Alphabet:  k = the set of strings of length k with symbols from   Example:  = {0, 1}   1 =  = {0, 1}   2 = {00, 01, 10, 11}   0 =   Question: How many strings are there in  3 ?  The set of all strings over  is denoted  *   * =  0   1   2   3  …  Also   + =  1   2   3  …   * =  +     + =  * - 

18 1.2:Strings and Languages (5)  Substring: any string of consecutive characters in some string w  If w = abc  , a, ab, abc are substrings of w  Prefix and suffix:  if w = vu  v is a prefix of w  u is a suffix of w  Example  If w = abc  a, ab, abc are prefixes of w  c, bc, abc are suffixes of w

19 1.2:Strings and Languages (6)   Suppose: S is the string banana  Prefix : ban, banana  Suffix : ana, banana  Substring : nan, ban, ana, banana

20 1.2:Strings and Languages (8)  Language: set of strings chosen from some alphabet  A language is a subset of  *  Example of languages:  The set of valid Arabic words  The set of strings consisting of n 0’s followed by n 1’s –{ , 01, 0011, , …}  The set of strings with equal number of 0’s and 1’s –{ , 01, 10, 0011, 0101, 1010, 1001, 1100, …}  Empty language:  = { }  The language  consisting of the empty string  Note:  

21 1.2:Strings and Languages (9)  Can concatenate languages  L 1 L 2 = {xy | x  L 1, y  L 2 }  L n = L concatenated with itself n times  L 0 = {  }; L 1 = L  Star-closure  L * = L 0  L 1  L 2   L + = L * - L 0  Languages can be finite or infinite  L = {a, aba, bba}  L = {a n | n > 0}

22 1.2:Strings and Languages (10) OPERATIONDEFINITION union of L and M written L  M concatenation of L and M written LM Kleene closure of L written L* positive closure of L written L + L  M = {s | s is in L or s is in M} LM = {st | s is in L and t is in M} L + =  L i, i=1,..,  L* denotes “zero or more concatenations of “ L L* =  L i, i=0,..,  L + denotes “one or more concatenations of “ L L + = L L *

23 1.2:Strings and Languages (11) L = {A, B, …, Z, a, b, …z} D = {1, 2, …, 9} L  D = the set of letters and digits LD = all strings consisting of a letter followed by a digit L 2 = the set of all two-letter strings L 4 = L 2 L 2 = the set of all four-letter strings L* = { All possible strings of L plus  }, L + = L* -  D + = set of strings of one or more digits L (L  D ) = set of all strings consisting of a letter followed by a a letter or a digit L (L  D )* = set of all strings consisting of letters and digits beginning with a letter

24 1.2:Strings and Languages (12)  The language L consists of strings over {a,b} in which each string begins with an a should have an even length  aa, ab  L  aaaa,aaab,aaba,aabb,abaa,abab,abba,abbb  L  baa  L  a  L

25 1.2:Strings and Languages (13)  The language L consists of strings over {a,b} in which each occurring of b is immediately preceded by an a      L  a  L  abaab  L  bb  L  bab  L  abb  L

26 1.2:Strings and Languages (14)  Let X = {a,b,c} and Y = {abb, ba}. Then  XY = {aabb, babb, cabb, aba, bba, cba}   X 0 = {  }  X 1 = X = {a,b,c}  X 2 = XX = {aa,ab,ac,ba,bb,bc,ca,cb,cc}  X 3 = XXX = {aaa,aab,aac,aba,abb,abc,aca,acb,acc,baa,bab,ba c,bba,bbb,bbc,bca,bcb,bcc,caa,cab,cac,cba,cbb, cbc,cca,ccb,ccc}

27 1.2:Strings and Languages (15)  The language L = {a,b} * {bb}{a,b} * =  * {bb}  *  consists of the strings over {a,b} that contain the substring bb  bb  L  abb  L  bbb  L  aabb  L  bbaaa  L  bbabba  L  abab  L  bab  L  b  L

28 1.2:Strings and Languages (16)  Let L be the language that consists of all strings that begin with aa or end with bb  L 1 = {aa}{a,b} *  L 2 = {a,b} * {bb}  L = L 1  L 2 = {aa}{a,b} *  {a,b} * {bb}  bb  L  abb  L  bbb  L  aabb  L  bbaaa  L  bbabba  L  abab  L  bab  L  ba  L

29 1.2:Strings and Languages (16)  Let L 1 = {bb} and L 2 = {, bb, bbbb} over b  Let L 1 = {bb} and L 2 = { , bb, bbbb} over b  The languages L 1 * and L 2 * both contain precisely the strings consisting of an even number of b’s.  , with length zero, is an element of L 1 * and L 2 *

30 1.2:Strings and Languages (17)  What is the language of all even-length strings over {a,b}  L = {aa, bb, ab, ba} * = (aa|bb|ab|ba) *  What is the language of all odd-length strings over {a,b}  L = {a,b} * - {aa, bb, ab, ba} * or  L = {a,b}{aa, bb, ab, ba} * or  L = {aa, bb, ab, ba} * {a,b}