1. CSCI 4325 / 6339 Theory of Computation Chapter One Zhixiang Chen Department of Computer Science University of Texas-Pan American

2. Background Experience in at least one high-level programming language Data structures Discrete math Others …

3. Topics What is computation?  Or what is an algorithm? Formal languagesand computing devices?  Computation vs. Formal language understanding? What can be computed?  Or what cannot be computed? What can be efficiently computed?  Or what cannot be efficiently computed?

4. Reading Materials Chapter 0, Introduction

5. The World of Theoretical Foundations of Computation … PARTIALLY COMPUTABLE COMPUTABLE PSPACE NEXP EXP NP P

6. CHAPTER0 Introduction

7. Automata, Computability, and Complexity Want to know  What are the fundamental capabilities and limitations of computers?

8. Complexity What makes some problem computationally hard and others easy? How to classify problems according to their computational difficulties? How to understand the computational difficulties in the worst case, randomized, or approximate settings? How to harness the computational difficulties for applications like cryptography?

9. Computability What are limitations of Computers? Want to know what problems are solvable by computers, and what are not? Note: In complexity, we are interested in “solvable” problem, but want to know how difficult to solve.

10. Automata Automata theory deals with the definitions of properties of mathematical models of computation. These models play a role in several applied areas of computer science.  Finite automata = regular expression: for text processing  Context-free grammar: for programming language design and implementation, AI, natural language processing  Context-sensitive grammar: for natural language processing

11. Sets What is a Set?  A set is a collection of objects Ex’s? Notes:  We use capital letters to denote sets  An object contained in a set is called a set element. We use lower- case letters to denote set elements. Notations  Given a set A and an object x, if x is an element of A, we write  Otherwise we write

12. Finite Sets and Infinite Sets A set is finite if it has a finite number of elements  Ex’s? A set is infinite if it has an infinite number of elements  Ex’s?

13. Set Cardinality Given a set A, the cardinality of A, denoted by |A|, is the number of elements in A. Ex’s?

14. Empty set The empty set is a set without elements. The empty is denoted as Ø.

15. Set Representation By enumeration:  List all elements. Only good for finite sets  Ex’s. A = {1, 2, 3, 4, 5},B = {a, b, c, d, e} C = { Joe, Jose, Chen_1, Chen_ 2} D = { Black_Cat, White_Cat} By Property:  Give property for set elements. Good for infinite sets (and finite sets as well)  Ex’s. A = {x: x is an integer} B = {y: y is a student at UTPA} C = {z: z is a student in the CS department} D = {w: and w is even}

16. Set Operations Union  A U B is a set of elements in either A or B  A U B = {x: } Intersection  is a set of elements in both A and B.  Difference  A – B is a set of elements in A but not in B. 

17. Set Operations Subset  if any implies Proper Subset  if and Equivalence  A = B if Want example’s?

18. Properties of Set Operation Idempotency  A U A = A Commutativity  Associativity 

19. Properties of Set Operation Distributivity  Absorption  DeMorgan’s Law 

20. Can You See DeMorgan’s Law? A B C

21. Disjoint Sets and Power Sets DisjointSets  A and B are disjoint if Power Sets  Given a set A, the power set of A, denoted by is the collection of all subsets of A.   Ex’s. A = {1, 2, 3} = {, {1}, {2}, {3}, {1, 2}, {1, 3}, {2, 3}, {1, 2, 3}}

22. Two Notations Let S be a set such that any element in S is also a set, define  Ex’s  What is ?

23. Set Partition Given a nonempty set A, we say is a partition of A if   Any two elements of are disjoint.

24. Set Partition Examples Ex’s  Let S = {a, b, c, d}  What partition of S has the fewest members? The most members?  List all partitions of S with exactly two members?

25. Relations and Functions - Ordered Pairs Given any two objects a, b, let (a, b) denote the ordered pair of a and b. The ordered pair of a and b tells that a and b is related under some relationship The differences between (a, b) and {a, b}:  (a, b) (b, a), while {a, b} = {b, a}  (a, a) is a good ordered pair, while {a, a} is not. The equivalence  (a, b) = (c, d)if and only if a = c, b = d.

26. Ordered Tuples Given any n objects  is an n-ary ordered tuple. Note:  The objects may not be distinct. Ordered tuple examples?

27. Cartesian Products Given two sets A and B, the Cartesian Product of A and B is  Given n sets, the n-fold Cartesian Product of these sets is  Ex’s?

28. Relations A binary relation is defined as  A n-ary relation is defined as 

29. Functions A function is a binary relation on A and B such that  there is exactly one such that  A is called the domain of f. For any, we denote y as f(x). f(x) is called the image (or value) of x. The range of f is defined as  f [A] = { y: y = f(x) for some }

30. Function Mapping AB f

31. One-to-One, Onto, Bijection One-to-One  A function is one-to-one if for any two distinct elements Onto  A function is onto B, if  Bijection  A function is a bijection if f is one-to-one and onto

32. Inverse The inverse of a binary relation is  The inverse of a function may not be a function any more.  Why? If a function is bijection then its inverse is still a function.  Why?

33. Composition Let Q and R to be two relations  Then RQ is also a relation  AB C QR

34. Composition Examples Let R = {(a, b), (a, c), (c, d), (a, a), (b, a)},  What is RR?  What is ?  Is R, RR or a function?

35. Graph Representation of a Binary Relation Given a binary relation for each  draw a node to represent a,  draw another node to represent b,  draw an edge from a to b.  Then, we obtain a directed graph for R.

36. An Graph Representation Example Let R = {(a, b), (b, a), (a, d), (d, c), (c, c), (c, a)}. The Graph is a b c d

37. Reflexive A relation R: A→A is reflexive,  if for any For a reflexive relation R, the directed graph for R has a self-circle for each node Graph example? Other examples?  The less-than-or-equal-to relation over integers

38. Symmetric A relation R: A→A is symmetric  if for any In the directed graph of a symmetric relation, if there is an edge from node a to node b, then there must be another edge from node b to node a. Graph examples? Other Examples?  The friend relation over a group of people.

39. Antisymmetric A relation R: A  A is antisymmetric  if for any In the direct graph of a antisymmetric relation, if there is an edge from a to b, then there must be no edge from b to a. Some graph examples? Other examples?  The parent relation over a group of people.

40. Transitive A relation R:A  A is transitive if (a, b) and (a, c) are in R, then (a, c) is also in R. In the directed graph of a transitive relation, if there a path from a to b, then there is an edge from a to b. Some graph examples? Other examples?  The ancestor relation over a group of people.  The less-than relation over integers.

41. Equivalence Relation A relation R: A  A is equivalent if R is reflexive, symmetric and transitive. Let R be an equivalence relation, for any a in A, we define the equivalence class containing the element a as  [a] = {x: (a, x) in R}  That is, [a] is the set of all elements related to a. Examples?  The same-family relation  The x=y (mod p) relation for the given p.

42. A Equivalence Relation Theorem Theorem. Let R: A→A to be an equivalence relation, then the equivalence classes of R is a partition of A. That is {[a]: a in A} is a partition of A. Proof. 

43. Partial Order A relation R: A→A is a partial order if R is reflexive, antisymmetric, and transitive. Ex’s  Some graph examples?  The boss-employee relation, assuming any person is the boss of him/herself. Minimal elements of a partial order R: A→A  An element a in A is minimal if for any b in A,  (b, a) in R implies b = a. Ex’s

44. Total Order, Path, Length, Cycles Total Order  A relation R: A  A is a total order, if R is a partial order, and for any a, b in A, either (a, b) in R or (b, a) in R. Path, Length, Cycles  Easy to see from the directed graph of a relation R.  Examples?

45. Countable and Uncountable Sets Countable  A set S is countable if one of the following is true S is finite S is infinite but there is a bijection f: S  N, where N is the set of natural numbers.  In other words, S is countable, if elements in S are enumerable. Countable set examples? Uncountable  Sets that are not countable

46. Unions of Countable Sets The union of a finite collection of countable sets is countable  Proof. By dovetailing The Union of a countably infinite collection of countable sets is countable  Proof. By dovetailing

47. There Are Uncountable Sets Proof. By diagonalization. Will give a proof later.

48. Definitions Definitions describe the objects and notions that we use. A definition may be simple, or complex. Precision is essential to any mathematical definition. When defining some object we must make clear what constitutes that object and what does not.

49. Theorems A theorem is a mathematical statement proved true. Generally a theorem is used to refer to a mathematical statement of special interest. Sometimes we prove statements that are interesting only because they assist in the proof of another, more significant statement. Such statements are called lemmas. Occasionally a theorem or its proof may allow us to conclude easily that other, related statements are true. These statements are called corollaries of the theorem.

50. Proofs A proof is a convincing logical argument that a statement is true. In mathematics an argument must be airtight, that is, convincing in an absolute sense. This is rather different from the notion of proof that we use in everyday life or in the law. A mathematician demands proof beyond any doubt.

51. Five Fundamental Proof Techniques Proof by Mathematical Induction Proof by Construction Proof by Contradiction Proof by Pigeon Hole Principle Proof by Diagonalization

52. Mathematical Induction The Principle of Math Induction: Let A be a set of natural numbers such that and For each natural number, if then, then, A = N Math Induction can be used to prove the following assertion  For all natural numbers n, property P(n) is true.

53. Three Steps of Math Induction To show property P(n) is true for all natural numbers, follow the following three steps:  Base case: show P(0) is true.  Induction hypothesis: Assume P(n) is true  Induction: show P(n+1) is true.

54. Math Induction Example Show that for any Proof. (Prove in class.)

55. Proof by Construction Many theorems state that a particular type of object exists. One way to prove such a theorem is by demonstrating how to construct the object. Theorem. For any even number n greater than 2, there exists a graph with n nodes such that every node has exactly 3 edges connecting to it. Proof. By Construction.

56. Proof by Contradiction We assume that the theorem is false and then show that this assumption leads to an obviously false consequence, called a contradiction. Theorem. is irrational. Proof. By contradiction. (do it in class.)

57. The Pigeonhole Principle The Principle. Let m pigeons to sit in n holes, if m > n then there is at least one hole with at least 2 pigeons. Ex: Let R to be a binary relation on a finite set A, and let a, b be in A. If there is a path from a to b in R, then there is a path of length at most |A|. Proof. By Pigeonhole Principle.

58. Diagonalization The Principle. Let R to be a binary relation on a set A, define  D = {a: a in A and (a, a) not in R} For any a in A, define  = {b: b in A and (a, b) in R} Then, for any a in A,

59. Applications of Diagonalizations Theorem 1. Let [0, 1] denote the set of all real numbers between 0 and 1. Then it is uncountable. Proof. By diagonalization. (Do it in class.) Theorem 2. is uncountable, where N is the set of all nature numbers. Proof. By diagonalization. (Do it in class.)

60. Closures and Algorithms The idea of reflexive transitive closure of a relation R is to find a smallest reflexive and transitive relation to cover R. Ex. R: a1 O O a4 a2 O O a3 R*: a1 a4 a2 a3 R* is reflexive and transitive closure of R.

61. Reflexive Transitive Closure Definition: Let R: A X A be a relation. The transitive closure of R is the relation  R* = {(a, b): a, b are in A and there is a path from a to b in R or a = b} Give R, how to find the transitive closure R* of R? Suppose A = {a1, a2,..., an}

62. Closure Algorithm 1 Algorithm 1  InitiallyR* =  For i = 1,..., n do  For each i-tuple (b1,..., bi) in do if (b1,..., bi) is a path in R then add (b1, bi) to R

63. Closure Algorithm 2 Algorithm 2  Initially R* = R {(ai, ai) : ai in A}  Forj = 1, 2,..., n do For i = 1, 2,..., n and k = 1, 2,..., n do  If (ai, aj), (aj, ak) are in R* but (ai, ak) are not in R* then add (ai, ak) to R*

64. Algorithm Analysis How to show the correctness of the two closure algorithms?  Do they halt on any given input?  If halt, Do they output the right result?  What are the performances of the two algorithms Time complexity Space complexity  How to analyze the performance?

65. Big-O Notation Let f, g be two functions from N to N.  f(n) = O(g(n))  if there are two constants c and such that for all  Ex.  But  More Ex’s ?

66. Time Complexity of the Two Closure Algorithms Algorithm 1  Algorithm 2  Which algorithm is better?

67. Alphabets and Languages Consider Pascal Consider English Observation:  Language = Alphabet + Syntax + Semantics Alphabet  A finite set of symbols Ex’s?  {0, 1},{a,..., z, A,..., Z}

68. Strings A string of an alphabet  A sequence of symbols from the alphabet. Ex’s  1010 of {0, 1}  ILovetheory of {a,..., z, A,..., Z} Empty string  a string without symbols  denoted by e.

69. String Length The length of a string w  the number of occurrences of symbols in w, denoted by |w| Ex’s  |1010|= 4  |ILoveTheory|= 11  |e|=0

70. String Concatenation Concatenation of two strings x, y:  xy is the string of x followed by y.  |xy|=|x| +|y|

71. Substring, Prefix, Suffix z is a substring of w if there are strings x, y such that w = xyz  Ex. Find substrings of “Iiovetheory” z is a prefix of w if there is a string x such that w = zx  Ex. Find prefixes of “Iiovetheory” z is a suffix of w if there is a string x such that w = xz  Ex. Find suffixes of “Iiovetheory”

72. Power of a String The power of a string w  Given any natural number i, 

73. Reversal of a String The reversal of a string w   where a is some letter in the alphabet Property 

74. Σ, Σ*, and Languages Let Σ be any finite alphabet {a1, a2,..., an} Σ* denote the set of all strings of Σ Any subset L of Σ* is called a language over Σ. Ex’s: Pascal, English, binary numbers, decimal numbers, etc. Note: Languages are defined as sets. Thus, all set operations such as union, intersection, difference, complement are all applicable to languages.

75. Concatenation and Kleene Star Concatenation of Languages: Given two languages and, the concatenation of and is  Kleene Star: the Kleene star of a language L is  Ex’s  = {a, b}, = {11}

76. The closure of a language The closure of a language L:  Ex. Find the closures of the following languages  L1 = {a, b}, L2 = {11}

77. Finite Representations of Languages The Large Picture  Given an alphabet Σ, the collection of all languages over Σ is when,  Σ* is countably infinite,  is uncountably infinite.

78. Finite Representation of a Language A finite representation of a language is a finite string of some other finite alphabet. The total number of languages represented by strings of is. is countably infinite. Can you get some conclusion?

79. The Language World All languages Finitely representable languages

80. Regular Expressions Definition. Regular expressions over a given alphabet Σ are defined as follows:  (1) Φ and any symbol of Σ are regular exp.  (2) If α and β are regular expressions, then so are (α β), (α U β ), and α*  (3) Only those derived from (1) and (2) are regular expressions Ex’s. Σ= {a, b}  Φ, a, b, (ab), (aa)*, (aabb)*, (aba)*(bb)*

81. Regular Languages Languages defined by regular expressions  Given a regular expression α, let L(α) denote the language represented by α. We have  L(Φ) = Φ, L(a) = {a} for a Σ  L(α U β ) = L(α) U L(β)  L(αβ) = L(α) L(β)  L(α*) = L(α)* Ex’s (give examples in class.) A language L is regular if there is a regular expression α such that L = L(α).

82. Non-Regular Languages There are languages which are not regular Ex’s  

83. Recognizable & Generatable A and B are recognizable Ex.  We can write a C++ program which, given any string w Σ*, can decide whether w in A or not A and B are generatable  Ex.  We can write a C++ program to enumerate all elements of A, provided we have infinite supply of paper and computing time and space.

84. A New World of Languages All languages Finitely representable languages Regular languages