Introduction to the Theory of Computation Qiuyue Wang qiuyuew@yahoo.com qiuyuew@ruc.edu.cn

Goals What are the fundamental capabilities and limitations of computers? What is a computer? What can and can not be computed? How quickly can a problem be computed?

Computational Models What is a computer? Computational models Formal definition of computation Computational models Turing machines (= real computers) Simpler computing devices: finite state automata, push-down automata

Computability What can and can not be computed? Some problems have no algorithms (we will prove this) Halting problem

Complexity How quickly can a problem be computed? Many problems probably have no efficient algorithms (no one knows how to prove this yet) NP-hardness (= what can not be computed efficiently) factoring an integer into primes determining the shortest tour of given n cities

What is Theory Good for? Elegant way of thinking Expanding your minds Useful tools in practice finite automata, regular expressions: text processing grammars: programming language design and specification NP-hardness: approximate solutions or randomized computation

Text Book and References Michael Sipser, Introduction to the Theory of Computation, 计算理论导论，机械工业出版社，2002. J. E. Hopcroft, R. Motivani, and J. D. Ullman, Introduction to Automata Theory, Languages, and Computation, Second Edition, Addison-Wesley, 2001. http://info.ruc.edu.cn/wangqiuyue/comp_theory.htm

Contents (1) 0. Introduction 1. Regular Languages Mathematical notations, proofs 1. Regular Languages 1.1. Finite Automata 1.2. Regular Expressions 1.3. Properties of Regular Languages 2. Context-Free Languages 2.1. Context-Free Grammar 2.2. Pushdown Automata 2.3. Properties of Context-Free Languages

Contents (2) 3. The Church-Turing Thesis 4. Decidability 3.1. Turing Machines 3.2. Variants of Turing Machines 3.3. The Definition of Algorithm 4. Decidability 4.1. Decidable Languages 4.2. The Halting Problem 5. Reducibility 5.1. Undecidable Problems 5.2. Mapping Reducibility

Contents (3) 7. Time Complexity Final Exam 7.1. P and NP 7.2. NP-Completeness 7.3. NP-Complete Problems Final Exam

Grading Assignments: 20% Project: 20% Final Exam: 60% Gradiance Online Accelerated Learning Project: 20% Final Exam: 60%

Mathematical Notations Sets Sequences and Tuples Functions and Relations Graphs Strings and Languages

Set No order, no repetition among members of a set. Subset: A  B Union: A  B Intersection: A  B Complement: A

Power Set Power set: the set of all subsets P (A) = {S| S  A} E.g. A = {0, 1} P (A) = { , {0}, {1}, {0, 1} } |P (A)| = 2|A|

Sequences and Tuples A sequence of objects is a list of these objects in some order. A tuple is a finite sequence of objects. k-tuple, pair Cartesian product of sets: AB = {(a, b)| aA, bB} A = {0, 1}, B={3, 4}, AB = {(0, 3), (0, 4), (1, 3), (1, 4)}

Functions f: DR k-ary function f is a function, D: domain, R: range E.g. f: {0, 1}  {2, 3} f(x) = x+2 A function takes an input and produces an output The same input always produces the same output k-ary function D = A1… Ak, i.e. k arguments k=1: unary function, k=2: binary function

Relations A set of k-tuples S…S is called an k-ary relation on S E.g. binary relation “<“ on N R is an equivalence relation if and only if Reflexive: xRx for all x in S; Symmetric: xRy implies yRx; Transitive: xRy and yRz imply xRz; E.g. m is an equivalence relation

Closures of Relations P-closure of a relation R is the smallest relation R’ that includes all the pairs of R and possesses the properties in P. Transitive closure of R, denoted R+, is 1) If (a, b) is in R, then (a, b) is in R+. 2) If (a, b) is in R+ and (b, c) is in R, then (a, c) is in R+. 3) Nothing is in R+ unless it so follows from (1) and (2). Reflexive and transitive closure of R, denoted R*, is easily seen to be R+  {(a,a)| aS}

Example R = {(1,2), (2,2), (2,3)} be a relation on the set {1, 2, 3}. Then R+ = {(1,2), (2,2), (2,3), (1,3)} R* = {(1,1), (1,2), (1,3), (2,2), (2,3), (3,3)}

Graphs G = (V, E), V is a set of vertices (or nodes), and E is a set of pairs of vertices (called edges). E.g. G=(V, E), V={1,2,3,4}, E={(1,2), (2,3), (1,3), (2,4), (1,4)} A path in a graph is a sequence of nodes connected by edges Directed graph Tree

Strings Symbol: e.g. 0, a, #, !, … String: a finite sequence of symbols E.g. 011001, abbc Length of string: |011001| = 6 Empty string: , || = 0 Substring: 100 is a substring of 011001 Concatenation of strings: 011001abbc

Languages Alphabet: a finite set of symbols E.g.  = {0, 1} Language: a set of strings from some one alphabet Empty set:  Set consisting of the empty string: {} E.g. L = {, 01, 0011, 000111, …}, English

Proofs Proof by construction Proof by contradiction Proof by induction

Inductive Proofs Prove a statement S(X) about a family of objects X (e.g., integers, trees) in two parts: Basis: Prove for one or several small values of X directly. Inductive step: Assume S(Y ) for Y “smaller than” X; prove S(X) using that assumption.

Example A binary tree with n leaves has 2n-1 nodes.

If-And-Only-If Proofs Often, a statement we need to prove is of the form “X if and only if Y .” We are then required to do two things: Prove the if-part: Assume Y and prove X. Prove the only-if-part: Assume X, prove Y .

Example Equivalence of two sets To prove two sets S=T, prove that x is in S if and only if x is in T