1. Introduction to the Theory of Computation
Qiuyue Wang

2. Goals
What are the fundamental capabilities and limitations of computers?
What is a computer?
What can and can not be computed?
How quickly can a problem be computed?

3. Computational Models What is a computer? Computational models
Formal definition of computation
Computational models
Turing machines (= real computers)
Simpler computing devices: finite state automata, push-down automata

4. Computability What can and can not be computed?
Some problems have no algorithms (we will prove this)
Halting problem

5. Complexity How quickly can a problem be computed?
Many problems probably have no efficient algorithms (no one knows how to prove this yet)
NP-hardness (= what can not be computed efficiently)
factoring an integer into primes
determining the shortest tour of given n cities

6. What is Theory Good for? Elegant way of thinking Expanding your minds
Useful tools in practice
finite automata, regular expressions: text processing
grammars: programming language design and specification
NP-hardness: approximate solutions or randomized computation

7. Text Book and References
Michael Sipser, Introduction to the Theory of Computation, 计算理论导论，机械工业出版社，2002.
J. E. Hopcroft, R. Motivani, and J. D. Ullman, Introduction to Automata Theory, Languages, and Computation, Second Edition, Addison-Wesley, 2001.

8. Contents (1) 0. Introduction 1. Regular Languages
Mathematical notations, proofs
1. Regular Languages
1.1. Finite Automata
1.2. Regular Expressions
1.3. Properties of Regular Languages
2. Context-Free Languages
2.1. Context-Free Grammar
2.2. Pushdown Automata
2.3. Properties of Context-Free Languages

9. Contents (2) 3. The Church-Turing Thesis 4. Decidability
3.1. Turing Machines
3.2. Variants of Turing Machines
3.3. The Definition of Algorithm
4. Decidability
4.1. Decidable Languages
4.2. The Halting Problem
5. Reducibility
5.1. Undecidable Problems
5.2. Mapping Reducibility

10. Contents (3) 7. Time Complexity Final Exam 7.1. P and NP
7.2. NP-Completeness
7.3. NP-Complete Problems
Final Exam

11. Grading Assignments: 20% Project: 20% Final Exam: 60%
Gradiance Online Accelerated Learning
Project: 20%
Final Exam: 60%

12. Mathematical Notations
Sets
Sequences and Tuples
Functions and Relations
Graphs
Strings and Languages

13. Set No order, no repetition among members of a set. Subset: A  B
Union: A  B
Intersection: A  B
Complement: A

14. Power Set Power set: the set of all subsets P (A) = {S| S  A}
E.g. A = {0, 1}
P (A) = { , {0}, {1}, {0, 1} }
|P (A)| = 2|A|

15. Sequences and Tuples
A sequence of objects is a list of these objects in some order.
A tuple is a finite sequence of objects.
k-tuple, pair
Cartesian product of sets: AB = {(a, b)| aA, bB}
A = {0, 1}, B={3, 4}, AB = {(0, 3), (0, 4), (1, 3), (1, 4)}

16. Functions f: DR k-ary function f is a function, D: domain, R: range
E.g. f: {0, 1}  {2, 3} f(x) = x+2
A function takes an input and produces an output
The same input always produces the same output
k-ary function
D = A1… Ak, i.e. k arguments
k=1: unary function, k=2: binary function

17. Relations A set of k-tuples S…S is called an k-ary relation on S
E.g. binary relation “<“ on N
R is an equivalence relation if and only if
Reflexive: xRx for all x in S;
Symmetric: xRy implies yRx;
Transitive: xRy and yRz imply xRz;
E.g. m is an equivalence relation

18. Closures of Relations
P-closure of a relation R is the smallest relation R’ that includes all the pairs of R and possesses the properties in P.
Transitive closure of R, denoted R+, is
1) If (a, b) is in R, then (a, b) is in R+.
2) If (a, b) is in R+ and (b, c) is in R, then (a, c) is in R+.
3) Nothing is in R+ unless it so follows from (1) and (2).
Reflexive and transitive closure of R, denoted R*, is easily seen to be R+  {(a,a)| aS}

19. Example
R = {(1,2), (2,2), (2,3)} be a relation on the set {1, 2, 3}. Then
R+ = {(1,2), (2,2), (2,3), (1,3)}
R* = {(1,1), (1,2), (1,3), (2,2), (2,3), (3,3)}

20. Graphs
G = (V, E), V is a set of vertices (or nodes), and E is a set of pairs of vertices (called edges).
E.g. G=(V, E), V={1,2,3,4}, E={(1,2), (2,3), (1,3), (2,4), (1,4)}
A path in a graph is a sequence of nodes connected by edges
Directed graph
Tree

21. Strings
Symbol: e.g. 0, a, #, !, …
String: a finite sequence of symbols
E.g , abbc
Length of string: |011001| = 6
Empty string: , || = 0
Substring: 100 is a substring of
Concatenation of strings: abbc

22. Languages Alphabet: a finite set of symbols
E.g.  = {0, 1}
Language: a set of strings from some one alphabet
Empty set: 
Set consisting of the empty string: {}
E.g. L = {, 01, 0011, , …}, English

23. Proofs
Proof by construction
Proof by contradiction
Proof by induction

24. Inductive Proofs
Prove a statement S(X) about a family of objects X (e.g., integers, trees) in two parts:
Basis: Prove for one or several small values of X directly.
Inductive step: Assume S(Y ) for Y “smaller than” X; prove S(X) using that assumption.

25. Example
A binary tree with n leaves has 2n-1 nodes.

26. If-And-Only-If Proofs
Often, a statement we need to prove is of the form “X if and only if Y .” We are then required to do two things:
Prove the if-part: Assume Y and prove X.
Prove the only-if-part: Assume X, prove Y .

27. Example Equivalence of two sets
To prove two sets S=T, prove that x is in S if and only if x is in T