1 Theory of Computation 計算理論

2 Instructor: 顏嗣鈞 Web: Time: 2:20-5:10 PM, Tuesday Place: BL 112 Office hours: by appointment Class web page:

3 : Introduction to Automata Theory, Languages, and Computation John E. Hopcroft, Rajeev Motwani, Jeffrey D. Ullman, (2nd Ed. Addison-Wesley, 2001) textbook

4 1st Edition Introduction to Automata Theory, Languages, and Computation John E. Hopcroft, Jeffrey D. Ullman, (Addison-Wesley, 1979)

5 Grading HW : 0-20% Midterm exam.: 35-45% Final exam.: 45-55%

6 Why Study Automata Theory? Finite automata are a useful model for important kinds of hardware and software: Software for designing and checking digital circuits. Lexical analyzer of compilers. Finding words and patterns in large bodies of text, e.g. in web pages. Verification of systems with finite number of states, e.g. communication protocols.

7 Why Study Automata Theory? (2) The study of Finite Automata and Formal Languages are intimately connected. Methods for specifying formal languages are very important in many areas of CS, e.g.: Context Free Grammars are very useful when designing software that processes data with recursive structure, like the parser in a compiler. Regular Expressions are very useful for specifying lexical aspects of programming languages and search patterns.

8 Why Study Automata Theory? (3) Automata are essential for the study of the limits of computation. Two issues: What can a computer do at all? (Decidability) What can a computer do efficiently? (Intractability)

9 Applications Theoretical Computer Science Automata Theory, Formal Languages, Computability, Complexity … Compiler Prog. languages Comm. protocols circuits Pattern recognition Supervisory control Quantum computing Computer-AidedVerification...

10 Aims of the Course To familiarize you with key Computer Science concepts in central areas like - Automata Theory - Formal Languages - Models of Computation - Complexity Theory To equip you with tools with wide applicability in the fields of CS and EE, e.g. for - Complier Construction - Text Processing - XML

11 Fundamental Theme What are the capabilities and limitations of computers and computer programs? –What can we do with computers/programs? –Are there things we cannot do with computers/programs?

12 Studying the Theme How do we prove something CAN be done by SOME program? How do we prove something CANNOT be done by ANY program?

13 Example: The Halting Problem (1) Consider the following program. Does it terminate for all values of n  1? while (n > 1) { if even(n) { n = n / 2; } else { n = n * 3 + 1; }

14 Example: The Halting Problem (2) Not as easy to answer as it might first seem. Say we start with n = 7, for example: 7, 22, 11, 34, 17, 52, 26, 13, 40, 20, 10, 5, 16, 8, 4, 2, 1 In fact, for all numbers that have been tried (a lot!), it does terminate but in general?

15 Example: The Halting Problem (3) Then the following important undecidability result should perhaps not come as a total surprise: It is impossible to write a program that decides if another, arbitrary, program terminates (halts) or not. What might be surprising is that it is possible to prove such a result. This was first done by the British mathematician Alan Turing.

16 Our focus Automata Computability Complexity

17 Topics 1. Finite automata, Regular languages, Regular grammars: deterministic vs. nondeterministic, one-way vs. two-way finite automata, minimization, pumping lemma for regular sets, closure properties. 2. Pushdown automata, Context-free languages, Context-free grammars: deterministic vs. nondeterministic, one-way vs. two-way PDAs, reversal bounded PDAs, linear grammars, counter machines, pumping lemma for CFLs, Chomsky normal form, Greibach normal form, closure properties. 3.

18 Topics (cont’d) 3. Linear bounded automata, Context- sensitive languages, Context-sensitive grammars. 4. Turing machines, Recursively enumerable sets, Type 0 grammars: variants of Turing machines, halting problem, undecidability, Post correspondence problem, valid and invalid computations of TMs.

19 Topics (cont’d) 5. Basic recursive function theory 6. Basic complexity theory: Various resource bounded complexity classes, including NLOGSPACE, P, NP, PSPACE, EXPTIME, and many more. reducibility, completeness. 7. Advanced topics: Tree Automata, quantum automata, probabilistic automata, interactive proof systems, oracle computations, cryptography.

20 Who should take this course? YOU

21 Languages The terms language and word are used in a strict technical sense in this course: A language is a set of words. A word is a sequence (or string) of symbols.  (or ) denotes the empty word, the sequence of zero symbols.

22 Symbols and Alphabets What is a symbol, then? Anything, but it has to come from an alphabet which is a finite set. A common (and important) instance is  = {0, 1}. , the empty word, is never an symbol of an alphabet.

23 Computation CPU memory

24 CPU input memory output memory Program memory temporary memory

25 CPU input memory output memory Program memory temporary memory compute Example:

26 CPU input memory output memory Program memory temporary memory compute

27 CPU input memory output memory Program memory temporary memory compute

28 CPU input memory output memory Program memory temporary memory compute

29 Automaton CPU input memory output memory Program memory temporary memory Automaton

30 Different Kinds of Automata Automata are distinguished by the temporary memory Finite Automata: no temporary memory Pushdown Automata: stack Turing Machines: random access memory

31 input memory output memory temporary memory Finite Automaton Finite Automaton Example: Vending Machines (small computing power)

32 input memory output memory Stack Pushdown Automaton Pushdown Automaton Example: Compilers for Programming Languages (medium computing power) Push, Pop

33 input memory output memory Random Access Memory Turing Machine Turing Machine Examples: Any Algorithm (highest computing power)

34 Finite Automata Pushdown Automata Turing Machine Power of Automata Less powerMore power Solve more computational problems

35 Mathematical Preliminaries

36 Mathematical Preliminaries Sets Functions Relations Graphs Proof Techniques

37 A set is a collection of elements SETS We write

38 Set Representations C = { a, b, c, d, e, f, g, h, i, j, k } C = { a, b, …, k } S = { 2, 4, 6, … } S = { j : j > 0, and j = 2k for some k>0 } S = { j : j is nonnegative and even } finite set infinite set

39 A = { 1, 2, 3, 4, 5 } Universal Set: all possible elements U = { 1, …, 10 } A U

40 Set Operations A = { 1, 2, 3 } B = { 2, 3, 4, 5} Union A U B = { 1, 2, 3, 4, 5 } Intersection A B = { 2, 3 } Difference A - B = { 1 } B - A = { 4, 5 } U A B A-B

41 A Complement Universal set = {1, …, 7} A = { 1, 2, 3 } A = { 4, 5, 6, 7} A A = A

even { even integers } = { odd integers } odd Integers

43 DeMorgan’s Laws A U B = A B U A B = A U B U

44 Empty, Null Set: = { } S U = S S = S - = S - S = U = Universal Set

45 Subset A = { 1, 2, 3} B = { 1, 2, 3, 4, 5 } A B U Proper Subset:A B U A B

46 Disjoint Sets A = { 1, 2, 3 } B = { 5, 6} A B = U AB

47 Set Cardinality For finite sets A = { 2, 5, 7 } |A| = 3

48 Powersets A powerset is a set of sets Powerset of S = the set of all the subsets of S S = { a, b, c } 2 S = {, {a}, {b}, {c}, {a, b}, {a, c}, {b, c}, {a, b, c} } Observation: | 2 S | = 2 |S| ( 8 = 2 3 )

49 Cartesian Product A = { 2, 4 } B = { 2, 3, 5 } A X B = { (2, 2), (2, 3), (2, 5), ( 4, 2), (4, 3), (4, 5) } |A X B| = |A| |B| Generalizes to more than two sets A X B X … X Z

50 FUNCTIONS domain a b c range f : A -> B A B If A = domain then f is a total function otherwise f is a partial function f(1) = a

51 RELATIONS R = {(x 1, y 1 ), (x 2, y 2 ), (x 3, y 3 ), …} x i R y i e. g. if R = ‘>’: 2 > 1, 3 > 2, 3 > 1 In relations x i can be repeated

52 Equivalence Relations Reflexive: x R x Symmetric: x R y y R x Transitive: x R y and y R z x R z Example: R = ‘=‘ x = x x = y y = x x = y and y = z x = z

53 Equivalence Classes For equivalence relation R equivalence class of x = {y : x R y} Example: R = { (1, 1), (2, 2), (1, 2), (2, 1), (3, 3), (4, 4), (3, 4), (4, 3) } Equivalence class of 1 = {1, 2} Equivalence class of 3 = {3, 4}

54 GRAPHS A directed graph Nodes (Vertices) V = { a, b, c, d, e } Edges E = { (a,b), (b,c), (b,e),(c,a), (c,e), (d,c), (e,b), (e,d) } node edge a b c d e

55 Labeled Graph a b c d e

56 Walk a b c d e Walk is a sequence of adjacent edges (e, d), (d, c), (c, a)

57 Path a b c d e Path is a walk where no edge is repeated Simple path: no node is repeated

58 Cycle a b c d e Cycle: a walk from a node (base) to itself Simple cycle: only the base node is repeated base

59 Euler Tour a b c d e base A cycle that contains each edge once

60 Hamiltonian Cycle a b c d e base A simple cycle that contains all nodes

61 Trees root leaf parent child Trees have no cycles

62 root leaf Level 0 Level 1 Level 2 Level 3 Height 3

63 Binary Trees

64 PROOF TECHNIQUES Proof by induction Proof by contradiction

65 Induction We have statements P 1, P 2, P 3, … If we know for some b that P 1, P 2, …, P b are true for any k >= b that P 1, P 2, …, P k imply P k+1 Then Every P i is true

66 Proof by Induction Inductive basis Find P 1, P 2, …, P b which are true Inductive hypothesis Let’s assume P 1, P 2, …, P k are true, for any k >= b Inductive step Show that P k+1 is true

67 Example Theorem: A binary tree of height n has at most 2 n leaves. Proof by induction: let L(i) be the number of leaves at level i L(0) = 1 L(1) = 2 L(2) = 4 L(3) = 8

68 We want to show: L(i) <= 2 i Inductive basis L(0) = 1 (the root node) Inductive hypothesis Let’s assume L(i) <= 2 i for all i = 0, 1, …, k Induction step we need to show that L(k + 1) <= 2 k+1

69 Induction Step From Inductive hypothesis: L(k) <= 2 k Level k k+1

70 L(k) <= 2 k Level k k+1 L(k+1) <= 2 * L(k) <= 2 * 2 k = 2 k+1 Induction Step

71 Remark Recursion is another thing Example of recursive function: f(n) = f(n-1) + f(n-2) f(0) = 1, f(1) = 1

72 Proof by Contradiction We want to prove that a statement P is true we assume that P is false then we arrive at an incorrect conclusion therefore, statement P must be true

73 Example Theorem: is not rational Proof: Assume by contradiction that it is rational = n/m n and m have no common factors We will show that this is impossible

74 = n/m 2 m 2 = n 2 Therefore, n 2 is even n is even n = 2 k 2 m 2 = 4k 2 m 2 = 2k 2 m is even m = 2 p Thus, m and n have common factor 2 Contradiction!