1. 1 CDT314 FABER Formal Languages, Automata and Models of Computation Lecture 1 - Intro Mälardalen University 2010

2. 2 Content Adminstrivia Mathematical Preliminaries Countable Sets (Uppräkneliga mängder) Uncountable sets (Överuppräkneliga mängder)

3. 3 Examiner Gordana Dodig-Crnkovic Lecturer Stefan Bygde

4. 4 http://www.idt.mdh.se/kurser/cd5560/11_10 visit home page regularly! Course Home Page

5. 5 How Much Work? 20 hours a week for this type of course (norm) 4 hours lectures 2 hours exercises 14 hours own work a week!

6. 6 Why Theory of Computation? 1.A real computer can be modelled by a mathematical object: a theoretical computer. 2.A formal language is a set of strings, and can represent a computational problem. 3.A formal language can be described in many different ways that ultimately prove to be identical. 4.Simulation: the relative power of computing models can be based on the ease with which one model can simulate another.

7. 7 5. Robustness of a computational model. 6. The Church-Turing thesis: anything that can be computed can be computed by a Turing machine. 7. Nondeterminism: languages can be described by the existence or nonexistence of computational paths. 8. Unsolvability: for some computational problems there is no corresponding algorithm that will unerringly solve them.

8. 8 Practical Applications 1.Efficient compilation of computer languages 2.String searching 3.Identifying the limits; Recognizing difficult problems 4.Applications to other areas: –circuit verification –economics and game theory (finite automata as strategy models in decision-making); –theoretical biology (L-systems as models of organism growth) –computer graphics (L-systems) –linguistics (modeling by grammars)

9. 9 History Euclid's attempt to axiomatize geometry (Archimedes realized, during his own efforts to define the area of a planar figure, that Euclid's attempt had failed and that additional postulates were needed. ) Leibniz's dream of a symbolic logic de Morgan, Boole, Frege, Russell, Whitehead: Mathematics as branch of symbolic logic!

10. 10 1900 Hilberts program 1880 -1936 first programming languages 1931 Gödels incompleteness theorem 1936 Turing machine (showed to be equivalent with recursive functions). Commonly accepted: TM as ultimate computer 1950 automata 1956 language/automata hierarchy

11. 11 Every mathematical truth expressed in a formal language is consisting of a fixed alphabet of admissible symbols, and explicit rules of syntax for combining those symbols into meaningful words and sentences

12. 12 Turing used a Universal Turing machine (UTM) to prove an even more powerful incompleteness theorem because it destroyed not one but two of Hilbert's dreams: 1.Finding a finite list of axioms from which all mathematical truths can be deduced 2.Solving the entscheidungsproblem, ("decision problem“) by producing a "fully automatic procedure" for deciding whether a given proposition (sentence) is true or false.

13. Examination Three midterms Regular Languages Context Free Languages Restriction Free Languages OR: Final Exam in three parts If you have finished the midterm of one time of language you don’t have to do corresponding part of the final exam 13

14. Form and Resources Lectures Exercises Labs (optional) Midterms Webpage (many links, news etc) Literature: Swe: Lennart Salling. Formella språk, automater och beräkningar Eng: Linz, An Introduction to Formal Languages and Automata 14

15. 15 Mathematical Preliminaries

16. 16 Sets Functions Relations Proof Techniques Languages, Alphabets and Strings Strings & String Operations Languages & Language Operations

17. 17 A set is a collection of elements SETS We write

18. 18 Set Representations C = { a, b, c, d, e, f, g, h, i, j, k } C = { a, b, …, k } S = { 2, 4, 6, … } S = { j : j > 0, and j = 2k for some k>0 } S = { j : j is nonnegative and even } finite set infinite set

19. 19 A = { 1, 2, 3, 4, 5 } Universal Set: All possible elements U = { 1, …, 10 } 1 2 3 4 5 A U 6 7 8 9 10

20. 20 Set Operations A = { 1, 2, 3 } B = { 2, 3, 4, 5} Union A U B = { 1, 2, 3, 4, 5 } Intersection A B = { 2, 3 } Difference A - B = { 1 } B - A = { 4, 5 } U A B A-B

21. 21 Complement Universal set = {1, …, 7} A = { 1, 2, 3 } A = { 4, 5, 6, 7} 1 2 3 4 5 6 7 A A A = A

22. 22 { even integers } = { odd integers } 0 2 4 6 1 3 5 7 even odd Integers

23. 23 DeMorgan’s Laws A U B = A B U A B = A U B U

24. 24 Empty, Null Set: = { } S U = S S = S - = S - S = U = Universal Set

25. 25 Subset A = { 1, 2, 3} B = { 1, 2, 3, 4, 5 } A B U Proper Subset:A B U A B

26. 26 Disjoint Sets A = { 1, 2, 3 } B = { 5, 6} A B = U A B

27. 27 Set Cardinality For finite sets A = { 2, 5, 7 } |A| = 3

28. 28 Powersets A powerset is a set of sets Powerset of S = the set of all the subsets of S S = { a, b, c } 2 S = {, {a}, {b}, {c}, {a, b}, {a, c}, {b, c}, {a, b, c} } Observation: | 2 S | = 2 |S| ( 8 = 2 3 )

29. 29 Cartesian Product A = { 2, 4 } B = { 2, 3, 5 } A X B = { (2, 2), (2, 3), (2, 5), ( 4, 2), (4, 3), (4, 5) } |A X B| = |A| |B| Generalizes to more than two sets A X B X … X Z

30. 30 PROOF TECHNIQUES Proof by construction Proof by induction Proof by contradiction

31. 31 Proof by Construction We define a graph to be k-regular if every node in the graph has degree k. Theorem. For each even number n > 2 there exists 3-regular graph with n nodes. 1 2 4 3 0 5 1 2 0 3 n = 4 n = 6

32. 32 Construct a graph G = (V, E) with n > 2 nodes. V= { 0, 1, …, n-1 } E = { {i, i+1}  for 0  i  n-2}  {{n-1,0}} (*)  {{i, i+n/2  for 0  i  n/2 –1} (**) The nodes of this graph can be written consecutively around the circle. (*) edges between adjacent pairs of nodes (**) edges between nodes on opposite sides Proof by Construction END OF PROOF

33. 33 Inductive Proof We have statements P 1, P 2, P 3, … If we know for some k that P 1, P 2, …, P k are true for any n  k that P 1, P 2, …, P n imply P n+1 Then Every P i is true

34. 34 Proof by Induction Inductive basis Find P 1, P 2, …, P k which are true Inductive hypothesis Let’s assume P 1, P 2, …, P n are true, for any n  k Inductive step Show that P n+1 is true

35. 35 Example Theorem A binary tree of height n has at most 2 n leaves. Proof let L(i) be the number of leaves at level i L(0) = 1 L(3) = 8

36. 36 We want to show: L(i)  2 i Inductive basis L(0) = 1 (the root node) Inductive hypothesis Let’s assume L(i)  2 i for all i = 0, 1, …, n Induction step we need to show that L(n + 1)  2 n+1

37. 37 Induction Step hypothesis: L(n)  2 n Level n n+1

38. 38 hypothesis: L(n)  2 n Level n n+1 L(n+1)  2 * L(n)  2 * 2 n = 2 n+1 Induction Step END OF PROOF

39. 39 Proof of induction: The cardinality of the powerset Claim: A set of n elements has 2 n subsets Check: The empty set {} has only one subset: {}. The set {a} (a set with exactly one element) has two subsets: {} och {a}

40. 40 Påstående: En mängd med n element har 2 n delmängder Check (cont.) The set of two elements {a, b} has four subsets: {}, {a}, {b} and {a,b} The set {a, b, c} has eight subsets: {}, {a}, {b}, {c} och {a,b}, {a,c}, {b,c}, {a,b,c} The claim holds for these basic cases.

41. 41 Base Case The simplest case is a set with no elements (there is just one such set), which has 2 0 = 1 subsets.

42. 42 Induction step Suppose that the claim holds for all sets with k elements, i.e., suppose that every set with k elements has 2 k subsets. Show that the claim in this case holds also for all sets with k+1 elements, i.e., show that every set with k+1 elements has 2 k+1 subsets.

43. 43 Consider an arbitrary set with k+1 elements. The subsets of this set can be divided into two groups: Subsets which does not contain the k+1:th element: Such a subset is a subset to the set of the k original elements, and the cardinality of subsets to a set of k elements is (according to the inductive hypothesis) 2 k.

44. 44 Subsets containing the k+1:th element: Such a subset can be constructed by taking any set not containing the k+1th element and add the k+1th element. Since there are 2 k subsets without the k+1th element (by hypothesis), it is also possible to create 2 k subsets including this element. In total there are 2 k + 2 k = 2. 2 k = 2 k+1 subsets to the considered set. END OF PROOF (Exemple taken from the book: Diskret matematik och diskreta modeller, K Eriksson, H. Gavel)

45. 45 Proof by Contradiction We want to prove that a statement P is true we assume that P is false then we arrive at a conclusion that contradicts our assumptions therefore, statement P must be true

46. 46 Example Theorem is not rational Proof Assume by contradiction that it is rational = n/m n and m have no common factors We will show that this is impossible

47. 47 Therefore, n 2 is even n is even n = 2 k 2 m 2 = 4k 2 m 2 = 2k 2 m is even m = 2 p Thus, m and n have common factor 2 Contradiction! = n/m 2 m 2 = n 2 END OF PROOF

48. 48 Countable Sets

49. 49 Infinite sets are either Countable or Uncountable

50. 50 Countable set There is a one to one correspondence between elements of the set and natural numbers

51. 51 We started with the natural numbers, then add infinitely many negative whole numbers to get the integers, then add infinitely many rational fractions to get the rationals, then added infinitely many irrational fractions to get the reals. Each infinite addition seem to increase cardinality: |N| < |Z| < |Q| < |R| But is this true? NO!

52. 52 Example Integers: The set of integers is countable Correspondence: Natural numbers:

53. 53 Example The set of rational numbers is countable Positive Rational numbers:

54. 54 Naive Idea Rational numbers: Natural numbers: Correspondence: Doesn’t work! we will never count numbers with nominator 2:

55. 55 Better Approach... Rows: constant numerator (täljare) Columns: constant denominator

56. 56...

57. 57 We proved: the set of rational numbers is countable by describing an enumeration procedure

58. 58 Definition An enumeration procedure for is an algorithm that generates all strings of one by one Let be a set of strings

59. 59 A set is countable if there is an enumeration procedure for it Observation

60. 60 Example The set of all finite strings is countable We will describe the enumeration procedure Proof

61. 61 Naive procedure: Produce the strings in lexicographic order: Doesn’t work! Strings starting with will never be produced

62. 62 Better procedure 1. Produce all strings of length 1 2. Produce all strings of length 2 3. Produce all strings of length 3 4. Produce all strings of length 4.......... Proper Order

63. 63 Produce strings in Proper Order length 2 length 3 length 1

64. 64 Theorem The set of all finite strings is countable Proof Find an enumeration procedure for the set of finite strings Any finite string can be encoded with a binary string of 0’s and 1’s

65. 65 Produce strings in Proper Order length 2 length 3 length 1 0101 00 01 10 11 000 001 …. 0101 23452345 6 7 …. String = programNatural number

66. 66 PROGRAM = STRING (syntactic way) PROGRAM = FUNCTION  (semantic way) PROGRAM string PROGRAM natural number n  natural number n 

67. 67 Uncountable Sets

68. 68 A set is uncountable if it is not countable Definition

69. 69 Theorem The set of all infinite strings is uncountable We assume we have an enumeration procedure for the set of infinite strings Proof(by contradiction)

70. 70 Infinite string Encoding... = = = Cantor’s diagonal argument...

71. 71 Cantor’s diagonal argument We can construct a new string that is missing in our enumeration! The set of all infinite strings is uncountable! Conclusion

72. 72 There are some integer functions that that cannot be described by finite strings (programs/algorithms). Conclusion An infinite string can be seen as FUNCTION  (n:th output is n:th bit in the string)

73. 73 Theorem Let be an infinite countable set The powerset of is uncountable Example of uncountable infinite sets

74. 74 Proof Since is countable, we can write

75. 75 Elements of the powerset have the form: ……

76. 76 We encode each element of the power set with a binary string of 0’s and 1’s Powerset element Encoding...

77. 77 Let’s assume (by contradiction) that the powerset is countable. we can enumerate the elements of the powerset Then:

78. 78 Powerset element Encoding...

79. 79 Take the powerset element whose bits are the complements in the diagonal

80. 80 New element: (binary complement of diagonal)...

81. 81 The new element must be some of the powerset However, that’s impossible: the i-th bit of must be the complement of itself from definition of Contradiction!

82. 82 Since we have a contradiction: The powerset of is uncountable END OF PROOF

83. 83 Example Alphabet : The set of all finite strings: infinite and countable uncountable infinite The powerset of contains all languages: An Application: Languages

84. 84 Finite strings (algorithms): countable Languages (power set of strings): uncountable There are infinitely many more languages than finite strings.

85. 85 There are some languages that cannot be described by finite strings (algorithms). Conclusion

86. 86 CardinalityCardinality - Kardinaltal A Cardinal number is a measure of the size of a set. The cardinality for a finite set is simply the number of elements in the set. Two sets has equal size if it is possible to pair the elements one by one in the two sets. That is, if there exists a bijection between the two sets. This concept can be generalised to infinite sets. For instance, the set of positive integers and the set of integers have equal size. On the contrary, it is not possible to pair the real numbers with the integers in this fashion. The set of real numbers is larger than the set of integers. Cardinal numbers can be defined as in the following way: two sets have the same cardinal number if and only if they have the same size (i.e., can be paired together one and one). As an example, the cardinal number for the integers is  0 (alef-0, alef is the first letter in the hebrew alphabet). These infinite cardinal numbers are called transfinite cardinal numbers.

87. 87 Georg Cantor developed, in the end of the 19th century, the logical foundation of mathematics: set theory. Cantor introduced the notion transfinite cardinal numbers. The simplest, ”smallest", infinity, he named  0. This is the countable infinite sets’ (for instance, the integers) cardinal number. The cardinality of the set of points on a line, or the points in a plane or field, Cantor named  1. Are there ”larger” infinities? More about Infinty

88. 88 Yes! Cantor showed that the set of functions on a line was ”even more infinite” than the points on the line, and he called that cardinality  2. Cantor discovered that it was possible to use artithmetic on cardinal numbers just as with finite numbers. But the arithmetic rules became quite simplistic:  0 + 1=  0  0 +  0 =  0  0 ·  0 =  0.

89. 89 However, with powers something happened:  0  0 (  0 to the power of  0 ) =  1. In general: 2  n (2 to the power of  n ) =  n+1 This implies that there are infinitly many infinities, in strictly increasing order.

90. 90 But is it really certain that there is no cardinality in between the countable infinity and the points on the line? Cantor tried to prove the so-called continuum hypothesis. Cantor: two different infinities  0 and  1 http://www.ii.com/math/ch/#cardinals Continuum Hypothesis:  0 <  1 = 2  0 See also: http://www.nyteknik.se/pub/ipsart.asp?art_id=26484