CS 3813: Introduction to Formal Languages and Automata Chapter 1 Introduction to the Theory of Computation These class notes are based on material from our textbook, An Introduction to Formal Languages and Automata, 3rd ed., by Peter Linz, published by Jones and Bartlett Publishers, Inc., Sudbury, MA, 2001. They are intended for classroom use only and are not a substitute for reading the textbook.

Acknowledgement Unless otherwise credited, all class notes on this website are based on the required textbook for this course, Linz, Peter. An Introduction to Formal Languages and Automata, 3rd ed. Sudbury, Mass.: Jones and Bartlett Publishers, 2001. These notes are intended solely for the use of the students in the CS 3813 class at Mississippi State University. Please assume any errors to be mine, and not the author of the textbook.

Topic of course What are the fundamental capabilities and limitations of computers? To answer this, we will study abstract mathematical models of computers These mathematical models abstract away many of the details of computers to allow us to focus on the essential aspects of computation It allows us to develop a mathematical theory of computation

Review of set theory Can specify a set in two ways: - list of elements: A = {6, 12, 28} - characteristic property: B = {x | x is a positive, even integer} Set membership: 12  A, 9  A Set inclusion: A  B (A is a subset of B) A  B (A is a proper subset of B) Set operations: union: A  {9, 12} = {6, 9, 12, 28} intersection: A  {9, 12} = {12} difference: A - {9, 12} = {6, 28}

Set theory (continued) Another set operation, called “taking the complement of a set”, assumes a universal set. Let U = {0, 1, 2, 3, 4, 5, 6, 7, 8, 9} be the universal set. Let A = {2, 4, 6, 8} Then = U - A = {0, 1, 3, 5, 7, 9} The empty set:  = {}

Set theory (continued) The cardinality of a set is the number of elements in a set. Let S = {2, 4, 6} Then |S| = 3 The powerset of S, represented by 2S, is the set of all subsets of S. 2S = {{}, {2}, {4}, {6},{2,4}, {2,6}, {4,6}, {2,4,6}} The number of elements in a powerset is |2S| = 2|S|

What does the title of this course mean? Formal language a subset of the set of all possible strings from from a set of symbols example: the set of all syntactically correct C programs Automata abstract, mathematical model of computer examples: finite automata, pushdown automata Turing machine, RAM, PRAM, many others Let’s consider each of these in turn

Formal language Alphabet = finite set of symbols or characters examples: S = {a,b}, binary, ASCII String = finite sequence of symbols from an alphabet examples: aab, bbaba, also computer programs A formal language is a set of strings over an alphabet Examples of formal languages over alphabet S = {a, b}: L1 = {aa, aba, aababa, aa} L2 = {all strings containing just two a’s and any number of b’s} A formal language can be finite or infinite.

Formal languages (continued) We often use string variables; u = aab, v = bbaba Operations on strings length: |u| = 3 reversal: uR = baa concatenation: uv = aabbbaba The empty string, denoted  , has some special properties: |  | = 0  w = w  = w

Formal languages (continued) If w is a string, then wn stands for the string obtained by repeating w n times. w0 =  S+ = S* - {} L0 = {} L1 = L

Operations on languages Set operations: L1  L2 = {x | x  L1 or x  L2} is union L1  L2 = {x | x  L1 and x  L2} is intersection L1 - L2 = {x | x  L1 and x  L2} is difference = * - L is complement L1  L2 = (L1 - L2)  (L2 - L1) is “symmetric difference” String operations: LR = {wR | w  L} is “reverse of language” L1L2 = {xy | x  L1, y  L2} is “concatenation of languages” L* = {x = x1…xk | k  0 and x1, …, xk  L} = L0  L1  L2 . . . . is “Kleene star” or "star closure" L+ = L1  L2 . . . . is positive closure

Some review questions What is {, 01, 001}  {, 00, 10}? What is the concatenation of {0, 11, 010} and {, 10, 010}? What are the 5 shortest strings in the language {0i1i | i  0}? What is the powerset {a, b, ab}?

Important example of a formal language alphabet: ASCII symbols string: a particular C++ program formal language: set of all legal C++ programs

Grammars A grammar G is defined as a quadruple: G = (V, T, S, P) Where V is a finite set of objects called variables T is a finite set of objects called terminal symbols S  V is a special symbol called the Start symbol P is a finite set of productions or "production rules" Sets V and T are nonempty and disjoint

Grammars Production rules have the form: x  y where x is an element of (V  T)+ and y is in (V  T)* Given a string of the form w = uxv and a production rule we can apply the rule, replacing x with y, giving z = uyv We can then say that w  z Read as "w derives z", or "z is derived from w"

Grammars If u  v, v  w, w  x, x  y, and y  z, then we say: * u  z This says that u derives z in an unspecified number of steps. Along the way, we may generate strings which contain variables as well as terminals. These are called sentential forms.

Grammars What is the relationship between a language and a grammar? Let G = (V, T, S, P) The set * L(G) = {w  T* : S  w} is the language generated by G.

Grammars Consider the grammar G = (V, T, S, P), where: S  aSb V = {S} T = {a, b} S = S, P = S  aSb S  

Grammars What are some of the strings in this language? S  aSb  ab S  aSb  aaSbb  aabb S  aSb  aaSbb  aaaSbbb  aaabbb It is easy to see that the language generated by this grammar is: L(G) = {anbn : n  0} (See proof on pp. 21-22 in Linz)

Grammars Let's go the other way, from a description of a language to a grammar that generates it. Find a grammar that generates: L = {anbn+1 : n  0} So the strings of this language will be: b (0 a's and 1 b) abb (1 a and 2 b's) aabbb (2 a's and 3 b's) . . . In order to generate a string with no a's and 1 b, you might want to write rules for the grammar that say: S  ab a   But you can't do this; a is a terminal, and you can't change a terminal, only variables

Grammars So, instead of: S  ab a   we create another variable, A (we often use capital letters to stand for variables), to use in place of the terminal, a: S  Ab A   Now you might think that we can use another S rule here to generate the other part of the string, the anbn part S  aSb But you can't, because that will generate ab, aabb, etc. Note, however, that if we use A in place of S, that will solve our problem: A  aAb

Grammars So, here are our rules: S  Ab A  aAb A   The S  Ab rule creates a single b terminal on the right, preceded by other strings (including possibly the empty string) on the left. The A   rule allows the single b string to be generated. The A  aAb rule and the A   rule allows ab, aabb, aaabbb, etc. to be generated on the left side of the string.

Language-recognition problem There are many types of computational problem. We will focus on the simplest, called the “language-recognition problem.” Given a string, determine whether it belongs to a language or not. (Practical application for compilers: Is this a valid C++ program?) We study simple models of computation called “automata,” and measure their computational power in terms of the class of languages they can recognize.

Automata Input File Control Unit (with finite states) Temporary Storage Output

“Computer” or Turing machine (Alan Turing 1936) 1 2 3 X B Finite-state control Infinite tape or “memory” Read/write head

Finite automata Developed in 1940’s and 1950’s for neural net models of brain and computer hardware design Finite memory! Many applications: text-editing software: search and replace many forms of pattern-recognition (including use in WWW search engines) compilers: recognizing keywords (lexical analysis) sequential circuit design software specification and design communications protocols

Pushdown automata Noam Chomsky’s work in 1950’s and 1960’s on grammars for natural languages infinite memory, organized as a stack Applications: compilers: parsing computer programs programming language design

Computational power TM LBA PDA FSA

Automata, languages, and grammars In this course, we will study the relationship between automata, languages, and grammars Recall that a formal language is a set of strings over a finite alphabet Automata are used to recognize languages Grammars are used to generate languages (All these concepts fit together)

Classification of automata, languages, and grammars Turing machine Recursively enumerable Linear-bounded automaton Context sensitive Nondeterministic push-down automaton Context free Finite-state automaton regular

Besides developing a theory of classes of languages and automata, we will study the limits of computation. We will consider the following two important questions: What problems are impossible for a computer to solve? What problems are too difficult for a computer to solve in practice (although possible to solve in principle)?

Uncomputable (undecidable) problems Many well-defined (and apparently simple) problems cannot be solved by any computer Examples: For any program x, does x have an infinite loop? For any two programs x and y, do these two programs have the same input/output behavior? For any program x, does x meet its specification? (i.e., does it have any bugs?)

Intractable problems We will learn how to mathematically characterize the difficulty of computational problems. There is a class of problems that can be solved in a reasonable amount of time and another class that cannot (What good is it for a problem to be solvable, if it cannot be solved in the lifetime of the universe?) The field of cryptography, for example, relies on the fact that the computational problem of “breaking a code” is intractable

Why study the theory of computing? Core mathematics of CS (has not changed in over 30 years) Many applications, especially in design of compilers and programming languages Important to be able to recognize uncomputable and intractable problems Need to know this in order to be a computer scientist, not simply a computer programmer