Welcome to Automata Theory Course Why Study Automata? What the Course is About This theory plays an important role in computer science. I am aware that many students don’t see the importance of a mathematical approach to CS. The feeling “just let me near a keyboard and let me code” is quite common I’ll try to give you some reasons why you should learn the material contained in this course.

Course info Instructor: Yrd. Doc. Dr. Cengiz Örencik E-mail: cengizorencik@beykent.edu.tr Course material myweb.sabanciuniv.edu/cengizo/courses

Grading Midterm 25% Inclass quizes (2) 25% (15+10) Final 50% Hws Self study Not be graded

This stuff is useful This class is about mathematical models of computing Ways of thinking Theory can drive practice Mathematical models of computation predated computers as we know them (1930-40’s)

What is Automata Theory? Cpt S 317: Spring 2009 What is Automata Theory? Study of abstract computing devices, or “machines” Automaton = an abstract computing device Note: A “device” need not even be a physical hardware! A fundamental question in computer science: Find out what different models of machines can do and cannot do School of EECS, WSU

Alan Turing (1912-1954) (A pioneer of automata theory) Cpt S 317: Spring 2009 (A pioneer of automata theory) Alan Turing (1912-1954) Father of Modern Computer Science English mathematician Studied abstract machines called Turing machines even before computers existed Heard of the Turing test? School of EECS, WSU

Why Study Automata? Regular expressions are used in many systems to describe patterns. E.g., UNIX a.*b. Finite automata model protocols, electronic circuits. Verify correctness So let’s see some of the ideas we’re going to learn about in the course and how they appear in practice. Click 1 One very commonly used idea is the regular expression, a simple notation for describing many of the patterns that arise naturally in practice. Many pieces of software that you might find yourself working on in the future need a simple input language to describe patterns, so you might well find yourself implementing some form of regular expressions. Click 2 For example, many UNIX text-processing commands use a variety of regular expressions. This expression describes a line of text that has a letter “a”, followed by any number of characters, followed by a letter “b”. Click 4 Finite automata are another topic we’ll see early on. They are, in fact the way regular-expression-based languages are implemented. They also have been used for decades to model electronic circuits, and in particular, to help design good circuits. They have also been used to model protocols, and we’ll give some examples later in the course. Especially, finite automata underlie the body of techniques known as “model checking,” which has been used to verify the correctness of both communication protocols and large electronic circuits.

Why? – (2) Context-free grammars are used to describe the syntax of essentially every programming language. Not to forget their important role in describing natural languages. Click 1 Another important aspect of the course is context-free grammars. These are used to put a tree structure on strings, typically text strings, according to some recursive rules. They are an essential for describing the syntax of programming languages, and are a part of every compiler. They also play an important role in describing the syntax of natural languages, and are used in software that does machine translation and other natural-language processing.

Why? – (3) When developing solutions to real problems, we often confront the limitations of what software can do. Undecidable things – no program whatever can do it. Intractable things – there are programs, but no fast programs. Automata theory gives you the tools. The topics we just mentioned are essentially tools for doing simple but important things. But there is a second broad theme in this course. Click 1 There are fundamental limitations on our ability to compute. A computer scientist should be aware of these limitations, because only then can you avoid spending time attempting something that is impossible. Click 2 One limitation is undecidability. There are problems that cannot be solved by computation. For example, you might imagine that you could write a compiler that would refuse to compile programs that printed out dirty words. Even assuming you had a precise definition of what words we “dirty,” you can’t do this. We’re going to prove that there is no way to tell whether a program will ever print a particular word, or even whether it will ever print anything at all. Click 3 And we also need to know about the class of problems called intractable. These are, colloquially, problems that we can solve, but whose solution takes time that is exponential in the input size. These problems generally need to be finessed in some way, such as by approximating the solution. The reality of the theory of intractability is a bit different from the colloquial version. While the undecidable problems have been proved not to have any solution, for the intractable problems we have very strong evidence that they require exponential time, but no proof. We’ll explain all this when we get to NP-completeness as the culmination of the course.

Cpt S 317: Spring 2009 The Chomsky Hierachy A containment hierarchy of classes of formal languages Recursively- enumerable (TM) Context- free (PDA) Regular (DFA) School of EECS, WSU

Course Outline Regular Languages and their descriptors: Finite automata, nondeterministic finite automata, regular expressions. Algorithms to decide questions about regular languages, e.g., is it empty? Closure properties of regular languages. So here’s a summary of what will be covered in the course. Click 1 We’ll start off with what are called regular languages. A language is just a set of strings, for example the set of character strings that are valid Java programs. Click 2 The regular languages are exactly those sets of strings that can be described by finite automata or regular expressions. This discussion will also introduce the important concept of nondeterminism – machines that can magically turn into many machines that each do something independently, but with a coordinated effect. This model of computation is fanciful to say the least, but we’ll see it plays a really important role in several places, including design of algorithms and in understanding the theory of intractable problems. Click 3 We’re then going to turn to properties of the regular languages. These properties include the ability to answer certain questions about finite automata and regular expressions that we cannot decide about programs in general. An example would be to tell whether a device makes an output in response to even one input. You can’t tell for programs in general, but you can for finite automata. It is this tractability – our ability to understand what simple formalisms like finite automata or regular expressions do that make them so valuable when they can be used. Click 4 We’re also going to talk about closure properties of regular languages. For example, the union or intersection of two regular languages is also a regular language.

Course Outline – (2) Context-free languages and their descriptors: Context-free grammars, pushdown automata. Decision and closure properties. Click 1 The next big topic will be context-free languages. This is a somewhat larger class of languages than the regular languages, and they enable us to do things that you can’t do with regular languages, such as match balanced parentheses or XML tags. Click 2 We’ll talk about two ways to describe such languages. First, by context-free grammars, a recursive system for generating strings, and then by pushdown automata, which are a generalization of the finite automata that we use for regular languages. Click 3 We’ll then repeat our examination of decision properties and closure properties for this larger class of languages. Many of things we can tell about regular languages that we cannot tell in general, we can also tell about context-free languages, but there are unfortunately some exceptions. Similarly, many of the operations under which regular languages are closed also yield a context free language when applied to context-free languages. But again we lose some operations. For example, context-free languages are closed under union, but not under intersection.

Course Outline – (3) Recursive and recursively enumerable languages. Turing machines, decidability of problems. The limit of what can be computed. Intractable problems. Problems that (appear to) require exponential time. NP-completeness and beyond. Click 1 Next, we take up the largest class of languages that we can reasonably say can be dealt with by a computer – the recursively enumerable languages. We also look at the smaller but important subset of the recursively enumerable languages, called the recursive languages. These are the languages for which there is an algorithm to tell whether or not a string is in the language. We introduce the Turing machine, an automaton in the spirit of the first kinds of automata we meet: finite and pushdown automata. The Turing machine is, however, much more powerful than either of these. In fact, it is as powerful as any model that has ever been thought of to answer the question “what can we compute?” The payoff for the study of Turing machines is that we can answer the question of what can be decided by computation, or what can be computed by any means at all. We shall develop tools for showing that certain problems cannot be decided – that is, answered by a computer. The example we already mentioned – does a program print a dirty word, should give you the idea of what we can achieve. Click 2 Finally, we’re going to cover the theory of NP-completeness. The critical question is what can we do with an algorithm that runs in time that is some polynomial in the length of the input. It would be lovely if we could distinguish between problems whose best algorithm took, say n^3 time from those that could be solved in, say n^2 time. But no theory has ever been devised to be that precise for general problems, although the best running time for some problems is known. Fortunately, we can learn a lot by dividing problems into those that can be solved in some polynomial amount of time from those that apparently require exponential time. We’ll give you the tools to do this. And Turing machines are the tool you need to build this theory as well as the theory of what can be computed at all.

Text (Not Required) Hopcroft, Motwani, Ullman, Automata Theory, Languages, and Computation 3rd Edition.

The Central Concepts of Automata Theory Cpt S 317: Spring 2009 The Central Concepts of Automata Theory School of EECS, WSU

Alphabet An alphabet is a finite, non-empty set of symbols Cpt S 317: Spring 2009 Alphabet An alphabet is a finite, non-empty set of symbols We use the symbol ∑ (sigma) to denote an alphabet Examples: Binary: ∑ = {0,1} All lower case letters: ∑ = {a,b,c,..z} Alphanumeric: ∑ = {a-z, A-Z, 0-9} DNA molecule letters: ∑ = {a,c,g,t} … School of EECS, WSU

Strings A string or word is a finite sequence of symbols chosen from ∑ Cpt S 317: Spring 2009 Strings A string or word is a finite sequence of symbols chosen from ∑ Empty string is  (or “epsilon”) Length of a string w, denoted by “|w|”, is equal to the number of (non- ) characters in the string E.g., x = 010100 |x| = 6 x = 01 0 1 00  |x| = ? xy = concatenation of two strings x and y School of EECS, WSU

Cpt S 317: Spring 2009 Languages A language L is a set of strings over a given alphabet ∑ with some rules Example L be the language of all strings with equal number of 0’s and 1’s We will use/construct automatons to decide a given string is an element of the given language or not School of EECS, WSU