1. Welcome to the Automata Theory Course
Why Study Automata?
What the Course is About

2. Why Study Automata?
A survey of Stanford grads 5 years were out asked which of their courses did they use in their job.
Basics like intro-programming took the top spots, of course.
But among optional courses, Automata stood remarkably high.
3X the score for AI, for example.

3. How Could That Be? Regular expressions are used in many systems.
E.g., UNIX a.*b.
E.g., DTD’s describe XML tags with a RE format like person (name, addr, child*).
Finite automata model protocols, electronic circuits.
So let’s see some of the ideas we’re going to learn about in the course and how they appear in practice.
Click 1
One very commonly used idea is the regular expression, a simple notation for describing many of the patterns that arise naturally in practice. Many pieces of software that you might find yourself working on in the future need a simple input language to describe patterns, so you might well find yourself implementing some form of regular expressions.
Click 2
For example, many UNIX text-processing commands use a variety of regular expressions. This expression describes a line of text that has a letter “a”, followed by any number of characters, followed by a letter “b”.
Click 3
For a more modern example, the XML document markup language invites us to describe the structure of documents by a DTD, or document type definition. The DTD language consists of descriptions of elements such as the example given here. A person element consists of a name element followed by an address element, followed by any number of child elements.
Click 4
Finite automata are another topic we’ll see early on. They are, in fact the way regular-expression-based languages are implemented. They also have been used for decades to model electronic circuits, and in particular, to help design good circuits. They have also been used to model protocols, and we’ll give some examples later in the course. Especially, finite automata underlie the body of techniques known as “model checking,” which has been used to verify the correctness of both communication protocols and large electronic circuits.

4. How? – (2)
Context-free grammars are used to describe the syntax of essentially every programming language.
Not to forget their important role in describing natural languages.
Click 1
Another important aspect of the course is context-free grammars. These are used to put a tree structure on strings, typically text strings, according to some recursive rules. They are an essential for describing the syntax of programming languages, and are a part of every compiler. They also play an important role in describing the syntax of natural languages, and are used in software that does machine translation and other natural-language processing.
Click 2
And the DTD’s we mentioned in connection with regular expressions are really context-free grammars. It is single rules of a DTD that look like regular expressions.

5. How? – (3)
When developing solutions to real problems, we often confront the limitations of what software can do.
Undecidable things – no program whatever can do it.
Intractable things – there are programs, but no fast programs.
Automata theory gives you the tools.
The topics we just mentioned are essentially tools for doing simple but important things. But there is a second broad theme in this course.
Click 1
There are fundamental limitations on our ability to compute. A computer scientist should be aware of these limitations, because only then can you avoid spending time attempting something that is impossible.
Click 2
One limitation is undecidability. There are problems that cannot be solved by computation. For example, you might imagine that you could write a compiler that would refuse to compile programs that printed out dirty words. Even assuming you had a precise definition of what words we “dirty,” you can’t do this. We’re going to prove that there is no way to tell whether a program will ever print a particular word, or even whether it will ever print anything at all.
Click 3
And we also need to know about the class of problems called intractable. These are, colloquially, problems that we can solve, but whose solution takes time that is exponential in the input size. These problems generally need to be finessed in some way, such as by approximating the solution. The reality of the theory of intractability is a bit different from the colloquial version. While the undecidable problems have been proved not to have any solution, for the intractable problems we have very strong evidence that they require exponential time, but no proof. We’ll explain all this when we get to NP-completeness as the culmination of the course.
Click 4
So another thing you will take away from this course is the ability to navigate the space of problems that you might encounter in a life of creative software construction. You will learn how to determine that a problem is undecidable and how to determine that it is intractable. That lets you avoid the problem altogether in the first case and to modify your approach in the second case.

6. Other Good Stuff
We’ll learn how to deal formally with discrete systems.
Proofs: You never really prove a program correct, but you need to be thinking of why a tricky technique really works.
We’ll gain experience with abstract models and constructions.
Models layered software architectures.
There are several less concrete benefits to the course.
Click 1
First, you will improve your skills at proving facts, especially inductive proofs, of which we’ll do several in great detail. I’m not one of the people who thinks that formal proofs of programs will ever be a serious software methodology. But as you construct code, you should have a sense of why what you’re doing works the way it’s supposed to. Often, the trickiest parts of a program deal with trees, graphs, or other recursive structures. Understanding inductive proofs lets you at least formulate a reason why you think your method works, even if you don’t try to dot the “I’s” in a formal proof.
Click 2
We’re also going to learn about a number of important abstractions – finite automata, regular expressions, context-free-grammars, and varieties of pushdown automata and Turing machines. Some of the essential parts of this course are proving equivalences among the models – that is, any example of one model can be simulated by some instance of another model. The process of simulation across models is essentially the same as the modern approach to programming in layered abstractions, where you write programs at one layer using the primitives of the layer below.

7. Automata Theory – Gateway Drug
This theory has attracted people of a mathematical bent to CS, to the betterment of all.
Ken Thompson – before UNIX was working on compiling regular expressions.
Jim Gray – thesis was automata theory before he got into database systems and made fundamental contributions there.
Click 1
A number of people taking the automata course were not computer scientists at all, but were mathematicians by inclination. That’s cool. Welcome to those of you out there. I probably won’t do things quite as formally as you would like, but more formally than the typical computer scientist likes. However, be warned that some in the past have found the subject sufficiently interesting that they saw the light and made major contributions to computer software.
Click 2
A case in point is Ken Thompson, the fellow who gave us UNIX. Before doing UNIX, Ken developed an interest in regular expressions, and saw that they were an important part of a text editor. He worked out efficient ways to compile regular expressions into programs that could process text, and his algorithms are an important part of what we teach about the subject today. It should be no surprise that regular expressions form an integral part of so many UNIX commands, and that these commands gave more flexibility and power to UNIX users than did earlier operating systems.
Click 3
Another interesting case is Jim Gray. Jim, before his mysterious dissapearance, gave us many important ideas in the database field, including two-phase locking for concurrency control, for which he won the Turing award.
But I knew Jim when he was a student at Berkeley and I was on leave there for a brief period. He did a thesis on two-way pushdown automata. We’re not going to talk about them. They turn out to have been a backwater of the theory, and I’m sure Jim would agree. They are related to ordinary pushdown automata, which we will talk about in connection with context-free grammars. But that’s not important.
What is interesting is that Gray told me, quite a bit later, that he decided to become a computer scientist because automata theory intrigued him. Only later did he switch into database systems. And I believe that the experience he got dealing formally with automata make him more capable as a designer of several very innovative systems.

8. Course Outline Regular Languages and their descriptors:
Finite automata, nondeterministic finite automata, regular expressions.
Algorithms to decide questions about regular languages, e.g., is it empty?
Closure properties of regular languages.
So here’s a summary of what will be covered in the course.
Click 1
We’ll start off with what are called regular languages. A language is just a set of strings, for example the set of character strings that are valid Java programs.
Click 2
The regular languages are exactly those sets of strings that can be described by finite automata or regular expressions. This discussion will also introduce the important concept of nondeterminism – machines that can magically turn into many machines that each do something independently, but with a coordinated effect. This model of computation is fanciful to say the least, but we’ll see it plays a really important role in several places, including design of algorithms and in understanding the theory of intractable problems.
Click 3
We’re then going to turn to properties of the regular languages. These properties include the ability to answer certain questions about finite automata and regular expressions that we cannot decide about programs in general. An example would be to tell whether a device makes an output in response to even one input. You can’t tell for programs in general, but you can for finite automata. It is this tractability – our ability to understand what simple formalisms like finite automata or regular expressions do that make them so valuable when they can be used.
Click 4
We’re also going to talk about closure properties of regular languages. For example, the union or intersection of two regular languages is also a regular language.

9. Course Outline – (2) Context-free languages and their descriptors:
Context-free grammars, pushdown automata.
Decision and closure properties.
Click 1
The next big topic will be context-free languages. This is a somewhat larger class of languages than the regular languages, and they enable us to do things that you can’t do with regular languages, such as match balanced parentheses or XML tags.
Click 2
We’ll talk about two ways to describe such languages. First, by context-free grammars, a recursive system for generating strings, and then by pushdown automata, which are a generalization of the finite automata that we use for regular languages.
Click 3
We’ll then repeat our examination of decision properties and closure properties for this larger class of languages. Many of things we can tell about regular languages that we cannot tell in general, we can also tell about context-free languages, but there are unfortunately some exceptions. Similarly, many of the operations under which regular languages are closed also yield a context free language when applied to context-free languages. But again we lose some operations. For example, context-free languages are closed under union, but not under intersection.

10. Course Outline – (3) Recursive and recursively enumerable languages.
Turing machines, decidability of problems.
The limit of what can be computed.
Intractable problems.
Problems that (appear to) require exponential time.
NP-completeness and beyond.
Click 1
Next, we take up the largest class of languages that we can reasonably say can be dealt with by a computer – the recursively enumerable languages. We also look at the smaller but important subset of the recursively enumerable languages, called the recursive languages. These are the languages for which there is an algorithm to tell whether or not a string is in the language.
We introduce the Turing machine, an automaton in the spirit of the first kinds of automata we meet: finite and pushdown automata. The Turing machine is, however, much more powerful than either of these. In fact, it is as powerful as any model that has ever been thought of to answer the question “what can we compute?” The payoff for the study of Turing machines is that we can answer the question of what can be decided by computation, or what can be computed by any means at all. We shall develop tools for showing that certain problems cannot be decided – that is, answered by a computer. The example we already mentioned – does a program print a dirty word, should give you the idea of what we can achieve.
Click 2
Finally, we’re going to cover the theory of NP-completeness. The critical question is what can we do with an algorithm that runs in time that is some polynomial in the length of the input. It would be lovely if we could distinguish between problems whose best algorithm took, say n^3 time from those that could be solved in, say n^2 time. But no theory has ever been devised to be that precise for general problems, although the best running time for some problems is known.
Fortunately, we can learn a lot by dividing problems into those that can be solved in some polynomial amount of time from those that apparently require exponential time. We’ll give you the tools to do this. And Turing machines are the tool you need to build this theory as well as the theory of what can be computed at all.