Introduction to formal languages and automata

What the course is about… A Models of computation: What defines a computer and how does it work? What is a computer language? There are automata which are less powerful than our computers – What are they used for? Is there anything more powerful than a computer? B What are the limits of computing? What can we compute in principle? What can we compute in practice? This course is about learning to think abstractly, in a different way than you usually do.

TOC overview The course discusses two very closely related concepts: models of computing and languages. The computing models are simplified as much as possible, so as to reduce them to their most important elements. Some of the models will be more powerful than others. Languages are effectively sets of strings which are built by particular rules, called a grammar. The more complex the grammar, the more complicated the language can be.

Models of computing Finite state machines (automata) Pattern recognition Simple circuits (e.g. elevators, sliding doors) Automata with stack memory (pushdown autom.) Parsing computer languages Automata with limited tape memory Automata with infinite tape memory Called “Turing machines” Most powerful model possible Capable of solving anything that is solvable

Chomsky hierarchy of grammars Regular grammars Context free grammars Context sensitive grammars Unrestricted grammars We’ll define what these mean later, but the important point is that the grammars become more complex as we go down the list, and contain those above it.

Computers recognise languages! Computers can be made to recognize, or accept, the strings of a language. There is a correspondence between the power of the computing model and the complexity of languages that it can recognize! Finite automata only accept regular grammars. Push down automata can also accept context free grammars. Turing machines can accept all grammars. This is why we study both in this course!

The course structure Introduction Regular languages Regular Expressions Finite Automata (finite state machines) Context free languages and pushdown automata Turing Machines Undecidability – Universal Turing Machines

Introduction Computer Science is a cluster of related scientific and engineering disciplines concerned with the study and application of computations Partitioned into two sub-disciplines: Theory of Computation, Theory of Programming.

Introduction(Contd.) The Theory of Computation -understanding the nature of computation, and specifically the inherent possibilities and limitations of efficient computations. The Theory of Programming - concerned with the actual task of implementing computations (i.e., writing computer programs). .

Introduction(Contd.) The Theory of Computation -understanding the nature of computation, and specifically the inherent possibilities and limitations of efficient computations. The Theory of Programming - concerned with the actual task of implementing computations (i.e., writing computer programs). .

Theory of Computation scientific discipline concerned with the study of general properties of computation be it natural, man-made, or imaginary. Most importantly, it aims to understand the nature of efficient computation. Theory of computation is a course of abstractions about what we can compute. sub-divided to complexity theory - computational resources algorithms - the tasks to be solved

Theory of Computation(Contd.) Though this is not a "math" course, it does make significant use of mathematical symbols, abstractions, definitions, theorems, proofs, lemmas, corollaries, logical reasoning, inductive proofs, and the like. Such concepts are difficult, but rewarding.

History of universality In 1500s, the introduction of symbolic formulas and algebraic variables. But the notion of abstract functions in mathematics reached its modern form only near the end of the 1800s Gottfried Leibniz in the 1600s considered the idea of setting up a universal language based on logic that would provide a precise description analogous to a mathematical proof of any fact or process. While Leibniz considered the possibility of checking his descriptions by machine. In the 1830s Charles Babbage and Ada Lovelace noted that a similar approach could be used to specify the mathematical procedure to be followed by a mechanical calculating machine

History of universality(Contd.) Starting in the 1880s attempts to build up both numbers and the operations of arithmetic from logic and set theory in the 1920s work by Moses Schönfinkel on combinators and by Emil Post on string rewriting systems provided fairly concrete examples of this. In 1930 Kurt Gödel used the same basic idea to set up Gödel numbers to encode logical and other procedures as numbers. But in 1935 Alonzo Church constructed a system in lambda calculus that he showed could be made to emulate any other system in lambda calculus in 1936 Alan Turing did the same thing for Turing machines

History of universality(Contd.) Turing’s results were used in the 1940s - notably in the work of Warren McCulloch and Walter Pitts John von Neumann’s by the late 1940s, practical computer engineering had also been led to the idea of storing programs - like data - electronically, and in the 1950s it became widely understood that general-purpose practical computers could be Viewed as universal systems. Many theoretical investigations of universality were made in the 1950s and 1960s, but gradually the emphasis shifted more towards issues of languages and algorithms.

Automaton What can be computed? Consider a very simple model of a problem. A problem is a set of strings (often binary strings) that is to be distinguished. If we have an algorithm to do this, then we have solved the problem. These kinds of problems are called decidable. Consider the set of all syntactic legal Java programs that never go to into an infinite loop on any input. There is no way to solve this problem. This problem is undecidable.

Automaton(Contd.) There are different kinds of machines, each one a little more powerful than the rest. What kinds of sets do different machines recognize? What features can you add to a machine to make it more powerful, which is able to recognize sets that it could not recognize before? Can we measure time and space requirements of a machine as a tradeoff to this power?

Applications Finite state machines are used in string searching algorithms, compiler design, control unit design in computer architecture NP-Complete theory helps us distinguish the tractable from the intractable. In the combinatorics community, and in scientific fields. This collaboration has been extremely beneficial to both the discrete math and theoretical computer science communities, with wealthy exchange of ideas, problems and techniques active research areas like computational number theory, computational algebra and computational group theory.

Applications(Contd.) the convergence analysis of some Monte Carlo algorithms used by statistical physicists, and the growing work on computational biology. Yet another exciting area of collaboration, where foundational work on theoretical computer science side goes hand-in-hand with experimental work in physics, is quantum computing. research area of computational learning - stock-market share values, DNA sequences, neuron recordings of  animals reacting to stimuli, etc., solve computer science problems, as multitudes of autonomous robots, or of independent programs on the Web, have to be programmed to function in adversarial environments.