Programming Languages Wrap-up

Your Toolkit Object-oriented Imperative Functional Logic

Your problem… Make a GUI front end for a store inventory system Go through a file and count occurrences of certain words Given a graph, find all paths from node A to node B.

Making a new language Why? What do you need to decide?

Theory Component Provides mathematical models of –Programming languages (grammars) –Computers (automata) –And how they work together Gives foundation for studying what programming languages can and can’t do.

Turing Machines Read input letter & tape letter, write tape letter, move left or right.

Example: a n b n c n Read an a, Replace it with an x. Move right over the rest of the a’s. Read a b, replace it with a y. Move right over the rest of the b’s Read a c, replace it with a z. Move all the way left until you see an x (then you’re at the beginning again). Repeat until all a’s have been taken. Make one last pass over string to make sure only x,y, and z are remaining.

Turing’s Thesis Can Turing Machines do anything computers can do? Hmm - depends on the definition of “computer”! Turing’s thesis(mid-30’s): Any computation carried out by mechanical means can be carried out by a Turing Machine.

Turing’s thesis: true or false? Can’t prove it, because we can’t foresee what “mechanical means” people might come up with in the future. Could disprove it with one counterexample...

Evidence Supporting Turing’s thesis Anything that can be done on an existing digital computer can be done on a Turing Machine –We can prove this - just write a TM for each machine instruction. No one has been able to suggest a problem solvable by an algorithm but not a TM Many alternative models of digital computers have been proposed - but none are more powerful than TMs

Algorithm definition An algorithm for a function is a Turing Machine that halts with the correct answer on the tape for any input in the domain.

Other types of automata Given another type of automata, how do you know which is more powerful? Simulation - show how anything done with one type of automata can be done with the other Example: I can simulate with a PDA any FSA. Therefore PDAs as powerful (maybe more) than FSAs.

Example: Any FSA can be written as a PDA: So PDAs are at least as powerful.

Variations on TMs Add a stay-option Tape is only unbounded in one direction Separate read-only tape for input (off-line TM) Multiple storage tapes Multi-dimensional tape (extends infinitely in 2 dimensions) Non-deterministic TMs (not presently possible with today’s computers) All these are equivalent automata to Turing Machines!

Non-deterministic TMs An interesting class of TMs... Equivalent in power to TMs (so everything computable by a ND-TM is computable by a TM) BUT, it takes more steps on a TM than a ND-TM. We think it takes exponentially more steps... This is where the classes P and NP come from... –P: set of all problems computable by a TM in polynomial time –NP: set of all problems computable by a ND-TM in polynomial time –P = NP? Unknown...

Detour into NP-Complete: set of all problems in NP that are polynomial-time reducible to ALL problems in NP. Meaning, if you can solve 1 NPC problem in polynomial time, you can solve ALL NP problems in polynomial time, and P = NP Cook’s Theorem: SAT is NP-Complete How did this work? He showed that every ND-TM has an equivalent SAT expression!! From that it is easy to reduce other NP-problems to SAT to prove that they too are NPC: –Hamiltonian paths (Traveling salesman) –Cliques

Recursively Enumerable Languages A language is recursively enumerable if there is a Turing Machine that accepts it. This allows the TM to go into an infinite loop if the string is not in the language.

Recursive Languages A language is Recursive if there is a TM that accepts the language and it halts on every string in Σ+. That is, a recursive language has to have a membership algorithm.

Recursively Enumerable, contd Not all languages are recursively enumerable. Some languages are recursively enumerable but not recursive. Unrestricted grammars produce recursively enumerable languages. These statements are proven, yet examples are hard to come by...

Context-Sensitive Grammars All productions are of the form x -> y, where x and y are elements of (VUT)+ and |x| <= |y| Example: S -> ABC AB -> aaab C -> DF DF -> cccab

Context-Sensitive Languages CSL example: a n b n c n Automata for CSLs are linear-bounded automata: TM’s where only the area of tape used by the input is allowed as storage. All CSL’s are recursive, but not all recursive languages are context-sensitive.

Chomsky’s Hierarchy Regular Det Context-Free Non-det Context-Free Context-Sensitive Recursive Recursively Enumerable

So can anything be computed? No! Computability – the study of which problems can and which cannot be solved by a Turing Machine.

The Halting Problem Given an algorithm decide will it halt? Assume it can be done: Halt(M,w) = yes if algorithm (TM) M halts on input w, and no if it doesn’t. That is: Halt(M,w) = yes if M(w) halts, else no. Create new algorithm: Strange(String s): if Halt(s,s)= “no” then return yes else while(true){ }

Halting Problem Strange(String s): if Halt(s,s)= “no” then return yes else while(true){ } So, for any program P, if P(P) halts then Strange(P) doesn’t halt. if P(P) doesn’t halt then Strange(P) does halt. But that means: if Strange(Strange) halts then Strange(Strange) doesn’t halt. if Strange(Strange) doesn’t halt then Strange(Strange) does halt. Contradiction!

Computability So, not all problems can be solved with a computer. The Halting Problem is said to be “undecidable”. DO NOT confuse this with an NP- complete problem! Remember, problems in NP are solvable by non-deterministic TMs and therefore also solvable by TMs, just not necessarily in polynomial time.

Reducing one problem to another... The state-entry problem: Given a TM and string w, tell whether or not a given state q is ever entered when processing w. Is it decidable? Why or why not?

Other Undecidable Problems Given a TM M, letter a, and string w, determine if the letter a ever gets written when processing M(w). Determine if two TMs are equal Determine if a CFG is ambiguous Given 2 CFGs, G1 and G2, determine if L(G1)  L(G2) = {}

Rice’s Theorem Any non-trivial property of a recursively enumerable language is undecidable. Particularly, the following are undecidable. –For language G, is L(G) = {}? –For language G, is string w in L(G)? –Is TM M finite? –Does TM M accept 2 different strings of the same length?

The moral of the story... Researchers first find the properties of the problem before trying to solve it. Know beforehand the possibilities of the best answer, or even that an answer is possible. Often leads to a definition of assumptions or concessions to make the problem solvable in a more practical way.