Algorithm Analysis

Assignment #8 Submit in PDF format ONLY !!! Assignment will NOT be accepted in format other than pdf. See email and blackboard announcements for places to find software for pdf conversion. engineering (ECJ) labs undergraduate library. There are also websites that let you do this for free e.g.- http://www.pdfonline.com/ http://createpdf.adobe.com/ Word doc files get corrupted in the submission system so they will not be accepted for this assignment.

Final Exam Course:E E 312 INTRODUCTION TO PROGRAMMING Instructor:BARBER, K WEDNESDAY, MAY 10, 9-12 N Exam Room:      JES A121A  

Algorithm Analysis Estimating the time required for a program Determining a relative computation time among programs Understanding the limits of computation The Halting Problem Classifying particularly hard problems NP-Complete problems

Understanding the limits of computation

Computability As we have seen, typical algorithms run the gamut from O(log n) to O(n3), where problems solved using O(n) or O(log n) are considered relatively easy On the other end of the spectrum are truly hard problems: Impossible ("undecidable") problems Very hard (NP-Complete) problems

Undecidable Problems [http://www.cgl.uwaterloo.ca/~csk/halt/] An algorithm is a solution to a problem if it correctly provides the appropriate answer to the problem and is guaranteed to always run in a finite amount of time. A problem is decidable if it has a solution. If there is no algorithm that solves the problem in a finite amount of time, the problem is undecidable.

Generally stated Halting Problem [wikipedia] In computability theory the halting problem is a decision problem which can be informally stated as follows: Given a description of a program and its initial input, determine whether the program, when executed on this input, ever halts (completes). The alternative is that it runs forever without halting. Alan Turing proved in 1936 that a general algorithm to solve the halting problem for all possible inputs cannot exist. We say that the halting problem is undecidable.

Impossible "Undecidable" Problems The Halting Problem Probably the most commonly cited "undecidable" problem Problem: Is it possible for your C compiler to have an extra feature that checks not only syntax errors but also checks for infinite loops? Answer: NO! Intuitive Reason: Such a program might have a hard time checking itself. Suggests the term "recursively undecidable"

Informal Proof of the Halting Problem If an infinite loop-checking program could be written, it should be able to check itself. Assume a program called Loop(P) Loop takes as input a program P, which takes itself as input (i.e., P(P) ) Loop returns "Yes" if P loops when run on itself Loop goes into an infinite loop if P terminates when run on itself (instead of returning "No")

Informal Proof of the Halting Problem (cont'd) What if P = Loop (i.e., input to Loop is Loop itself) ? By definition, Loop either halts or does not halt However, both these possibilities lead to contradictions.

Informal Proof of the Halting Problem (cont'd) Suppose P = Loop and we were to run Loop(Loop) Case 1: If P(P) terminates then Loop(P) should go into an infinite loop However, then we would have a contradiction: Loop(Loop) would both terminate and go into an infinite loop at the same time Case 2: If P(P) goes into an infinite loop then Loop(P) should terminate Once again, Loop(Loop) would go into an infinite loop and terminate at the same time Therefore, Loop cannot exist. Assume a program called Loop(P) Loop takes as input a program P, which takes itself as input (i.e., P(P) ) Loop returns "Yes" if P loops when run on itself Loop goes into an infinite loop if P terminates when run on itself (instead of returning "No")

Another informal proof for the Halting problem Now suppose that someone claims to have found an algorithm halt(p, i) that returns true if p describes the name of a program that halts when given as input the string I false otherwise. Construct another program trouble(s) that uses halt as a subroutine: function trouble(string s) if halt(s, s) == false return true else loop forever

Another informal proof for the Halting problem function trouble(string s) if halt(s, s) == false return true else loop forever All programs can be represented by strings, there is a string t that represents the program trouble. Does trouble(t) halt? Consider both cases: If trouble(t) halts, it must be because halt(t, t) returned false, but that would mean that trouble(t) should not have halted. If trouble(t) runs forever, it is either because halt itself runs forever, or because it returned true. This would mean either that halt does not work for every valid input, or that trouble(t) should have halted. Either case concludes that halt did not give a correct answer, contrary to the original claim. Since the same reasoning applies to any program that someone might offer as a solution to the halting problem, there can be no solution.

Pitfalls in Understanding [wikipedia] Would you ask, “Is there an algorithm that can return a third option for some programs, such as "undecidable" or "would lead to a contradiction." ? This reflects a misunderstanding of decidability. It is easy to construct one algorithm that always answers "halts" and another that always answers "doesn't halt." For any specific program and input, one of these two algorithms answers correctly, even though nobody may know which one. The difficulty of the halting problem lies not in particular programs, but in the requirement that a solution must work for all programs.

Other Reference for the Halting Problem http://plus.maths.org.uk/issue5/turing/

Classifying particularly hard problems

NP problems Computational complexity theory is part of the theory of computation dealing with the resources required during computation to solve a given problem. time (how many steps it takes to solve a problem) space (how much memory it takes to solve a problem). the class P consists of all those decision problems that can be solved on a deterministic sequential machine in an amount of time that is polynomial in the size of the input; the class NP consists of all those decision problems whose positive solutions can be verified in polynomial time given the right information, or equivalently, whose solution can be found in polynomial time on a non-deterministic machine.

Polynomial Time polynomial time refers to the computation time of a problem where the time, m(n), is no greater than a polynomial function of the problem size, n. Written mathematically, m(n) = O(nk) where k is a constant (which may depend on the problem). Polynomial time is the smallest time-complexity class on a deterministic machine which is robust in terms of machine model changes, and is the smallest class closed under composition of subproblems. Subclasses of polynomial time Linear time: O(n) Quadratic time: O(n2) Cubic time: O(n3)

Very hard (NP-Complete) problems Problems in class NP (nondeterministic polynomial-time) Not impossible, but still very hard A "Nondeterministic computer" is a theoretical construct A deterministic machine executes an instruction and goes to the next instruction, which is unique A nondeterministic machine is free to "guess" the next instruction (and guess very well) Can't build a nondeterministic computer, but still a useful theoretical construct

Problems in the Class NP Phrase the problem as a yes/no question. The problem is in NP if, in polynomial time, we can prove that any "yes" instance is correct. We don't have to worry about "no" instances because the program always makes the "right" choice. Includes all problems that have polynomial-time solutions.

NP-complete problems [wikipedia] In complexity theory, the NP-complete problems are the most difficult problems in NP ("non-deterministic polynomial time") in the sense that they are the ones most likely not to be in P. The complexity class consisting of all NP-complete problems is sometimes referred to as NP-C. One example of an NP-complete problem is the subset sum problem which is: given a finite set of integers, determine whether any non-empty subset of them sums to zero. A supposed answer is very easy to verify for correctness, but no one knows a significantly faster way to solve the problem than to try every single possible subset, which is very slow.

NP-Complete NP-complete problems are a subset of all problems known to be in NP that are the hardest. An NP-complete problem has the property that any problem in NP can be reduced to it. Reducing a problem P1 to P2 Provide a mapping to transform an instance of P1 to P2 Solve P2 Map the P2 answer back to a P1 answer Therefore, an NP-complete problem can be used as a "subroutine" to solve any problem in NP If any NP-complete problem can be solved in polynomial time, then every problem in NP must have a polynomial time solution

Example NP-Complete Problems Hamiltonian Cycle problem Find an ordering of the vertices in an undirected graph such that each vertex is visited exactly once.

Example NP-Complete Problems (cont'd) Traveling Salesman Given a complete graph with edge costs, and an integer K, is there a simple cycle that visits all vertices and has total cost ≤ K?

Example NP-Complete Problems [wikipedia] The Boolean satisfiability problem (SAT) is a decision problem considered in complexity theory. An instance of the problem is a Boolean expression written using only AND, OR, NOT, variables, and parentheses. The question is: given the expression, is there some assignment of TRUE and FALSE values to the variables that will make the entire expression true? In mathematics, a formula of propositional logic is said to be satisfiable if truth-values can be assigned to its variables in a way that makes the formula true. The class of satisfiable propositional formulas is NP-complete.

Example NP-Complete Problems Satisfiability Takes as input a boolean expression and asks whether the expression has an assignment to the variable that gives a value of 1.

Transforming Hamiltonian Cycle into Traveling Salesman Construct a new graph GTS which has same vertices as GH. For GTS, each edge has weight of 1 if it was in GH, and 2 otherwise. Choose K = |V|. If GH has a Hamiltonian cycle if and only if GTS has a traveling salesman tour of total weight |V|.

Conclusion As an engineer developing software, you must… Know the limits of computation when attacking a problem. Is the problem decidable? Is your algorithm decidable? Be able to analyze the efficiency of your algorithms. What is the algorithm's Big-Oh growth with respect to input? Understand the relative difficulty of the problem you're trying to solve by comparing to problems with known difficulty. Can the problem be related to an NP-Complete problem?

References "The Stony Brook Algorithm Repository," <http://www.cs.sunysb.edu/~algorith/> Data Structures and Algorithm Analysis in C, Weiss