1. Algorithm Analysis

2. Assignment #8 Submit in PDF format ONLY !!!
Assignment will NOT be accepted in format other than pdf.
See and blackboard announcements for places to find software for pdf conversion.
engineering (ECJ) labs
undergraduate library.
There are also websites that let you do this for free e.g.-
Word doc files get corrupted in the submission system so they will not be accepted for this assignment.

3. Final Exam Course:E E 312 INTRODUCTION TO PROGRAMMING
Instructor:BARBER, K
WEDNESDAY, MAY 10, 9-12 N
Exam Room:      JES A121A  

4. Algorithm Analysis Estimating the time required for a program
Determining a relative computation time among programs
Understanding the limits of computation
The Halting Problem
Classifying particularly hard problems
NP-Complete problems

5. Understanding the limits of computation

6. Computability
As we have seen, typical algorithms run the gamut from O(log n) to O(n3), where problems solved using O(n) or O(log n) are considered relatively easy
On the other end of the spectrum are truly hard problems:
Impossible ("undecidable") problems
Very hard (NP-Complete) problems

7. Undecidable Problems [http://www.cgl.uwaterloo.ca/~csk/halt/]
An algorithm is a solution to a problem if it correctly provides the appropriate answer to the problem and is guaranteed to always run in a finite amount of time.
A problem is decidable if it has a solution.
If there is no algorithm that solves the problem in a finite amount of time, the problem is undecidable.

8. Generally stated Halting Problem [wikipedia]
In computability theory the halting problem is a decision problem which can be informally stated as follows:
Given a description of a program and its initial input, determine whether the program, when executed on this input, ever halts (completes). The alternative is that it runs forever without halting.
Alan Turing proved in 1936 that a general algorithm to solve the halting problem for all possible inputs cannot exist. We say that the halting problem is undecidable.

9. Impossible "Undecidable" Problems The Halting Problem
Probably the most commonly cited "undecidable" problem
Problem: Is it possible for your C compiler to have an extra feature that checks not only syntax errors but also checks for infinite loops?
Answer: NO!
Intuitive Reason: Such a program might have a hard time checking itself.
Suggests the term "recursively undecidable"

10. Informal Proof of the Halting Problem
If an infinite loop-checking program could be written, it should be able to check itself.
Assume a program called Loop(P)
Loop takes as input a program P, which takes itself as input (i.e., P(P) )
Loop returns "Yes" if P loops when run on itself
Loop goes into an infinite loop if P terminates when run on itself (instead of returning "No")

11. Informal Proof of the Halting Problem (cont'd)
What if P = Loop (i.e., input to Loop is Loop itself) ?
By definition, Loop either halts or does not halt
However, both these possibilities lead to contradictions.

12. Informal Proof of the Halting Problem (cont'd)
Suppose P = Loop and we were to run Loop(Loop)
Case 1:
If P(P) terminates then Loop(P) should go into an infinite loop
However, then we would have a contradiction:
Loop(Loop) would both terminate and go into an infinite loop at the same time
Case 2:
If P(P) goes into an infinite loop then Loop(P) should terminate
Once again, Loop(Loop) would go into an infinite loop and terminate at the same time
Therefore, Loop cannot exist.
Assume a program called Loop(P)
Loop takes as input a program P, which takes itself as input (i.e., P(P) )
Loop returns "Yes" if P loops when run on itself
Loop goes into an infinite loop if P terminates when run on itself (instead of returning "No")

13. Another informal proof for the Halting problem
Now suppose that someone claims to have found an algorithm halt(p, i) that returns
true if p describes the name of a program that halts when given as input the string I
false otherwise.
Construct another program trouble(s) that uses halt as a subroutine:
function trouble(string s)
if halt(s, s) == false
return true
else
loop forever

14. Another informal proof for the Halting problem
function trouble(string s)
if halt(s, s) == false
return true
else
loop forever
All programs can be represented by strings, there is a string t that represents the program trouble. Does trouble(t) halt?
Consider both cases:
If trouble(t) halts, it must be because halt(t, t) returned false, but that would mean that trouble(t) should not have halted.
If trouble(t) runs forever, it is either because halt itself runs forever, or because it returned true. This would mean either that halt does not work for every valid input, or that trouble(t) should have halted.
Either case concludes that halt did not give a correct answer, contrary to the original claim.
Since the same reasoning applies to any program that someone might offer as a solution to the halting problem, there can be no solution.

15. Pitfalls in Understanding [wikipedia]
Would you ask, “Is there an algorithm that can return a third option for some programs, such as "undecidable" or "would lead to a contradiction." ?
This reflects a misunderstanding of decidability.
It is easy to construct one algorithm that always answers "halts" and another that always answers "doesn't halt."
For any specific program and input, one of these two algorithms answers correctly, even though nobody may know which one.
The difficulty of the halting problem lies not in particular programs, but in the requirement that a solution must work for all programs.

16. Other Reference for the Halting Problem

17. Classifying particularly hard problems

18. NP problems
Computational complexity theory is part of the theory of computation dealing with the resources required during computation to solve a given problem.
time (how many steps it takes to solve a problem)
space (how much memory it takes to solve a problem).
the class P consists of all those decision problems that can be solved on a deterministic sequential machine in an amount of time that is polynomial in the size of the input;
the class NP consists of all those decision problems whose positive solutions can be verified in polynomial time given the right information, or equivalently, whose solution can be found in polynomial time on a non-deterministic machine.

19. Polynomial Time
polynomial time refers to the computation time of a problem where the time, m(n), is no greater than a polynomial function of the problem size, n.
Written mathematically, m(n) = O(nk) where k is a constant (which may depend on the problem).
Polynomial time is the smallest time-complexity class on a deterministic machine which is robust in terms of machine model changes, and is the smallest class closed under composition of subproblems.
Subclasses of polynomial time
Linear time: O(n)
Quadratic time: O(n2)
Cubic time: O(n3)

20. Very hard (NP-Complete) problems
Problems in class NP (nondeterministic polynomial-time)
Not impossible, but still very hard
A "Nondeterministic computer" is a theoretical construct
A deterministic machine executes an instruction and goes to the next instruction, which is unique
A nondeterministic machine is free to "guess" the next instruction (and guess very well)
Can't build a nondeterministic computer, but still a useful theoretical construct

21. Problems in the Class NP
Phrase the problem as a yes/no question. The problem is in NP if, in polynomial time, we can prove that any "yes" instance is correct.
We don't have to worry about "no" instances because the program always makes the "right" choice.
Includes all problems that have polynomial-time solutions.

22. NP-complete problems [wikipedia]
In complexity theory, the NP-complete problems are the most difficult problems in NP ("non-deterministic polynomial time") in the sense that they are the ones most likely not to be in P.
The complexity class consisting of all NP-complete problems is sometimes referred to as NP-C.
One example of an NP-complete problem is the subset sum problem which is: given a finite set of integers, determine whether any non-empty subset of them sums to zero.
A supposed answer is very easy to verify for correctness, but no one knows a significantly faster way to solve the problem than to try every single possible subset, which is very slow.

23. NP-Complete
NP-complete problems are a subset of all problems known to be in NP that are the hardest.
An NP-complete problem has the property that any problem in NP can be reduced to it.
Reducing a problem P1 to P2
Provide a mapping to transform an instance of P1 to P2
Solve P2
Map the P2 answer back to a P1 answer
Therefore, an NP-complete problem can be used as a "subroutine" to solve any problem in NP
If any NP-complete problem can be solved in polynomial time, then every problem in NP must have a polynomial time solution

24. Example NP-Complete Problems
Hamiltonian Cycle problem
Find an ordering of the vertices in an undirected graph such that each vertex is visited exactly once.

25. Example NP-Complete Problems (cont'd)
Traveling Salesman
Given a complete graph with edge costs, and an integer K, is there a simple cycle that visits all vertices and has total cost ≤ K?

26. Example NP-Complete Problems [wikipedia]
The Boolean satisfiability problem (SAT) is a decision problem considered in complexity theory. An instance of the problem is a Boolean expression written using only AND, OR, NOT, variables, and parentheses. The question is: given the expression, is there some assignment of TRUE and FALSE values to the variables that will make the entire expression true?
In mathematics, a formula of propositional logic is said to be satisfiable if truth-values can be assigned to its variables in a way that makes the formula true. The class of satisfiable propositional formulas is NP-complete.

27. Example NP-Complete Problems
Satisfiability
Takes as input a boolean expression and asks whether the expression has an assignment to the variable that gives a value of 1.

28. Transforming Hamiltonian Cycle into Traveling Salesman
Construct a new graph GTS which has same vertices as GH. For GTS, each edge has weight of 1 if it was in GH, and 2 otherwise. Choose K = |V|.
If GH has a Hamiltonian cycle if and only if GTS has a traveling salesman tour of total weight |V|.

29. Conclusion As an engineer developing software, you must…
Know the limits of computation when attacking a problem.
Is the problem decidable?
Is your algorithm decidable?
Be able to analyze the efficiency of your algorithms.
What is the algorithm's Big-Oh growth with respect to input?
Understand the relative difficulty of the problem you're trying to solve by comparing to problems with known difficulty.
Can the problem be related to an NP-Complete problem?

30. References
"The Stony Brook Algorithm Repository," <
Data Structures and Algorithm Analysis in C, Weiss