1. CSC 380: Design and Analysis of Algorithms
Dr. Curry Guinn

2. Quick Info Dr. Curry Guinn CIS 2015 guinnc@uncw.edu
Office Hours: MWF: 10:00am-11:00m and by appointment

3. Outline of today
Limits
Halting problem
P vs NP

4. Lower Bounds (cont.) Lower bound can be
an exact count
an efficiency class ()
Tight lower bound: there exists an algorithm with the same efficiency as the lower bound
Problem Lower bound Tightness
sorting (nlog n) yes
searching in a sorted array (log n) yes
element uniqueness (nlog n) yes
n-digit integer multiplication (n) unknown
multiplication of n-by-n matrices (n2) unknown
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 11 ©2012 Pearson Education, Inc. Upper Saddle River, NJ. All Rights Reserved.

5. Trivial Lower Bounds
Trivial lower bounds: based on counting the number of items that must be processed in input and generated as output
Examples
finding max element
sorting
element uniqueness
Hamiltonian circuit existence
Conclusions
may and may not be useful
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 11 ©2012 Pearson Education, Inc. Upper Saddle River, NJ. All Rights Reserved.

6. Theory of Computers
1936:
Computability (The Halting Problem)
The Turing Machine
The Universal Machine
The Turing machine, computability, universal machine

7. The Halting Problem An example of something that is not computable.
Created by Turing in 1936 to define a problem which no algorithmic procedure can solve.
Can we write a program that will take in a user's program and inputs and decide whether
it will eventually stop, or
it will run infinitely in some infinite loop ?

8. Proof (by contradiction)
Assume that it is possible to write a program to solve the Halting Problem.
Denote this program by Halt(program,input).
Halt (program,input) will
return yes if program will halt on input and
no otherwise.
A program is just a string of characters
E.g. your Java program is just a long string of characters
An input can also be considered as just a string of characters
So Halt is effectively just working on two strings

9. Proof (cont.)
We can now write another program Loopy(program) that uses Halt
The program Loopy(program) does the following:
[1] If Halt (program,program) returns yes,
Loopy will go into an infinite loop.
[2] If Halt(program,program) returns no,
Loopy will halt.

10. Proof (cont.) [1] If Halt (program,program) returns yes,
Loopy will go into an infinite loop.
[2] If Halt (program,program) returns no,
Loopy will halt.
Consider what happens when we run Loopy(Loopy).
If Loopy loops infinitely,
Halt(Loopy,Loopy) return no which by [2] above means Loopy will halt.
If Loopy halts,
Halt (Loopy,Loopy) will return yes which by [1] above means Loopy will loop infinitely.
Conclusion: Our assumption that it is possible to write a program to solve the Halting Problem has resulted in a contradiction.

11. Other non-computable problems
Almost every problem related to the behavior of a program is non-computable. For example:
Showing that 2 programs are equivalent (for each input you always get the same output).
2) Determining whether a given output will occur
3) Determining the correctness of a program
etc.

12. Your boss asks you to find …
A solution:
Can you find a solution -> feasibility
Can you find a cheaper solution -> optimality
Can you find the cheapest solution -> uniqueness
Can you find a cheap solution -> suboptimal
How cheap is your solution -> degree of suboptimality
Why can’t you find a cheap solution -> hardness

13. Your boss asks you to …
Have the computer find the solution of the problem
Is your algorithm fast? -> polynomial time
Is your algorithm slow? -> non-polynomial time

14. Performance Categories of Algorithms
Sub-linear O(Log N)
Linear O(N)
Nearly linear O(N Log N)
Quadratic O(N2)
Exponential O(2N)
O(N!)
O(NN)
Polynomial

15. Reasonable vs. Unreasonable
Reasonable algorithms have polynomial factors
O (Log N)
O (N)
O (NK) where K is a constant
Unreasonable algorithms have exponential factors
O (2N)
O (N!)
O (NN)

16. Effects of Exponents 1050 1045 1040 1035 1030 1025 1020 NN 2N 1015
trillion
billion
million
1000
100 10 NN 2N N5 N
Don’t Care! N

17. Effects of Exponents Consider an 2N algorithm for N of only 256.
Time cost is a number with 78 digits to the left of the decimal.
For comparison:
Number of protons (estimated) in the known universe is a number with 77 digits.

18. What about a Faster Computer?
What if:
Instructions took ZERO time to execute
CPU registers could be loaded at the speed of light
These algorithms are still unreasonable!
The speed of light is only so fast!

19. Relations between Problems, Algorithms, and Programs
Algorithm
Program
Program
Program
Program

20. P vs NP P = Polynomial NP = Non-deterministic Polynomial
What is a non-deterministic algorithm?

21. Classifying Problem Complexity
Is the problem tractable, i.e., is there a polynomial-time (O(p(n)) algorithm that solves it?
Possible answers:
yes (give examples) no
because it’s been proved that no algorithm exists at all (e.g., Turing’s halting problem)
because it’s been be proved that any algorithm takes exponential time
unknown

22. Problems Solvable by Computers …
Some problems can be answered in polynomial time.
This means that the time taken to find the answer is proportional to some polynomial function of the size of the problem.
For example, the time taken to perform a naïve sort on a list is proportional to the square of the size of the list.
This means that if you double the size of the list, the list may take four times as long to order.
Such problems are often called ‘P’ problems.

23. Problem Types: Optimization and Decision
Optimization problem: find a solution that maximizes or minimizes some objective function
Decision problem: answer yes/no to a question
Many problems have decision and optimization versions.
E.g.: traveling salesman problem
optimization: find Hamiltonian cycle of minimum length
decision: find Hamiltonian cycle of length  m
Decision problems are more convenient for formal investigation of their complexity.

24. Class P
P: the class of decision problems that are solvable in O(p(n)) time, where p(n) is a polynomial of problem’s input size n
Examples:
searching
element uniqueness
graph connectivity
graph acyclicity

25. Problems Solvable by Computers …
A term related to ‘P’ is ‘NP’.
NP stands for non-deterministic polynomial.
If a problem is NP, it means that it is possible to guess an answer and determine whether it is correct in polynomial time.
All P problems are also NP. (why?)
P problems are a subset of NP.
It is possible that all NP problems can be reduced in some way to P problems, but this has not been either proved or disproved.

26. Class NP
NP (nondeterministic polynomial): class of decision problems whose proposed solutions can be verified in polynomial time = solvable by a nondeterministic polynomial algorithm
A nondeterministic polynomial algorithm is an abstract two- stage procedure that:
generates a random string purported to solve the problem
checks whether this solution is correct in polynomial time
By definition, it solves the problem if it’s capable of generating and verifying a solution on one of its tries
Why this definition?
led to development of the rich theory called “computational complexity”

27. Example: CNF satisfiability
Problem: Is a boolean expression in its conjunctive normal form (CNF) satisfiable, i.e., are there values of its variables that makes it true?
This problem is in NP. Nondeterministic algorithm:
Guess truth assignment
Substitute the values into the CNF formula to see if it evaluates to true
Example: (A | ¬B | ¬C) & (A | B) & (¬B | ¬D | E) & (¬D | ¬E)
Truth assignments:
A B C D E
Checking phase: O(n)

28. For Next Class, Wednesday
Chapter 11: P vs. Np
Homework due Thursday