1. CSC 380: Design and Analysis of Algorithms
Dr. Curry Guinn

2. Quick Info Dr. Curry Guinn CIS 2015 guinnc@uncw.edu
Office Hours: MWF: 10:00am-11:00m and by appointment

3. Outline of today Homework 6 Limits Lower bounds
Easy-to-find lower bounds
A bit harder to find lower bounds
Is the lower bound “tight”?
Intractable vs Impossible
Alan Turing, an introduction
Halting problem

4. Lower Bounds
Lower bound: an estimate on a minimum amount of work needed to solve a given problem
Examples:
number of comparisons needed to find the largest element in a set of n numbers
number of comparisons needed to sort an array of size n
number of comparisons necessary for searching in a sorted array
number of multiplications needed to multiply two n-by-n matrices
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 11 ©2012 Pearson Education, Inc. Upper Saddle River, NJ. All Rights Reserved.

5. Lower Bounds (cont.) Lower bound can be
an exact count
an efficiency class ()
Tight lower bound: there exists an algorithm with the same efficiency as the lower bound
Problem Lower bound Tightness
sorting (nlog n) yes
searching in a sorted array (log n) yes
element uniqueness (nlog n) yes
n-digit integer multiplication (n) unknown
multiplication of n-by-n matrices (n2) unknown
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 11 ©2012 Pearson Education, Inc. Upper Saddle River, NJ. All Rights Reserved.

6. Methods for Establishing Lower Bounds
trivial lower bounds
information-theoretic arguments (decision trees)
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 11 ©2012 Pearson Education, Inc. Upper Saddle River, NJ. All Rights Reserved.

7. Trivial Lower Bounds
Trivial lower bounds: based on counting the number of items that must be processed in input and generated as output
Examples
finding max element
sorting
element uniqueness
Hamiltonian circuit existence
Conclusions
may and may not be useful
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 11 ©2012 Pearson Education, Inc. Upper Saddle River, NJ. All Rights Reserved.

8. Decision Trees
Decision tree — a convenient model of algorithms involving
comparisons in which:
internal nodes represent comparisons
leaves represent outcomes
Decision tree for 3-element insertion sort
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 11 ©2012 Pearson Education, Inc. Upper Saddle River, NJ. All Rights Reserved.

9. Decision Trees and Sorting Algorithms
Any comparison-based sorting algorithm can be represented by a decision tree
Number of leaves (outcomes)  n!
Height of binary tree with n! leaves  log2n!
Minimum number of comparisons in the worst case  log2n! for any comparison-based sorting algorithm
log2n!  n log2n
This lower bound is tight (mergesort)
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 3rd ed., Ch. 11 ©2012 Pearson Education, Inc. Upper Saddle River, NJ. All Rights Reserved.

10. Who was Alan Turing?
1912 (23 June) Paddington, London

11. Theory of Computers
1936:
Computability (The Halting Problem)
The Turing Machine
The Universal Machine
The Turing machine, computability, universal machine

12. The Halting Problem An example of something that is not computable.
Created by Turing in 1936 to define a problem which no algorithmic procedure can solve.
Can we write a program that will take in a user's program and inputs and decide whether
it will eventually stop, or
it will run infinitely in some infinite loop ?

13. Proof (by contradiction)
Assume that it is possible to write a program to solve the Halting Problem.
Denote this program by Halt(program,input).
Halt (program,input) will
return yes if program will halt on input and
no otherwise.
A program is just a string of characters
E.g. your Java program is just a long string of characters
An input can also be considered as just a string of characters
So Halt is effectively just working on two strings

14. Proof (cont.)
We can now write another program Loopy(program) that uses Halt
The program Loopy(program) does the following:
[1] If Halt (program,program) returns yes,
Loopy will go into an infinite loop.
[2] If Halt(program,program) returns no,
Loopy will halt.

15. Proof (cont.) [1] If Halt (program,program) returns yes,
Loopy will go into an infinite loop.
[2] If Halt (program,program) returns no,
Loopy will halt.
Consider what happens when we run Loopy(Loopy).
If Loopy loops infinitely,
Halt(Loopy,Loopy) return no which by [2] above means Loopy will halt.
If Loopy halts,
Halt (Loopy,Loopy) will return yes which by [1] above means Loopy will loop infinitely.
Conclusion: Our assumption that it is possible to write a program to solve the Halting Problem has resulted in a contradiction.

16. Turing Machine
Simulator:

17. The Universal Turing Machine
There are an infinite number of Turing Machines
There are an infinite number of calculations that can be done with a finite set of rules.
However, we can define a Universal Turing Machine which can simulate all possible TMs
Comes from the definition of TMs
You convert the description of the TM and its input into two tapes, and use these as the input to the UTM

18. Cryptography
The military Enigma has 158,962,555,217,826,360,000 (158 quintillion) different settings.
: The Bombe, machine for Enigma decryption
Colossus ( ):

20. Programmable Electronic Computers
: Programming
Beaten out by Americans (ENIAC led by John von Neumann) on the construction of the 1st “Turing complete” programmable electronic computer
Neural nets and artificial intelligence
Interests shifted to neurology and physiology

21. The Turing Test 1950: The Turing Test for machine intelligence
How do you know if something is intelligent?

22. Computational Biology
1951: Non-linear theory of biological growth

23. Tragic End 1952: Arrested as a homosexual, loss of security clearance
1954 (7 June): Death (suicide) by cyanide poisoning, Wilmslow, Cheshire
Irony

24. Benedict Cumberbatch

25. Other non-computable problems
Almost every problem related to the behavior of a program is non-computable. For example:
Showing that 2 programs are equivalent (for each input you always get the same output).
2) Determining whether a given output will occur
3) Determining the correctness of a program
etc.

26. Your boss asks you to find …
A solution:
Can you find a solution -> feasibility
Can you find a cheaper solution -> optimality
Can you find the cheapest solution -> uniqueness
Can you find a cheap solution -> suboptimal
How cheap is your solution -> degree of suboptimality
Why can’t you find a cheap solution -> hardness

27. Your boss asks you to …
Have the computer find the solution of the problem
Is your algorithm fast? -> polynomial time
Is your algorithm slow? -> non-polynomial time

28. Performance Categories of Algorithms
Sub-linear O(Log N)
Linear O(N)
Nearly linear O(N Log N)
Quadratic O(N2)
Exponential O(2N)
O(N!)
O(NN)
Polynomial

29. Reasonable vs. Unreasonable
Reasonable algorithms have polynomial factors
O (Log N)
O (N)
O (NK) where K is a constant
Unreasonable algorithms have exponential factors
O (2N)
O (N!)
O (NN)

30. Effects of Exponents 1050 1045 1040 1035 1030 1025 1020 NN 2N 1015
trillion
billion
million
1000
100 10 NN 2N N5 N
Don’t Care! N

31. Effects of Exponents Consider an 2N algorithm for N of only 256.
Time cost is a number with 78 digits to the left of the decimal.
For comparison:
Number of protons (estimated) in the known universe is a number with 77 digits.

32. What about a Faster Computer?
What if:
Instructions took ZERO time to execute
CPU registers could be loaded at the speed of light
These algorithms are still unreasonable!
The speed of light is only so fast!

33. Relations between Problems, Algorithms, and Programs
Algorithm
Program
Program
Program
Program

34. For Next Class, Monday Chapter 11: Limitations of Algorithm Power
P vs. NP