1. More Computational Theory
Code analysis
Mathematical induction
September 12, 2017
Cinda Heeren / Geoffrey Tien

2. Cinda Heeren / Geoffrey Tien
Analyzing Code
Types of analysis
Bound flavour
Upper bound (O)
Lower bound (Ω), useful for problems
Asymptotically tight (Θ)
Analysis case
Worst case (adversary)
Average case
Best case / "lucky" case
"common" case
Analysis quality
Loose bound (any true analysis)
Tight bound (no better "meaningful" bound that is asymptotically different)
September 12, 2017
Cinda Heeren / Geoffrey Tien

3. Cinda Heeren / Geoffrey Tien
Input Varies
The number of operations usually varies based on the size of the input
Though not always – consider array lookup
In addition algorithm performance may vary based on the organization of the input
For example consider searching a large array
If the target is the first item in the array the search will be very fast
September 12, 2017
Cinda Heeren / Geoffrey Tien

4. Cinda Heeren / Geoffrey Tien
Linear Search
// Iterative linear search
int linSearch(int arr[], int size, int x) {
for (int i = 0; i < size; i++)
if (x == arr[i])
return i; }
return -1; //target not found
The function returns as soon as a match is found
-1 is returned if the target item is not found by the time the array end is reached
September 12, 2017
Cinda Heeren / Geoffrey Tien

5. Linear Search Comparisons
For different input organization
Worst case
The target is not in the array or
The target is at the last position in the array
Make n comparisons in either case
Best/lucky case (this rarely occurs)
The target is the first element of the array
Make 1 comparison
Average case
Is it (best case + worst case) / 2, i.e. (n + 1) / 2?
Average/usual case analysis is tricky and requires several assumptions that may not hold in all situations
𝑂 𝑛
𝑂 1
𝑂 𝑛
September 12, 2017
Cinda Heeren / Geoffrey Tien

6. Searching Sorted Arrays
If we sort the target array first we can change the linear search average cost to around 𝑛 2
Once a value equal to or greater than the target is found the search can end
So, if a sequence contains 8 items, on average, linear search compares 4 of them,
If a sequence contains 1,000,000 items, linear search compares 500,000 of them, etc.
However, if the array is sorted, it is possible to do much better than this by using binary search
September 12, 2017
Cinda Heeren / Geoffrey Tien

7. Binary Search Algorithm
int binSearch(int arr[], int size, int target) {
int low = 0;
int high = size - 1;
int mid = 0;
while (low <= high)
mid = (low + high) / 2;
if (target == arr[mid])
return mid;
else if (target > arr[mid])
low = mid + 1;
else //target < arr[mid]
high = mid - 1; }
return -1; //target not found
While subarray has at least 1 element
September 12, 2017
Cinda Heeren / Geoffrey Tien

8. Cinda Heeren / Geoffrey Tien
Worst Case
Binary Search
What is the worst case for binary search?
Either the target is not in the array, or
It is found when the search space consists of one element
How many times does the while loop iterate in the worst case?
binSearch(arr, n, 64);
mid = (0+15) / 2 = 7
mid = (8+15) / 2 = 11
mid = (8+10) / 2 = 9
mid = (10+10) / 2 = 10
Done 7 11 15 21 29 32 44 45 57 61 64 73 79 81 86 92 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
September 12, 2017
Cinda Heeren / Geoffrey Tien

9. Cinda Heeren / Geoffrey Tien
Worst Case Analysis
Binary Search
Each iteration of the while loop halves the search space
For simplicity assume that 𝑛 is a power of 2
So 𝑛= 2 𝑘 (e.g. if 𝑛=128, 𝑘=7)
How large is the search space?
The first iteration halves the search space to 𝑛 2
After the second iteration the search space is 𝑛 4
After the kth iteration the search space consists of just one element, since 𝑛 2 𝑘 = 𝑛 𝑛 =1
Because 𝑛= 2 𝑘 , 𝑘= log 2 𝑛
Therefore at most log 2 𝑛 iterations of the while loop are made in the worst case!
𝑂 log 𝑛
September 12, 2017
Cinda Heeren / Geoffrey Tien

10. Cinda Heeren / Geoffrey Tien
Average Case
Binary Search
Is the average case more like the best case or the worst case?
What is the chance that an array element is the target
1/n the first time through the loop
1/(n/2) the second time through the loop
… and so on …
It is more likely that the target will be found as the search space becomes small
That is, when the while loop nears its final iteration
We can conclude that the average case is more like the worst case than the best case
𝑂 log 𝑛
September 12, 2017
Cinda Heeren / Geoffrey Tien

11. Analysing nested loops
Generally, we count the number of times an inner loop is repeated, for each time an outer loop is executed
e.g. a function which scans an array looking for duplicates, worst case
bool hasDuplicate(int arr[], int size) {
for (int i = 0; i < size-1; i++) {
for (int j = i+1; j < size; j++) {
if (arr[i] == arr[j])
return true; }
return false;
outer loop: 𝑛−1 times, 𝑂 𝑛
inner loop: between 1 to 𝑛−1 times, 𝑂 𝑛
Is it 𝑂 𝑛 ×𝑂 𝑛 =𝑂 𝑛 2 ?
September 12, 2017
Cinda Heeren / Geoffrey Tien

12. Analysing nested loops
hasDuplicate, worst case
bool hasDuplicate(int arr[], int size) {
for (int i = 0; i < size-1; i++) {
for (int j = i+1; j < size; j++) {
if (arr[i] == arr[j])
return true; }
return false; 𝑖
# of inner loop executions
𝑛−1 1
𝑛−2 2
𝑛−3
...
# of inner loop executions
In this case, there is a constant number of elementary operations in the inner loop, so worst case is 𝑂 𝑛 2 ∙𝑂 1 =𝑂 𝑛 2
September 12, 2017
Cinda Heeren / Geoffrey Tien

13. Mathematical Induction
Warning! Clipart lion cannibalism inside!
Viewer discretion is advised.
September 12, 2017
Cinda Heeren / Geoffrey Tien

14. Proof techniques The adventure thus far… Witness Counterexample
Show an example satisfying an existence theorem
Counterexample
Provide a specific example showing the falsity of a universal quantification
Contradiction
Assume the opposite of the theorem
Derive a contradiction
Mathematical induction
Prove a property for one or more base cases (e.g., 𝑛=1)
Assume the property is true for one or more anonymous values (e.g. 𝑛=𝑘, 𝑛≤𝑘)
Using the assumption above, prove for the next value (e.g. 𝑛=𝑘+1)
So, we use an unproven theorem, to prove itself. WHAT??
September 12, 2017
Cinda Heeren / Geoffrey Tien

15. Cinda Heeren / Geoffrey Tien
Proof by Induction
Principle and insight
Let 𝑃 𝑛 be a property defined on the integers 𝑛, and let 𝑎 be a fixed integer.
Suppose the following two statements are true:
𝑃 𝑎 is true.
For all integers 𝑘≥𝑎, if 𝑃 𝑘 is true, then 𝑃 𝑘+1 is true.
Then the statement:
For all integers 𝑛≥𝑎, 𝑃 𝑛 is true.
Assume this is true
Derive the truth of this
"peel away" one item,
replace with 𝑃 𝑘
Directly prove base cases
Break the property down into smaller cases
Assuming property is true for smaller cases, build up to 𝑘 case
September 12, 2017
Cinda Heeren / Geoffrey Tien

16. Cinda Heeren / Geoffrey Tien
Lion cuisine
Induction example
In the African savanna, lions are starving
They may eat one another if given the opportunity
They want to avoid being eaten by another lion
They are experts in mathematical induction
Suppose a zebra drops dead surrounded by a group of hungry lions.
Does a lion approach and eat the zebra (without being eaten)?
September 12, 2017
Cinda Heeren / Geoffrey Tien

17. Who eats the zebra? 𝑛=1 𝑛=2 𝑛=3 𝑛=4 Pattern? How many lions are there?
The lion eats the zebra, will not be attacked
A base case!
𝑛=2
Nobody eats the zebra, the lions fear each other
Another base case!
𝑛=3
One lion eats the zebra, the other two fear each other
The eating lion becomes like the zebra
Reduction to the 2-lion case
𝑛=4
If a lion eats, the problem is reduced to the 3-lion case, and the eating lion will be eaten
So, nobody eats the zebra
Pattern?
September 12, 2017
Cinda Heeren / Geoffrey Tien

18. Lions Theorem: Basis step: Inductive step: And mathematical induction
For 𝑛≥1 lions, the zebra will be eaten only if 𝑛 is an odd number.
i.e. 𝑃 𝑛 = the zebra will be eaten only if 𝑛 is odd
Basis step:
𝑛=1, 𝑛=2, already demonstrated on previous slide.
Inductive step:
Inductive hypothesis (I.H.): Assume the theorem holds for an arbitrary integer 𝑘≥1, i.e. 𝑃 𝑘 is true
Show that 𝑃 𝑘+1 is true
September 12, 2017
Cinda Heeren / Geoffrey Tien

19. Cinda Heeren / Geoffrey Tien
A feast for 𝑘+1 lions
Inductive step, case 1: 𝑘 is odd
By the I.H., one among the first 𝑘 lions will eat the zebra and become vulnerable
This lion can be eaten by the 𝑘+1 𝑡ℎ lion, thus backs away to defend himself
Therefore, if 𝑘 is odd, 𝑘+1 is even, and nobody eats the zebra
Inductive step, case 2: 𝑘 is even
By the I.H., nobody among the first 𝑘 lions eats the zebra
The 𝑘+1 𝑡ℎ lion safely eats the zebra
Therefore, if 𝑘 is even, 𝑘+1 is odd, and the zebra gets eaten
QED
September 12, 2017
Cinda Heeren / Geoffrey Tien

20. A numeric example Prove using mathematical induction: Basis step:
Sum of the first 𝑛 integers
Prove using mathematical induction:
Basis step:
Prove 𝑃 1
Inductive step:
Prove that if 𝑃 𝑘 is true, then 𝑃 𝑘+1 is true
September 12, 2017
Cinda Heeren / Geoffrey Tien

21. Breaking down sums What does really look like? What about ?
Sum of the first 𝑛 integers
What does really look like?
What about ?
Substitute this with our assumption from the inductive hypothesis
September 12, 2017
Cinda Heeren / Geoffrey Tien

22. A numeric example Inductive step Sum of the first 𝑛 integers
Inductive hypothesis: Assume that 𝑃 𝑘 is true, i.e.
If 𝑃 𝑘+1 is true,
Substitute by I.H.
Multiply by 2 2
𝑘+1 common factor
QED
September 12, 2017
Cinda Heeren / Geoffrey Tien

23. Readings for this lesson
Epp
Chapter 5.2 – 5.3
Next class:
Koffman: Chapters P.7, 10.1 – 10.2, 10.4
Try it!
Prove using mathematical induction, the following properties:
is divisible by 8
Hint: an integer 𝑥 is divisible by 8 if 𝑥=8∙𝑚
Lion induction example from
September 12, 2017
Cinda Heeren / Geoffrey Tien