1. analysis, plug ‘n’ chug, & induction
Tuesday, February 3, 2015
analysis, plug ‘n’ chug, & induction
CS16: Introduction to Data Structures & Algorithms

2. Announcements 2/3/15 Sections have started
Tuesday, February 3, 2015
Announcements 2/3/15
Sections have started
Note that rooms this week may be different than last week!
If you didn’t receive your graded HW1 and a grade report via , let us know
Homework 2 due Thursday 11:59pm
Seamcarve due Monday 11:59pm
Thursday is Python Lab part 2
Please go to the room you went to last week

3. Outline Recurrence Review Recurrence Relations Plug ‘n’ Chug Induction
Tuesday, February 3, 2015
Outline
Recurrence Review
Recurrence Relations
Plug ‘n’ Chug
Induction
Strong vs. Weak Induction

4. Tuesday, February 3, 2015
Recursion Review
Recursion is way of decomposing problems into smaller, simpler sub-tasks that are similar to the original.
Thus, each sub-task can be solved by applying a similar technique.
The whole problem is solved by combining the solutions to the smaller problems.
Requires a BASE CASE (A case simple enough to solve without recursion) to end recursion.

5. Recursion Example Compute the factorial of a number, n.
Tuesday, February 3, 2015
Recursion Example
def factorial(n):
if n == 1:
return 1
else:
return n * factorial(n-1)
Compute the factorial of a number, n.

6. Recursion Simulation Calculate 3 factorial
Tuesday, February 3, 2015
Recursion Simulation
Calculate 3 factorial
This is a call to factorial(3), so we put factorial(3) on the call stack
def factorial(n):
if n == 1:
return 1
else:
return n*factorial(n-1)
Call Stack

7. Recursion Simulation Calculate 3 factorial
Tuesday, February 3, 2015
Recursion Simulation
Calculate 3 factorial
This is a call to factorial(3), so we put factorial(3) on the call stack
def factorial(n):
if n == 1:
return 1
else:
return n*factorial(n-1)
factorial(3)
Call Stack

8. Tuesday, February 3, 2015
Recursion Simulation
n != 1, so we return n*factorial(n-1), which includes a call to factorial(2).
Remember, the call to factorial(3) has not returned yet, so it is still on the call stack!
def factorial(n):
if n == 1:
return 1
else:
return n*factorial(n-1)
factorial(3)
Call Stack

9. Tuesday, February 3, 2015
Recursion Simulation
n != 1, so we return n*factorial(n-1), which includes a call to factorial(2).
Remember, the call to factorial(3) has not returned yet, so it is still on the call stack!
def factorial(n):
if n == 1:
return 1
else:
return n*factorial(n-1)
factorial(2)
factorial(3)
Call Stack

10. Tuesday, February 3, 2015
Recursion Simulation
n still != 1, so we return n*factorial(n-1), which includes a call to factorial(1).
Neither factorial(2) nor factorial(3) has returned at this point!
def factorial(n):
if n == 1:
return 1
else:
return n*factorial(n-1)
factorial(2)
factorial(3)
Call Stack

11. Tuesday, February 3, 2015
Recursion Simulation
n still != 1, so we return n*factorial(n-1), which includes a call to factorial(1).
Neither factorial(2) nor factorial(3) has returned at this point!
def factorial(n):
if n == 1:
return 1
else:
return n*factorial(n-1)
factorial(1)
factorial(2)
factorial(3)
Call Stack

12. Recursion Simulation Now n =1, so we return 1!
Tuesday, February 3, 2015
Recursion Simulation
Now n =1, so we return 1!
This is not a recursive call, so factorial(1) returns, and we take it off of the call stack.
def factorial(n):
if n == 1:
return 1
else:
return n*factorial(n-1)
factorial(1)
factorial(2)
factorial(3)
Call Stack

13. Recursion Simulation Now n =1, so we return 1!
Tuesday, February 3, 2015
Recursion Simulation
Now n =1, so we return 1!
This is not a recursive call, so factorial(1) returns, and we take it off of the call stack.
def factorial(n):
if n == 1:
return 1
else:
return n*factorial(n-1)
factorial(2)
factorial(3)
Call Stack

14. Recursion Simulation Call Stack
Tuesday, February 3, 2015
Recursion Simulation
Now factorial(2) is at the top of the call stack, so we return to where we were in factorial(2).
So we return 2*factorial(1), which we now know is 2*1, so factorial(2) returns 2 and is removed from the call stack!
def factorial(n):
if n == 1:
return 1
else:
return n*factorial(n-1)
factorial(2)
factorial(3)
Call Stack

15. Recursion Simulation Call Stack
Tuesday, February 3, 2015
Recursion Simulation
Now factorial(2) is at the top of the call stack, so we return to where we were in factorial(2).
So we return 2*factorial(1), which we now know is 2*1, so factorial(2) returns 2 and is removed from the call stack!
def factorial(n):
if n == 1:
return 1
else:
return n*factorial(n-1)
factorial(3)
Call Stack

16. Recursion Simulation Call Stack def factorial(n): if n == 1: return 1
Tuesday, February 3, 2015
Recursion Simulation
Now factorial(3) is at the top of the call stack, so we’re back in factorial(3).
Return 3*factorial(2), which we now know is 3*2.
Factorial(3) returns 6 and removes itself from the call stack.
def factorial(n):
if n == 1:
return 1
else:
return n*factorial(n-1)
factorial(3)
Call Stack

17. Recursion Simulation Call Stack def factorial(n): if n == 1: return 1
Tuesday, February 3, 2015
Recursion Simulation
Now factorial(3) is at the top of the call stack, so we’re back in factorial(3).
Return 3*factorial(2), which we now know is 3*2.
Factorial(3) returns 6 and removes itself from the call stack.
def factorial(n):
if n == 1:
return 1
else:
return n*factorial(n-1)
Call Stack

18. Tuesday, February 3, 2015
Recursion Simulation
Now our call stack is empty, and we know that factorial(3) is 6!
def factorial(n):
if n == 1:
return 1
else:
return n*factorial(n-1)
Call Stack

19. Determining Running Times
Tuesday, February 3, 2015
Determining Running Times
For many algorithms, one can investigate code to see how size impacts running time
For seamcarve algorithm, each pixel is only evaluated in the for loops a constant number of times, and therefore the algorithm is O(n)
But for some algorithms, counting can be tricky!
For example, determining the running time of a recursive algorithm can be complex

20. Tuesday, February 3, 2015
Recurrence Relations
Recurrence relations describe the runtime of a recursive algorithm in two parts:
base case
how many instructions are executed in the base case of the recursive function, usually when n=0 or n=1
general case
how many instructions are executed in the recursive case

21. Tuesday, February 3, 2015
Recursive array_max
# Returns the maximum value of the first n elements in the array
# Example: array_max([5,1,9,2], 4)  9
def array_max(array, n):
if n == 1:
return array[0]
else:
return max(array[n-1], array_max(array, n-1))
T(n), the number of instructions executed as a function of the input size, can be expressed as a recurrence relation
T(1) = c0
constant number of operations to compare and return
T(n) = c1 + T(n-1)
constant number to do compare and calculate max
plus the operations of the recursive call
But how do we get a big-O out of this?

22. Tuesday, February 3, 2015
Recursive array_max
def array_max(array, n): if n == 1: return array[0] else: return max(array[n-1], array_max(array, n-1)) array_max([5,1,9,2], 4) = max(2, array_max([5,1,9], 3)) = max(2, max(9, array_max([5,1], 2))) = max(2, max(9, max(1, array_max([5], 1)))) = max(2, max(9, max(1, 5))) = max(2, max(9, 5)) = max(2, 9) = 9
Note: We only show the portion of the list we are working with, because we decrease n by 1 each time. Actually shrinking the list by 1 each time would require creating a copy, which is linear, and thus not optimal.
Hand simulate **do max, not min**
add elements starting at the left
draw on board for list of 5 numbers

23. Tuesday, February 3, 2015
Plug ‘n’ Chug
Given the recurrence relation (the base case T(1) and the general case T(n)), we can “plug ‘n’ chug” to find a recurrence solution
A recurrence solution is the “answer” to a recurrence relation: it turns the recursive definition into a simpler, closed-form mathematical expression
Simplifying, we get:
T(n) = c1n – c1 + c0
We can see that T(n) is a linear function, which makes array_max O(n)
T(1) = c0 T(2) = c1 + T(2-1) = c1 + T(1) = c1 + c0 T(3) = c1 + T(3-1) = c1 + T(2) = c1 + c1 +c0 = 2c1 + c0 T(4) = c1 + T(4-1) = c1 + T(3) = c1 + 2c1 + c0 = 3c1 + c0
⋮ T(n) = c1 + T(n-1) = (n-1)c1 + c0
(can do this whole thing on the board and skip)

24. Tuesday, February 3, 2015
How can we be sure?
We just used the plug ‘n’ chug method to posit that the runtime of array_max was O(n)
But are we sure? Can we prove it?
We observed a pattern in order to reach a recurrence solution, but a “pattern” isn’t a formal proof
In order to prove that the algorithm has the runtime we think it does, we’ll need to prove that our recurrence solution is correct!

25. Tuesday, February 3, 2015
Induction
Induction is a method of mathematical proof used to establish that a statement is true for all positive integers
first demonstrate the statement’s truth for a single positive integer
second prove that, given the assumption that the statement is true for an arbitrary input, the statement is true for the next input
In other words:
If want to prove something for all n
Prove for:
n = 1
n = k + 1 if true for n = k
Celebrate!
See the handout on website for another example.

26. Tuesday, February 3, 2015
Induction Example
Claim: The solution for T(1) = c0, T(n) = c1 + T(n-1) is T(n) = (n-1)c1 + c0
Base Case
Prove the base case by plugging in n=1 to the recurrence solution:
T(1) = (1-1)c1 + c0 = c0
We’re given that T(1) = c0 in the base case of the recurrence relation, so recurrence solution works for the base case!
Inductive Assumption
Assume the solution works for k
T(k) = (k-1)c1 + c0
Inductive Step
Show that it works for k+1 given the assumption that it works for k -- ultimately, we want to show that T(k+1) = (k)c1 + c0
T(k+1) = c1 + T(k) according to recurrence relation
T(k+1) = c1 + (k-1)c1 + c0 substituting inductive assumption
T(k+1) = (k)c1 + c0 simplifying
Conclusion
Because we’ve proven our claim for the base case n = 1 and shown truth for n = k implies truth for n = k+1, therefore T(n) = (n-1)c1 + c0 for all positive integers n.

27. Induction Example 2 Given: By plug ‘n’ chug: 2, 4, 8, 16, 32, …, A(n)
Tuesday, February 3, 2015
Induction Example 2
Given:
A(n) is the number of subsets of a set of size n
A(1) = 2
empty set and set of one element
A(n) = A(n-1) + A(n-1) = 2A(n-1)
all the sets in n-1 case with and without nth element included
By plug ‘n’ chug: 2, 4, 8, 16, 32, …, A(n)
Looks like the recurrence solution is A(n) = 2n
Can we prove it?
Statement to prove:
A(n) = 2n is the recurrence solution for A(n) = 2A(n-1)

28. Induction Example 2 Given:
Tuesday, February 3, 2015
Induction Example 2
Given:
A(n) is the number of subsets of a set of size n
A(1) = 2 (empty set and set of one element)
A(n) = 2A(n-1)
Claim: A(n) = 2n is the recurrence solution for A(n) = 2A(n-1)
Proof:
base case A(1) = 21 = 2 given
inductive assumption A(k) = 2k assume for n = k
inductive step A(k+1) = 2A(k) given A(k+1) = 2 * 2k substituting assumption A(k+1) = 2k simplifying
conclusion: Because we’ve proved our claim for n = 1 and shown that n = k implies n = k+1, therefore A(n) = 2n for all positive integers

29. Sometimes called the “predicate”
Tuesday, February 3, 2015
Induction Example 3
Prove P(n) is true for all positive integers, n:
Base case P(1):
Assume P(k) is true:
Sometimes called the “predicate”

30. Induction Example 3 (2) Prove AWESOME! “implies”
Tuesday, February 3, 2015
Induction Example 3 (2)
Prove
“implies”
Start with definition of Σ
Plug in inductive assumption
Multiply by 2/2
Factor out (k+1)
AWESOME!

31. Strong vs. Weak Induction
Tuesday, February 3, 2015
Strong vs. Weak Induction
This induction example is weak induction
Logic of the inductive step (n = k +1) relies only on the previous step (n = k)
This differs from strong induction
Logic of the inductive step relies on a “stronger” assumption of more than one value (n ≤ k)
Sometimes makes inductive step easier
The strong and weak refer to the assumptions you have to make to complete the proof, not the strength of the proof
For much of CS16, weak induction is sufficient

32. Tuesday, February 3, 2015
Readings
Read the “Induction Handout” on the website! (in the Docs section)