CS 312: Algorithm Analysis Lecture #7: Recurrence Relations a.k.a. Difference Equations Slides by: Eric Ringger, with contributions from Mike Jones, Eric Mercer, Sean Warnick This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License.Creative Commons Attribution-Share Alike 3.0 Unported License

Announcements  HW #4 Due Now  Project #2  Early: next Wednesday  Due: Friday 2/1  Much more challenging than Project #1 – plan accordingly  Career Fair!

Objectives  Goal: Analyze Divide and Conquer recursive algorithms using recurrence relations (RRs)  Also known as Difference Equations  Working up to such RRs  Today: Focus on a special type of RRs  Homogeneous  Linear  Constant Coefficients  Leading up to a proof of the Master Theorem

Recall: Analysis Framework Given a problem,  Identify platform’s elementary operations (sometimes implicit)  Formulate solution as an algorithm  Define the measure of the input size  Measure time efficiency by counting the number of times an elementary operation is executed  Measure space efficiency by counting the number of memory units consumed by the algorithm  The efficiency of some algorithms may differ significantly for inputs of the same size.  Distinguish among worst-case, average-case, and best-case efficiencies. Choose one.  Plot efficiency vs. input size  Establish order of growth. Use asymptotic notation. n0n0 c 1 g(n) c 2 g(n) f(n) Where is the difficulty for Analyzing Recursive Functions?

Nonrecursive Algorithms Example: Element Uniqueness Problem, check whether all elements in an array are distinct. function UniqueElements(A[0…n-1]) for i=0 to n-2 for j = i+1 to n-1 if A[i]=A[j] return false return true Define input size Define elementary operation Distinguish worst-case

Recurrence Relations

Recursive Algorithms Example: Compute Factorial function N! function Factorial(N) if N=0 return 1 else return Factorial(N-1)*N Assume 32-bit integers Define input size: _____ Define elementary operations: Focus on worst-case What sequence is generated by this recurrence relation? In order to answer that question we need an initial condition! Now we can build a table of values: How long to compute C(1,000,000)? Want: C(N) in closed form for quick computation. N C(N)

Recursive Algorithms Example: Tower of Hanoi, move all disks to third peg without ever placing a larger disk on a smaller one. How to solve?

Recursive Algorithms Example: Tower of Hanoi, move all disks to third peg without ever placing a larger disk on a smaller one.

Recursive Algorithms Example: Tower of Hanoi, move all disks to third peg without ever placing a larger disk on a smaller one.

Recursive Algorithms Example: Tower of Hanoi, move all disks to third peg without ever placing a larger disk on a smaller one.

Recursive Algorithms Example: Tower of Hanoi, move all disks to third peg without ever placing a larger disk on a smaller one.

Recursive Algorithms Example: Tower of Hanoi, move all disks to third peg without ever placing a larger disk on a smaller one. Can you figure out an explicit (closed form) formula for this sequence? As before, build a table, recognize the pattern OR Use substitution, recognize the pattern OR Appeal to theory of recurrence relations n C(n)

Recursive Algorithms Example: Tower of Hanoi, move all disks to third peg without ever placing a larger disk on a smaller one. Can you figure out an explicit (closed form) formula for this sequence? As before, build a table, recognize the pattern OR Use substitution, recognize the pattern OR Appeal to theory of recurrence relations n C(n)

Substitution k y(k)

Recurrence Relations The most general form for our purposes:

Constant Coefficients

Example Capital growth in your savings account y(n)= $$ in account end of day n i = interest rate earned each day (at start of day) D(n) = Deposit on day n W(n) = Withdraw on day n You try it! Write a recurrence relation for y(n)

Order of Recurrence Relations What is the order of this RR?

Forcing Function

Homogeneous

What kind of RR is this? k y(k)

Terminology Review Note the alternate notations.

Terminology Review Note the alternate notations.

Existence and Uniqueness Theorem

Starting Point

Example Goal: find solution t n

Example Goal: find solution t n

General Solution

Solution?

Assignment  Read: Recurrence Relations Notes, Part II  HW #5: Part I (Section 1.2) Exercises in the RR notes.

End

Extra: Big Picture  Look at the big picture: functions as “systems”

Recurrence Relations So why should we care about recurrence relations? Generalizes the idea of a function to an operator or dynamic system A function is like a table that takes a number in and gives a number out. f xy xy x y x y

Recurrence Relations Functions can also take in a vector of numbers or yield a vector out. f x1x2y1y2y y1y1 x1x1 x2x2 y2y2 x1x1 x2x2 y3y3 x1x1 x2x2

Recurrence Relations What if the input and output vectors of a function were infinitely long? f Then x and y themselves are functions! f becomes a map from one function to another! We call maps that take functions as inputs or generate functions as outputs operators, or systems. An operator is like a rule, it’s the mathematical description of a subroutine! k x k y

Recurrence Relations S x(k) y(k) k x k y

Recurrence Relations S x(k) y(k) Unlike a simple function, an operator can have local variables that store intermediate calculations—just like a subroutine. In other words, S is a map that can have memory!

Recurrence Relations S x(k) y(k) Subroutine Description static memory: w = 0 function S (x,k) if k=0 then y  y o else y  w + x w  y return y Math Description y(0)=y o

Math Description Subroutine Description static memory: w = 0 function S (x,k) if k=0 then y  y o else y  5w + x w  y return y Recurrence Relations S x(k) y(k) y(0)=y o

Subroutine Description static memory: w 0, w 1 = 0 function S (x,k) if k=0 then y  y o else if k=1 then y  y 1 else y  a 1 w 1 + a o w 0 +x w 0  w 1 w 1  y return y Math Description Recurrence Relations S What if more than one number is stored in memory? x(k) y(k) y(0)=y o, y(1) = y 1