Advanced Algorithms Analysis and Design By Dr. Nazir Ahmad Zafar Dr Nazir A. Zafar Advanced Algorithms Analysis and Design

Lecture No. 7 Recurrence Relations Mathematical Models, Analysis Techniques Dr Nazir A. Zafar Advanced Algorithms Analysis and Design

What is Recursion? Recursive Mathematical Models Solving Recurrence Relations First Order Linear Homogenous Recurrence Relations, with Constant Coefficients Second Order Linear Homogenous Recurrence Relations, Constant Coefficients –Characteristics of Second Order Recurrences –Solution to Second order with distinct roots –Solution to Second order with repeated roots General Homogenous Recurrences –Characteristics and solution Dr Nazir A. Zafar Advanced Algorithms Analysis and Design Today Covered

Some times problems are too difficult or too complex to solve because these are too big. A problem can be broken down into sub-problems and find a way to solve these sub-problems Then build up to a solution to the entire problem. This is the idea behind recursion Recursive algorithms break down problem in pieces which you either already know the answer, or can be solved applying same algorithm to each piece And finally combine the results of these sub- problems to find the final solution Dr Nazir A. Zafar Advanced Algorithms Analysis and Design What is Recursion?

More concisely, a recursive function or definition is defined in terms of itself. Recursion is a computer algorithm that calls itself in a steps having a termination condition. The successive repetitions are processed up to the critical step where the condition is met. In recursive algorithms, each repetition is processed from the last one called to the first. Recursion is a wonderful technique for dealing with many problems where problems and sub- problems have same mechanism to solve it. Dr Nazir A. Zafar Advanced Algorithms Analysis and Design What is Recursion?

Recursive solutions are much easier to conceive of and code than their iterative counterparts. Every problem is not solvable using this approach What kinds of problems are solved with recursion? Generally, problems which are defined in terms of themselves are usually good candidates for recursive techniques. Example Finding factorial of a number is an easiest example one can think using recursion Dr Nazir A. Zafar Advanced Algorithms Analysis and Design Merits and Demerits of Recursion

Since n! can be computed as:5! = 5*4*3*2*1. If we have a close look at this, we notice that 5! = 5*4!. Now if denote F(n) = n! then it can be written as F(n) = n.F(n-1) Assume 0! = 1 i.e. F(0) = 1, and solve till termination F(n) = n.F(n-1) = n.(n-1).F(n-2) = n.(n-1).(n-2).F(n-3)... F(n) = n.(n-1).(n-2) F(0) = n.(n-1).(n-2) Dr Nazir A. Zafar Advanced Algorithms Analysis and Design Recursive Mathematical Model

The indispensable last step when analyzing an algorithm is often to solve a recurrence equation. With little experience recurrence equation can be solved by intelligent guesswork. However, there exist a powerful techniques that can be use to solve certain classes of recurrence almost automatically. This is a main topic of this lecture, techniques solving recurrence equations of form: –First order linear homogenous recurrences –Second order linear homogenous recurrences –Higher order recurrences Dr Nazir A. Zafar Advanced Algorithms Analysis and Design Solving Recurrence

First Order Linear Homogenous Recurrence Relations with Constant Coefficients (FOLHRRCC) Dr Nazir A. Zafar Advanced Algorithms Analysis and Design

No YES No Dr Nazir A. Zafar Advanced Algorithms Analysis and Design Definition and Examples of FOLHRRCC No

Solve the recurrence:if b 0 = c Solution b k = a.b k-1 b k = a.(a. b k-2 ) = a 2. b k-2 b k = a 2.(a. b k-3 ) = a 3. b k-3... b k = a k. b k-k = a k. b 0 b k = a k. c Now the explicit formula is c.a k which is a geometric sequence. Hence first order recurrence is nothing but G.S. Dr Nazir A. Zafar Advanced Algorithms Analysis and Design Solving First Order Recurrences

Example Solve the recurrence:if b 0 = 3 Solution b k = 5.b k-1 b k = 5.(5. b k-2 ) = 5 2. b k-2 b k = 5 2.(5. b k-3 ) = 5 3. b k-3... b k = 5 k. b k-k = 5 k. b 0 = 5 k. 3 Now the solution sequence becomes 3.5 0, 3.5 1, 3.5 2, 3.5 3, 3.5 4,.., 3.5 k,... 3, 15, 75, 375,...geometric sequence Dr Nazir A. Zafar Advanced Algorithms Analysis and Design Solving First Order Recurrences

Second Order Linear Homogenous Recurrence Relations with Constant Coefficients Dr Nazir A. Zafar Advanced Algorithms Analysis and Design

YES NO YES Dr Nazir A. Zafar Advanced Algorithms Analysis and Design Definition and Examples of SOLHRRCC

NO YES NO YES NO Dr Nazir A. Zafar Advanced Algorithms Analysis and Design Some More Examples

Theorem : Let A and B are real numbers. A recurrence of the form is satisfied by the sequence Proof : Let us suppose that the sequence Satisfies the recurrence relation Dr Nazir A. Zafar Advanced Algorithms Analysis and Design Theorem

It mean each form of sequence is equal to A times the previous form plus B times the form before it Hence Dividing both sides by Conversely suppose that satisfies the above recurrence relation Dr Nazir A. Zafar Advanced Algorithms Analysis and Design Contd...

Definition : Given a second order linear homogeneous recurrence relation with constant coefficients The characteristic equation of the relation is Example 1: Using characteristic equation, find solution to a recurrence relation Find all sequences that satisfy relation and have the form Solution : Above recurrence relation is satisfied by a sequence Dr Nazir A. Zafar Advanced Algorithms Analysis and Design Characteristic Equation

Hence and are both particular solutions for this recurrence relation Dr Nazir A. Zafar Advanced Algorithms Analysis and Design Contd...

Theorem Let A and B be real numbers, and suppose that the characteristic equation has a single root r, than the sequences Repeated Roots Both satisfy the recurrence relation Dr Nazir A. Zafar Advanced Algorithms Analysis and Design

Repeated Roots Proof has a single repeated root ‘r’ than We know that r n is a solution. Now we prove that is also a solution, i.e. satisfies the recurrence Dr Nazir A. Zafar Advanced Algorithms Analysis and Design

Repeated Roots Dr Nazir A. Zafar Advanced Algorithms Analysis and Design

Repeated Roots Example 1:Single root case Suppose sequence, b0, b1, b2,... satisfies recurrence relation then find the explicit formula for b0, b1, b2,... Solution: Characteristic equation Dr Nazir A. Zafar Advanced Algorithms Analysis and Design

Repeated Roots Dr Nazir A. Zafar Advanced Algorithms Analysis and Design

b n = (1 + n/2).2 n First term b 0 = (1 + 0/2).2 0 = 1.1 = 1 Second term b 1 = (1 + 1/2).2 1 = 3/2.2 = 3 Third term b 2 = (1 + 2/2).2 2 = 2.4 = 8 Fourth term b 3 = (1 + 3/2).2 3 = 5/2.8 = 20, and so on Checking Explicit Formula Dr Nazir A. Zafar Advanced Algorithms Analysis and Design

Theorem : are sequences that satisfy the some second order linear homogeneous recurrence relation with constant coefficients, and if C and D are any numbers then the sequence also satisfies the same recurrence relation ifand defined by the formula Proof : Since and Satisfy the same second order LHRRWCC Dr Nazir A. Zafar Advanced Algorithms Analysis and Design Theorem (Linear Combination is also a Solution)

and Then we have to prove that Consider R. H. S It proves that satisfies same recurrence Dr Nazir A. Zafar Advanced Algorithms Analysis and Design Theorem

Example 1: Find a sequence that satisfies the recurrence relation And that also satisfies the initial condition Solution:The characteristics equation of the relation is Dr Nazir A. Zafar Advanced Algorithms Analysis and Design Solution as Linear Combination

Are both sequence which satisfy above relation but neither one is having Now since also satisfies the same recurrence relation and and Dr Nazir A. Zafar Advanced Algorithms Analysis and Design Contd..

General Homogenous Recurrence Relations (Constant Coefficients) Dr Nazir A. Zafar Advanced Algorithms Analysis and Design

First order: linear combination of two cons. terms, Second order: linear combination of three consecutive terms, We extend technique of recurrence equation with resolution of homogeneous linear recurrence with constant coefficient, that is recurrence of the form a 0 t n + a 1 t n-1 + …..+a k t n-k = 0 (1) where t i are value we are looking for In equation (1), values of t i such that (1 < i < k) are need to determine a sequence. Equation 1 typically has infinite many solutions, because linear combination is also a solution. Dr Nazir A. Zafar Advanced Algorithms Analysis and Design K order: General Homogenous Recurrence

a 0 t n + a 1 t n-1 + …..+a k t n-k = 0 (1) If we put k = 1 then above equation becomes a 0 t n + a 1 t n-1 = 0 t n = -(a 1 / a 0 )t n-1 The resultant equation becomes a recurrence relation which is –first order –linear –homogenous –with constant coefficients. Dr Nazir A. Zafar Advanced Algorithms Analysis and Design K = 1: General Homogenous Recurrence

a 0 t n + a 1 t n-1 + …..+a k t n-k = 0 (1) If we put k = 2 then above equation becomes a 0 t n + a 1 t n-1 + a 2 t n-2 = 0 t n = -(a 1 / a 0 )t n-1 - (a 2/ a 0 )t n-2 = c 1 t n-1 + c 0 t n-2 This time we have a recurrence relation which is –second order –linear –homogenous –with constant coefficients. Dr Nazir A. Zafar Advanced Algorithms Analysis and Design K = 2: General Homogenous Recurrence

a 0 t n + a 1 t n-1 + …..+a k t n-k = 0 (1) If we put k = 3 then above equation becomes a 0 t n + a 1 t n-1 + a 2 t n-2 + a 3 t n-3 = 0 t n = -(a 1 / a 0 )t n-1 - (a 2/ a 0 )t n-2 - (a 3/ a 0 )t n-3 This is a recurrence relation which is –third order –linear –homogenous –with constant coefficients. Similarly we can have fourth order and higher order recurrence relations. Dr Nazir A. Zafar Advanced Algorithms Analysis and Design K = 3: General Homogenous Recurrence

a 0 t n + a 1 t n-1 + …..+a k t n-k = 0 (1) The recurrence is: Linear because it does not contain terms of the form t n-i.t n-j,t 2 n-i and so on Homogeneous because the linear combination of the t n-i equal to zero This is K th order because current term is linear combination of previous k number of terms Constant coefficients because all a i are constants Example: Consider Fibonacci sequence: f n = f n-1 + f n-2 This recurrence fits Eq. (1) after obvious rewriting. f n - f n-1 - f n-2 = 0 Dr Nazir A. Zafar Advanced Algorithms Analysis and Design Characteristics of General Recurrence

f n - f n-1 - f n-2 = 0 The Fibonacci sequence corresponds to a second homogeneous linear recurrence relation with constant coefficient, where k = 2, a 0 =1, a 1 = -1 a 2 = -1 Before we even start to look for solutions to Equation 1, it would be interesting to note that any linear combination of solutions is itself a solution. Dr Nazir A. Zafar Advanced Algorithms Analysis and Design Observation of Homogenous Recurrence

Statement: Prove that any linear combination of solutions of equation given below is also a solution. a 0 t n + a 1 t n-1 + …..+a k t n-k = 0 Proof: Assume that f n and g n are solutions of above equation and hence satisfy it, i.e. If we set t n = c.f n + d.g n for arbitrary constants c and d, we have to prove that t n is also solution, i.e., a 0 t n + a 1 t n-1 +……+ a k t n-k = 0 Dr Nazir A. Zafar Advanced Algorithms Analysis and Design Theorem

a 0 t n + a 1 t n-1 +……+ a k t n-k = a 0 (cf n + dg n ) + a 1 (cf n-1 + dg n-1 ) +...+a k (cf n-k +dg n-k ) = c( a 0 f n + a 1 f n a k f n-k ) +d(a 0 g n + a 1 g n a k g n-k ) = = c.0 + d.0 = 0 Hence a 0 t n +a 1 t n-1 +……+ a k t n-k = 0 Note : It is to be noted that this rule can be generalized to linear combinations of any number of solutions. Dr Nazir A. Zafar Advanced Algorithms Analysis and Design Theorem

Statement: If f n, g n and h n are solutions to recurrence below then their linear combination is also a solution a 0 t n + a 1 t n-1 + …..+a k t n-k = 0 Proof: As f n, g n and h n are solutions of above, hence Let t n = c.f n + d.g n + e.h n for arbitrary constants c, d and e, now we prove that t n is also solution, i.e., a 0 t n + a 1 t n-1 +……+ a k t n-k = 0, easy to prove Dr Nazir A. Zafar Advanced Algorithms Analysis and Design More Generalized Results

Why recurrence? Type of recurrences Techniques solving recurrences Generalized form of recurrence Linear combination of solutions itself solution Reusability of models when initial conditions are changed but pattern remains same It empowers the recursive models to be used in more general phenomena Easy implementation Conclusion Dr Nazir A. Zafar Advanced Algorithms Analysis and Design