Algorithm Design Techniques: Induction Chapter 5 (Except Section 5.6)

Induction Main objective: Using induction or inductive reasoning as a “recursive” algorithm design technique Why recursion –Concise algorithms for complex problems can be developed –The algorithms are easy to comprehend –Development time –Proof of correctness of the designed algorithm is usually simple.

How Does Induction Produce Recursive Algorithms? Having a problem with input size n, it is sometimes easier to start with a solution to the problem with a smaller size and extend the solution to include the input size n.

Problems to Discuss Selection sort Insertion sort Radix sort Integer exponentiation Evaluating polynomials Finding the majority element

Recursive Selection Sort Induction Hypothesis: We know how to sort A[2..n] Inductive Reasoning: We sort A[1..n] as follows –Find the minimum A[j], 1  j  n –Swap(A[1],A[j]) –Sort A[2..n] // induction hypothesis What is the base case? What is the initial call to the sorting algorithm? This kind of recursion is called tail-recursion.

Recursive Selection Sort Algorithm Algorithm SELECTIONSORTREC Input: An array A[1..n] of n elements. Output: A[1..n] sorted in nondecreasing order. Procedure sort(i) {Sort A[i..n]} 1.if i < n then Begin 2. k := i; 3. for j := i + 1 to n 4. if A[j] < A[k] then k := j; 5. if k != i then Swap(A[i], A[k]); 6. sort(i + 1); End

Complexity Analysis of Recursive Selection Sort What is the recurrence relation that describes the number of comparisons carried out by the algorithm? What is the solution to the recurrence?

Recursive Insertion Sort Induction Hypothesis: We know how to sort A[1..n-1] Inductive Reasoning: We sort A[1..n] as follows: –Sort A[1..n-1] // induction hypothesis –Insert A[n] in its proper position in A[1..n] This may involve copying zero or more elements one position ahead in order to insert A[n]

Recursive Insertion Sort Algorithm Algorithm INSERTIONSORTREC Input: An array A[1..n] of n elements. Output: A[1..n] sorted in non-decreasing order. Procedure sort(i) {Sort A[1..i]} 1. if i >1 then 2. sort(i – 1){Recursive Call} 3. x := A[i] 4. j := i – 1 5. while j > 0 and A[j] > x 6. A[j + 1] := A[j] 7. j := j – 1 end while 8. A[j + 1] := x end if

Complexity Analysis of Recursive Insertion Sort Unlike selection sort, the analysis has to differentiate between the best case and the worst case. Why? Recurrence of the best case: Solution to the best case: Recurrence of the worst case: Solution to the worst case:

Radix Sort Treats keys as numbers in a particular radix or base. This is NOT an elements comparison-based sorting algorithm It only works under the following assumption:

Radix Sort Derivation Assume that our keys are of the form d k d k – 1 … d 1 Induction Hypothesis: Suppose that we know how to sort numbers lexicographically according to their least k – 1 digits, d k – 1, d k – 2, …, d 1, k > 1. Inductive reasoning: We sort the numbers based on their first k digits as follows: –Use induction hypothesis to sort the numbers based on their 1..k-1 digits. –Sort the numbers based on their corresponding k th digits

Description of Radix Sort Algorithm 1.Distribute the input numbers into 10 sublists L 0, L 1,…, L 9 according to the least significant digit, d 1. 2.Form a new list, we denote by main list, by removing the input from the 10 lists, starting from L 0, L 1, …etc. in order. 3.Distribute the main list into the 10 sublists according to the second digit, d 2, and then repeat step 2. 4.Repeat step 3 for the rest of the digits in order of d 3, d 4, …, d k.

Example Sort the following numbers using radix sort:

Radix Sort Algorithm Algorithm RADIXSORT Input: A linked list of numbers L = {a 1, a 2,…,a n } and k, the maximum number of digits. Output: L sorted in nondecreasing order. 1. for j ←1 to k do 2. Prepare 10 empty lists L 0, L 1, …, L while L is not empty 4. a ← next element in L. Delete a from L. 5. i ← j th digit in a. Append a to list L i. 6. end while 7. L ← L 0 8. for i ←1 to 9 do 9. L ← L. L i {append list L i to L} 10. end for 11. end for 12. return L

Time and Space Complexity Analysis Time Complexity Space Complexity

Integer Exponentiation What is the straightforward algorithm to raise x to the power n? Such an algorithm is exponential in input size!!!!!!!!

Algorithm Derivation Assume we have integer n which is represented in binary as (b k b k-1 …b 1 b 0 ) 2 Induction Hypothesis: Assume that we know how to compute x  n/2 . Inductive Reasoning: We raise x to the power n as follows: –Use induction hypothesis to compute y=x  n/2 . x  n/2  –If n is even, x n = y –If n is odd, x n = x. y

Integer Exponentiation Algorithm ALGORITHM EXPRECURSIVE Input: A real number x and a nonnegative integer n. Output: x n. /* the first call to the algorithm is power(x,n) */ Procedure power(x, m) {Compute x m } 1. if m=0 then y ← 1 2. else 3. y ← power(x,  m/2  ) 4. y ← y 2 5. if m is odd then y ← x. y end if 7. return y

Complexity Analysis of the Algorithm When n = 0, the number of multiplications is Therefore, T(0)= Best Case Analysis: –When does the best case occur? –What is the recurrence equation? Solution? Worst Case Analysis: –When does the worst case occur? –What is the recurrence equation? Solution?

Iterative Exponentiation Algorithm Assume we have integer n which is represented in binary as (b k b k-1 …b 1 b 0 ) 2  Starting with y = 1, scan the binary digits of the number n from left to right (j = k down to 0): –If b j = 0, square y –If b j = 1, square y and multiply it by x Example: Compute 2 12 using the iterative exponentiation algorithm

Polynomial Evaluation A polynomial of degree n is generally written as: where, a 0, a 1, …, a n and x, is a sequence of n + 2 real numbers. Our objective here is to evaluate this general polynomial at the point x.

Polynomial Representation Dense Representation –Disadvantage: Sparse Representation –Disadvantage: Examples:

Two Straightforward Solutions 1.Evaluate each term a i x i separately What is its time complexity? 2.Compute x i by multiplying x by the previously computed value x i-1 How many multiplications, assignments, and additions do we have?

Horner’s Rule

Induction Hypothesis: Suppose we know how to evaluate the following: Inductive Reasoning: We evaluate P n (x) as follows: –Using induction hypothesis, we know how to evaluate P n-1 (x) –P n (x) = x P n-1 (x) + a 0 What is the base step?

Horner’s Algorithm Input: A sequence of n+2 real numbers a 0, a 1, …, a n, x Output: P n (x)=a n x n +a n-1 x n-1 +…+a 1 x+a 0 p ← a n for j ← 1 to n do p ← x. p + a n-j return p How many multiplications, additions, and assignments do we have?

Finding the Majority Element Definition: Given a sequence of n integers, say A[1..n], an element in A is called the majority element if it appears more than times in A. Objective: To use induction to develop an efficient algorithm that finds the majority element in a sequence of n integers, if such element exists. –In any sequence of n elements, how many majority elements can there be?

Examples What is the majority element in the following examples:  1, 6, 8, 3, 4, 1, 6, 8, 1, 6, 1, 1  1, 6, 8, 3, 4, 1, 6, 1, 6, 1, 1  1, 6, 3, 4, 1, 6, 1, 6, 1, 1  1, 6, 4, 1, 6, 1, 6, 1, 1

Various Algorithms to Find the Majority The brute-force method: –Its cost: Compare elements from 1 to with every other element. –Its cost: Sort the elements and count how many times each element appears in the sequence. –Its cost

Using Induction to Find the Majority Theorem: If two different elements in the original sequence are removed, then the majority element in the original sequence remains the majority element in the new sequence. Proof:

Procedure Candidate This procedure applies the previous theorem by “eliminating” pairs of distinct elements and returning a possible majority element: Procedure candidate (m) 1. j  m; c  A[m]; count  1 2. while j 0 3. j  j if A[j]=c then count  count else count  count – 1 6. end while 7. if j = n then return c 8.else return candidate(j+1)

Finding the Majority The idea of algorithm Majority is as follows: –Find a candidate majority element using the procedure candidate –Check whether this candidate element is a majority element or not.

Algorithm Majority Algorithm MAJORITY Input: An array A[1..n] of n elements. Output: The majority element if it exists; otherwise none. 1.c  candidate(1) 2.count  0 3.for j  1 to n 4. if A[j]=c then count  count end for 6.if count >  n/2  then return c 7.else return none

Example

Time Complexity Analysis of Algorithm Majority The time complexity is calculated in terms of the number of element comparisons. How many element comparisons do we have in Procedure candidate? How many element comparisons do we have in Algorithm Majority (excluding those performed within Procedure candidate) Hence, Algorithm Majority will perform …………….. Comparisons in total. Which of all the algorithms discussed, related to finding the majority element, is better?