UNIT- I Problem solving and Algorithmic Analysis Problem solving principles: Classification of problem, problem solving strategies, classification of time complexities (linear, logarithmic etc) problem subdivision { Divide and Conquer strategy}. Asymptotic notations, lower bound and upper bound: Best case, worst case, average case analysis, amortized analysis. Performance analysis of basic programming constructs. Recurrences: Formulation and solving recurrence equations using Master Theorem.

Recurrences The expression: is a recurrence. Recurrence: an equation that describes a function in terms of its value on smaller functions The expression: is a recurrence.

Recurrence Examples

Solving Recurrences Substitution method Iteration method Master method

The Master Theorem Given: a divide and conquer algorithm An algorithm that divides the problem of size n into a subproblems, each of size n/b Let the cost of each stage (i.e., the work to divide the problem + combine solved subproblems) be described by the function f(n) Then, the Master Theorem gives us a cookbook for the algorithm’s running time:

The Master Theorem if T(n) = aT(n/b) + f(n) then

Divide and Conquer Recursive in structure Divide the problem into sub-problems that are similar to the original but smaller in size Conquer the sub-problems by solving them recursively. If they are small enough, just solve them in a straightforward manner. Combine the solutions to create a solution to the original problem Comp 122

Control abstraction of D&C Method 1. Algorithm DAndC(P) 2. { 3. if small(P) then return S(P); 4. else 5. { 6. divide P into smaller instances P1, P2… Pk, k>=1; Apply DAndC to each of these sub problems; return combine (DAndC(P1), DAndC(P2),..,DAndC(Pk)); 9. } 10. }

If size of P is n and the size of the k subproblems are n1, n2 … If size of P is n and the size of the k subproblems are n1, n2 …..nk ten computing time of DandC is described by the recurrence relation T(n)= g(n) n small T(n1)+T(N2)+…+T(nk)+f(n) otherwise T(n) time for DAndC on any input of size n g(n) time to compute the answer directly for small input f(n) time for dividing P and combining the solution of subproblems.

The complexity of many divide and conquer algorithm is given by recurrence of the form T(n)= T(1) n small aT(n/b)+f(n) otherwise Where a and b are known constants. We assume that T(1) is known and n is power of b that is n=bk

Example of Divide and Conquer Binary Search Merge Sort Quick Sort

Binary Search Let ai, 1 ≤ i ≤ n be a list of elements which are sorted in nondecreasing order. Consider the problem of determining whether a given element x is present in the list. If x is present, we are to determine a value j such that aj = x. If x is not in the list then j is to be set to zero. Divide-and-conquer suggests breaking up any instance I = (n, a1, ... , an. x) of this search problem into subinstances.

Binary Search Algorithm BINSRCH(a, i, l, x) //given an array A(i:l) of elements in nondecreasing order, 1 ≤ i ≤ l determine if x is present, and if so, set j such that x = a[j] else return 0 { if (l=i) then if( x=a[i]) then return i; Else return 0; } Else { //reduce P into a smaller subproblem mid= ⌊ (i+l)/2 ⌋; if (x=a[mid] )then return mid; Else if (x<a[mid]) then Return BINSRCH(a, i, mid-1,x); Else Return BINSRCH(a, i, mid+1,x);

Binary Search Ex. Binary search for 33. 6 13 14 25 33 43 51 53 64 72 84 93 95 96 97 1 2 3 4 5 6 7 8 9 10 11 12 13 14 lo hi

Binary Search Ex. Binary search for 33. 6 13 14 25 33 43 51 53 64 72 84 93 95 96 97 1 2 3 4 5 6 7 8 9 10 11 12 13 14 lo mid hi

Binary Search Ex. Binary search for 33. 6 13 14 25 33 43 51 53 64 72 84 93 95 96 97 1 2 3 4 5 6 7 8 9 10 11 12 13 14 lo hi

Binary Search Ex. Binary search for 33. 6 13 14 25 33 43 51 53 64 72 84 93 95 96 97 1 2 3 4 5 6 7 8 9 10 11 12 13 14 lo mid hi

Binary Search Ex. Binary search for 33. 6 13 14 25 33 43 51 53 64 72 84 93 95 96 97 1 2 3 4 5 6 7 8 9 10 11 12 13 14 lo hi

Binary Search Ex. Binary search for 33. 6 13 14 25 33 43 51 53 64 72 84 93 95 96 97 1 2 3 4 5 6 7 8 9 10 11 12 13 14 lo mid hi

Binary Search Ex. Binary search for 33. 6 13 14 25 33 43 51 53 64 72 84 93 95 96 97 1 2 3 4 5 6 7 8 9 10 11 12 13 14 lo hi

Binary Search Ex. Binary search for 33. 6 13 14 25 33 43 51 53 64 72 84 93 95 96 97 1 2 3 4 5 6 7 8 9 10 11 12 13 14 lo hi mid

Binary Search Ex. Binary search for 33. 6 13 14 25 33 43 51 53 64 72 84 93 95 96 97 1 2 3 4 5 6 7 8 9 10 11 12 13 14 lo hi mid

Successful Search Unsuccessful search Ө(1) Best case Ө(log n) Average Case Worst Case

Merge sort Given a sequence of n. elements (also called keys) A(l), ... , A(n) the general idea is to imagine them split into two sets A(1), . , A( n/2) and A( n/2 + 1), ... , A(n). Each set is individually sorted and the resulting sequences are merged to produce a single sorted sequence of n elements.

Algorithm MERGESORT(low, high) //A(low : high) is a global array containing high - low + 1 ≥0 //values which represent the elements to be sorted. if low < high then mid  ⌊(low + high)/2 ⌋ //find where to split the set// call MERGESORT(low, mid) //sort one subset// call MERGESORT(mid + 1, high) //sort the other subset// call MERGE(low, mid, high) //combine the results/ Endif end MERGESORT

T(n) = 2T(n/2) + (n) if n > 1 Analysis of Merge Sort Running time T(n) of Merge Sort: Divide: computing the middle takes (1) Conquer: solving 2 subproblems takes 2T(n/2) Combine: merging n elements takes (n) Total: T(n) = (1) if n = 1 T(n) = 2T(n/2) + (n) if n > 1  T(n) = (n log n) Comp 122

If the time for the merging operation is proportional to n then the computing time for mergesort is described by the recurrence relation T(n) = a n = 1, a constant 2T(n/2) + cn n > l, c constant By using recurrence we got T(n) = O(n log2n)

Merging The key to Merge Sort is merging two sorted lists into one, such that if you have two lists X (x1x2…xm) and Y(y1y2…yn) the resulting list is Z(z1z2…zm+n) Example: L1 = { 3 8 9 } L2 = { 1 5 7 } merge(L1, L2) = { 1 3 5 7 8 9 }

Merging (cont.) X: 3 10 23 54 Y: 1 5 25 75 Result:

Merging (cont.) X: 3 10 23 54 Y: 5 25 75 Result: 1

Merging (cont.) X: 10 23 54 Y: 5 25 75 Result: 1 3

Merging (cont.) X: 10 23 54 Y: 25 75 Result: 1 3 5

Merging (cont.) X: 23 54 Y: 25 75 Result: 1 3 5 10

Merging (cont.) X: 54 Y: 25 75 Result: 1 3 5 10 23

Merging (cont.) X: 54 Y: 75 Result: 1 3 5 10 23 25

Merging (cont.) X: Y: 75 Result: 1 3 5 10 23 25 54

Merging (cont.) X: Y: Result: 1 3 5 10 23 25 54 75

Merge Sort Example 99 6 86 15 58 35 4

Merge Sort Example 99 6 86 15 58 35 4 99 6 86 15 58 35 86 4

Merge Sort Example 99 6 86 15 58 35 4 99 6 86 15 58 35 86 4 99 6 86 15 58 35 86 4

Merge Sort Example 99 6 86 15 58 35 4 99 6 86 15 58 35 86 4 99 6 86 15 58 35 86 4 99 6 86 15 58 35 86 4

Merge Sort Example 99 6 86 15 58 35 4 99 6 86 15 58 35 86 4 99 6 86 15 58 35 86 4 99 6 86 15 58 35 86 4 4

Merge Sort Example 99 6 86 15 58 35 86 4 Merge 4

Merge Sort Example 6 99 15 86 35 58 4 86 99 6 86 15 58 35 86 4 Merge

Merge Sort Example 6 15 86 99 4 35 58 86 6 99 15 86 58 35 4 86 Merge

Merge Sort Example 4 6 15 35 58 86 99 6 15 86 99 4 35 58 86 Merge

Merge Sort Example 4 6 15 35 58 86 99