1 Divide and Conquer Binary Search Mergesort Recurrence Relations CSE Lecture 4 – Algorithms II

2 Divide & Conquer Basic approach is … break problem in parts separately solve each part rejoin resulting partial solutions into whole

3 Binary Search Algorithm Case 1: A match occurs. The search is complete and mid is the index that locates the target. if (midValue == target) // found match return mid;

4 Binary Search Algorithm Case 2: The value of target is less than midvalue and the search must continue in the lower sublist. if (target < midvalue) // search lower sublist <search sublist arr[first]…arr[mid-1]

5 Binary Search Algorithm Case 3: The value of target is greater than midvalue and the search must continue in the upper sublist. if (target > midvalue)// search upper sublist

6 Successful Binary Search Search for target = 23 Step 1: Indices first = 0, last = 9, mid = (0+9)/2 = 4. Since target = 23 > midvalue = 12, step 2 searches the upper sublist with first = 5 and last = 9.

7 Successful Binary Search Step 2: Indices first = 5, last = 9, mid = (5+9)/2 = 7. Since target = 23 < midvalue = 33, step 3 searches the lower sublist with first = 5 and last = 7.

8 Successful Binary Search Step 3: Indices first = 5, last = 7, mid = (5+7)/2 = 6. Since target = midvalue = 23, a match is found at index mid = 6.

9 Unsuccessful Binary Search Search for target = 4. Step 1: Indices first = 0, last = 9, mid = (0+9)/2 = 4. Since target = 4 < midvalue = 12, step 2 searches the lower sublist with first = 0 and last = 4.

10 Unsuccessful Binary Search Step 2: Indices first = 0, last = 4, mid = (0+4)/2 = 2. Since target = 4 < midvalue = 5, step 3 searches the lower sublist with first = 0 and last 2.

11 Unsuccessful Binary Search Step 3: Indices first = 0, last = 2, mid = (0+2)/2 = 1. Since target = 4 > midvalue = 3, step 4 should search the upper sublist with first = 2 and last =2. However, since first >= last, the target is not in the list and we return index last = 9.

12 Binary Search Implementation int binSearch (const int arr[], int first, int last, int target) { int origLast = last; // original value of last while (first < last){ int mid = (first+last)/2; if (target == arr[mid]) return mid; // a match so return mid else if (target < arr[mid]) last = mid; // search lower sublist else first = mid+1; // search upper sublist } return origLast; // target not found }

13 Binary Search Analysis Each comparison divides the problem size in half Assume for simplicity that n = 2 k So, worst case … How many times do we divide n in half before there are no more values to check? T(n) = 1+ k = 1 + lg n Logarithmic Θ(lg n)

14 Merge Sort Recursively divide vector in half … … until sub-vectors reach size 1 Merge results into larger sorted sub-vectors while backing out of recursion

15 Mergesort() // sort v in range [first,last) template void mergeSort(vector & v, int first, int last) { if (first+1 < last) // more than 1 element { int mid = (first + last) / 2; mergesort(v, first, mid); // sort 1st half mergesort(v, mid, last); // sort 2nd half merge(v, first, mid, last); // merge halves }

16 Partitioning and Merging of Sublists in mergeSort()

17 Merge Algorithm Use a temporary vector Scan both sub-vectors left to right If value in vector A is smaller Append value to temp vector and advance in vector A Else value in vector B is smaller Append value to temp vector and advance in vector B Now one of vectors (A or B) is empty, so copy remaining values from non-empty vector to temp Now copy all values back from temp to area originally occupied by vectors A and B

18 Merge() template void merge(vector & v, int first, int mid, int last) { vector temp; // temporary vector needed int iA = first, iB = mid; while (iA < mid && iB < last) // values in both halves if (v[iA] < v[iB]) // smallest in 1 st half temp.push_back(v[iA++]); else // smallest in 2 nd half temp.push_back(v[iB++]); while (iA < mid) // still values in 1 st half temp.push_back(v[iA++]); while (iB < last) // still values in 2 nd half temp.push_back(v[iB++]); iA = first; for (int iV=0; iV<temp.size();iV++) // copy from temp v[iA++] = temp[iV]; }

19 Merge Algorithm Example The merge algorithm takes a sequence of elements in a vector v having index range [first, last). The sequence consists of two ordered sublists separated by an intermediate index, called mid.

20 Merge Algorithm Example …

21 Merge Algorithm Example …

22 Merge Algorithm Example …

23 Function calls in mergeSort()

24 Mergesort Efficiency (Informal) Assume n = 2 k Calls continue until subarrays have 1 element Levels in call tree (1 + k = 1 + lg n) Elements at each level Level 0 (1 vector of n elements) Level 1 (2 vectors of n/2 elements) Level 2 (4 vectors of n/4 elements) …. Level k (2 k vectors of n/2 k elements) Each level has n elements & merges across entire level requires Θ(n) effort Mergesort is always logarithmic Θ (n lg n)

25 Mergesort Efficiency (Formal) For any divide and conquer algorithm … Cost is expressed by the recurrence a is the number of subproblems n/b is the size of each subproblem D(n) is the cost to divide into subproblems C(n) is the cost to combine the results of the subproblems

26 Mergesort Efficiency … Divide: compute middle index – Θ(1) Conquer: solve 2 subproblems of size n/2 Combine: merge results – Θ(n) Putting it together … … and noting that Θ(n) + Θ(1) is still only Θ(n) we get the specific recurrence …

27 Mergesort Efficiency Rewriting the recurrence as …... and assuming n = 2 k We can create a cost tree by expanding the recurrence at each level T(n/2) T(n)cn T(n/4) cn/2 T(n/2) cn/2 T(n/4)

28 Mergesort Efficiency Total cost at each level of the tree is … 2c(n/2), 4c(n/4), 8c(n/8), … or in general … 2 i c(n/2 i ) = cn At the bottom level there are n subarrays of 1 element each and the cost there is … n*T(1) = cn Depth of tree for sort of n = 2 k elements is … k = lg n, so the tree has … 1 + k = 1 + lg n levels, but only n lg n merges Therefore, the total cost of the algorithm is … cn lg n + cn, which is logarithmic Θ(n lg n)

29 Asymptotic Growth Notation Big Theta A function f(n) is Θ(g(n)) if c 1 g(n) ≤ f(n) ≤ c 2 g(n), for some c 1, c 2 and all n ≥ n 0 Example: 3n 2 + 2n is Θ(n 2 ), Let c 1 =3 and c 2 =5. 3n 2 ≤ 3n 2 + 2n ≤ 5n 2 for all n ≥ 1 Essentially f(n) is bounded by scalar multiples of g(n) for sufficiently large values of n So, g(n) is an asymptotically tight bound for f(n)

30 Asymptotic Growth Notation Big O A function f(n) is O(g(n)) if f(n) ≤ cg(n), for some c and all n ≥ n 0 Example: 3n 2 + 2n is O(n 2 ), Let c=5. 3n 2 + 2n ≤ 5n 2 for all n ≥ 1 Essentially f(n) is upper bounded by a scalar multiple of g(n) for sufficiently large values of n So, g(n) is an asymptotic upper bound for f(n)

31 Asymptotic Growth Notation Big Omega A function f(n) is Ω(g(n)) if cg(n) ≤ f(n), for some c and all n ≥ n 0 Example: 3n 2 + 2n is Ω(n 2 ), Let c=3. 3n 2 ≤ 3n 2 + 2n for all n ≥ 1 Essentially f(n) is lower bounded by a scalar multiple of g(n) for sufficiently large values of n So, g(n) is an asymptotic lower bound for f(n)