1 Efficiency of Algorithms: Analyzing algorithm segments, Insertion sort algorithm Logarithmic Orders Binary Search Algorithm

2 Computing an Order of an Algorithm Segment Consider the following algorithm segment: max:=a[1]; min:=b[1] for i:=2 to n if max < a[i] then max:=a[i] ; if min > b[i] then min:=b[i] next i if min ≥ max then med:=(min+max)/2 Analysis: Number of loop iterations: n-1 Comparisons in each iteration: 2 Thus, elementary operations in the loop: 2(n-1) Operations after the loop: 3 Total number of operations in the segment: 2(n-1)+3 = 2n+1 Order of the segment: O(n)

3 The Insertion Sort Algorithm Insertion sort arranges the elements of one-dimensional array a[1], …, a[n] into increasing order. for i:=2 to n (insert a[i] into the sorted sequence a[1],…,a[i-1]) key:=a[i] for j:=1 to i-1 if key<a[j] then goto (*) next j (*) (shift the values of a[j],…,a[i-1] to a[j+1],…,a[i]) for k:=i to j+1 a[k]:=a[k-1] next k a[j]:=key next i

4 The Insertion Sort Algorithm: Example Arrange array into increasing order

5 The Insertion Sort Algorithm: Analysis For each i (outer loop), maximum number of elementary operations is i – 1. Thus, the total number of elementary operations: 1+2+…+(n-1) = n(n-1)/2 =.5n 2 -.5n The insertion sort algorithm is O(n 2 ).

6 Logarithmic function Definition: The logarithmic function with base b (b>0, b  1) is the following function from R + to R: log b (x) = the exponent to which b must raised to obtain x. Symbolically, log b x = y  b y = x. Property: If the base b>1, then the logarithmic function is increasing: if x 1 <x 2, then log b (x 1 ) < log b (x 2 ). Note: Logarithmic function grows very slowly, e.g., log 2 (1,024)=10, log 2 (1,048,576)=20.

A property of logarithmic function Proposition 1: If k is an integer and x is a real number with 2 k  x < 2 k+1, then  log 2 x  = k. Proof: 2 k  x < 2 k+1 log 2 (2 k )  log 2 (x) < log 2 (2 k+1 ) (since log 2 x increasing) k  log 2 (x) < k+1 (by definition of log 2 x)  log 2 x  = k (by definition of floor function) ■

8 An application of logarithms Question: Given a positive integer n, how many binary digits are needed to represent n? Solution: Binary representation of n: 1c k-1 c k-2 …c 2 c 1 c 0 which corresponds to n = 2 k + c k-1 ∙2 k-1 + … + c 2 ∙2 2 + c 1 ∙2 + c 0 Since c i  1, n = 2 k + c k-1 ∙2 k-1 + … + c 2 ∙2 2 + c 1 ∙2 + c 0  2 k + 2 k-1 + … = (2 k+1 -1) / (2-1) (as a sum of geometric sequence) = 2 k+1 -1 < 2 k+1 (1) On the other hand, n = 2 k + c k-1 ∙2 k-1 + … + c 2 ∙2 2 + c 1 ∙2 + c 0 ≥ 2 k (2) Combining (1) and (2): 2 k  n < 2 k+1 (3) Based on (3) and Proposition 1, k =  log 2 n  and the number of binary digits is  log 2 n  + 1.

9 Exponential and Logarithmic Orders  For all real numbers b and r with b>1 and r>0 and for all sufficiently large values of x,  log b x  x r ; ( which implies that log b x is O(x r ) )  x r  b x ( which implies that x r is O(b x ) )  For all real numbers b with b>1 and for all sufficiently large values of x, x  x log b x  x 2 (which implies that x is O(x log b x) and x log b x is O(x 2 ) )

Binary Search Algorithm The algorithm searches for an element x in an ascending array of elements a[1],…,a[n]. Algorithm body: index:=0, bot:=1, top:=n while (top ≥ bot and index=0) mid :=  (bot+top) / 2  if a[mid] = x then index := mid if a[mid] > x then top := mid-1 else bot := mid+1 end while Output: index (If index has the value 0 then x is not in the array; otherwise, index gives the index of the array where x is located.)

11 Binary Search Algorithm: Example Suppose a[1]=Amy, a[2]=Bob, a[3]=Dave, a[4]=Erin, a[5]=Jeff, a[6]=John, a[7]=Larry, a[8]=Mary, a[9]=Mike, a[10]=Sam, a[11]=Steve, a[12]=Tom. (sorted in alphabetical order) Search for x=Erin. The table tracing the binary search algorithm: index04 bot114 top1255 mid634

12 The Efficiency of the Binary Search Algorithm At each iteration, the length of the new subarray to be searched is approximately half of the previous one. If n = 2 k +m, where 0  m < 2 k, then n can be split approximately in half k times. Since 2 k  n < 2 k+1, then k =  log 2 n  (by proposition 1) Thus, the number of iterations of the while loop in a worst-case execution of the algorithm is  log 2 n  +1. The number of operations in each loop is constant (doesn’t increase with n). Thus, the binary search algorithm is O(log 2 n).