Algorithm Analysis Lakshmish Ramaswamy

Formal Definitions Big-Oh: T(N) is O(F(N)) if there exists positive constants N 0 and c such that T(N) N 0 Big-Omega: T(N) is Ω(F(N)) if there exists positive constants N 0 and c such that T(N) >= cF(N) for all N => N 0 Big-Theta: T(N) is Θ(F(N)) if and only if T(N) is both O(F(N)) and Ω(F(N)) Little-Oh: T(N) is o(F(N)) if and only if T(N) is O(F(N)) and T(N) is not Θ(F(N))

Meaning of Notations

Logarithm Definition – For any B, N > 0 log B N = k if B k = N Base has no effect on Big-Oh For any B > 1 log B N = O(log N) –log B (N) = log 2 (N)/log 2 (B) Log functions are characterized by slow growth –log 10 = 2.3; log 1000 = 6.9; log = 11.5

Examples of Logarithm Bits required to represent N (in binary) –Upper limit of log 2 N Repeated doubling –Starting from X = 1, how many times should X be doubled before it is at least as large as N? Repeated halving –Starting from X = N, how many times should X be halved before it is lesser than or equal to 1?

Sequential Search Problem – In an array of numbers, search whether a given number is present public static boolen sequentialsearch(int[] A, num) for (i = 0; i < A.length; i++){ if (A[i] == num) return(1); } return(0) O(N) algorithm

Binary Search Algorithm Problem – In an sorted array of numbers, search whether a given number is present Can the fact that the array is sorted be used to reduce comparisons?

Illustration Depending upon the relationship part of the array can be safely eliminated from future comparisons

Logic of Binary Search Compare given number to center of the array If found terminate Eliminate one half of the array Continue until no more comparisons left

Binary Search Algorithm public static int (int[] Arr, int given){ int high = Arr.length-1; int low = 0; int mid; while(low <= high){ mid = (low+high)/2; if(Arr[mid] < given) low = mid+1; else if(Arr[mid] > given) high = mid-1; else return (mid); } return(-1); }

Trace [2, 4, 6, 9, 13, 16, 20, 24, 26, 29, 30, 32, 36, 38, 41, 50] given = 36 given = 4 given =3

Analysis At most two comparisons at each iteration Constant time per comparison Repeated halving log 2 N comparisons O(log N) algorithm

Sorting Algorithms Problem – Given an array, rearrange the elements such that they are in increasing (or decreasing order) Fundamental operation with wide range of applications Several algorithms –Selection sort –Insertion sort –Bubble sort –Merge sort –Quick sort –Radix sort

Selection Sort Logic – In i th iteration select the i th smallest element and place it i th location How to select the i th minimum element –Repeated use of minimum element algorithm

Selection Sort Algorithm public static void SelectionSort(int[] Arr){ for(int i = 0; i < Arr.length-1; i++){ minElement = Arr[i]; minIndex = i; for(j = (i+1); j < Arr.length; j++) if(Arr[j] < minElement){ minElement = Arr[j]; minIndex = j; } swap(Arr[i], Arr[minIndex]); }

Trace [ 9, 3, 8, 12, 1, 5, 22, 18, 14, 2]

Analysis Constant time for comparison (N-1) comparisons when i = 0 (N-2) comparisons when i = 1 1 comparison when i = (N-2) Total comparisons = 1+2+…+(N-2) + (N-1) Total comparisons = (N-1)N/2 O(N 2 ) algorithm