Data Structures, Algorithms & Complexity GRIFFITH COLLEGE DUBLIN Data Structures, Algorithms & Complexity Analysis of Algorithms Lecture 4

Efficiency of Algorithms Computers are so fast nowadays, why bother about efficiency? Suppose a travelling salesman has to visit 100 different locations over a period of time. What is the shortest route? Although phrased in terms of the travelling salesperson, many optimization problems can be formulated as TSP. The total number of distinct tours is 100!  30100 A supercomputer checking 100 billion tours per second can check about 1020 tours in one year. Millions of years needed to check all tours. Lecture 4

Sample Table of Running Times Number of microseconds since the Big Bang has 24 digits Number of protons in the universe has 126 digits Lecture 4

Sample Analysis Insertion-Sort(A) cost times for j = 2 to length[A] c1 n key = A[j-1] c2 n-1 i = j – 2 c3 n-1 while i >= 0 and A[i] > key c4 A[i+1] = A[i] c5 i = i – 1 c6 endwhile A[i+1] = key c7 n-1 endfor endalg Lecture 4

Express Formula in terms of N Here, tj is the number of times the while loop is executed for that value of j Therefore, T(n)= running time on input of size n = c1n + c2(n-1) + c3(n-1) + c4 + c5 + c6 + c7(n-1) Can we simplify this equation to give us one in terms of n only? Yes, but T(n) can depend on which input of size N is given Lecture 4

Best Case Scenario Best Case for the insertion sort is that the array is already sorted. In that case tj = 1 for every j. Therefore, T(n) = c1n + c2(n-1) + c3(n-1) + c4(n-1) + c7(n-1) = (c1 + c2 + c3 +c4 + c7)n – (c2 + c3 + c4 + c7) This running time can be expressed in the form, T(n) = an + b where a and b depend on the costs ci. From this we can see that, in the best case, T(n) is a linear function of n. Lecture 4

Worst Case Scenario Worst case for the insertion sort is that the array is reverse sorted order. In this case, tj = j for every j Therefore, T(n) = c1n + c2(n-1) + c3(n-1) + c4 + c5 + c6 + c7(n-1) To simplify this we need to solve the summations, and = 2 + 3 + ... + n Can I find a form of this in terms of n? Lecture 4

in terms of n First add 1 to both sides. + 1 = 1 + 2 + 3 + ... + n Now, reverse the order and add it to itself. 2( + 1) = 1 + 2 + 3 + ... + (n-1) + n + n + (n-1) + ... + 1 = (n + 1) + (n + 1) + ... + (n + 1) and there are n terms in the series, = n (n + 1) Now simplify the left hand side, 2( + 1) = n(n + 1) + 1 = n(n + 1) => = n(n + 1) - 1 2 2 Lecture 4

in terms of n = 1 + 2 + 3 + ... + n-1 Now, reverse and add again, = n + n + n + .... + n and there are (n-1) terms. = n(n - 1) and simplifying the left-hand side, = Lecture 4

Substitute Results in Original T(n) = c1n + c2(n-1) + c3(n-1) + c4 + c5 + c6 + c7 (n-1) = c1n + c2(n-1) + c3(n-1) + c4 + c5 + c6 + c7(n-1) = 0.5(c4 + c5 + c6)n2 + (c1 + c2 + c3 + c4/2– c5/2 - c6/2 + c7) n – (c2 + c3 + c4+ c7) This can be expressed as, an2 + bn + c for constants a, b and c that depend on the statement costs ci. Lecture 4

Worst Case Scenario Thus, for this worst case, T(n) is a quadratic function of n. In most case we concentrate on worst-case running times because, it is an upper bound on running time for any input it often happens in practice average case is often roughly as bad as the worst case. Lecture 4

Summary Efficient algorithms are important because many real world problems can be huge Analysis of Insertion Sort Look at Summations which often need to be solved when dealing with loops With any summation try to simplify it to a form similar to that used in text Concentrate on worst case scenario. This gives us a good benchmark to judge from Lecture 4