1 Data Structures Performance Analysis

2 Fundamental Concepts Some fundamental concepts that you should know: –Dynamic memory allocation. –Recursion. –Performance analysis.

3 Performance Analysis There are problems and algorithms to solve them. Problems and problem instances. Example: Sorting data in ascending order. –Problem: Sorting –Problem Instance: e.g. sorting data ( ) –Algorithms: Bubble sort, Merge sort, Quick sort, Selection sort, etc. Which is the best algorithm for the problem? How do we judge?

4 Performance Analysis Two criteria are used to judge algorithms: (i) time complexity (ii) space complexity. Space Complexity of an algorithm is the amount of memory it needs to run to completion. Time Complexity of an algorithm is the amount of CPU time it needs to run to completion.

5 Space Complexity Memory space S(P) needed by a program P, consists of two components: –A fixed part: needed for instruction space (byte code), simple variable space, constants space etc.  c –A variable part: dependent on a particular instance of input and output data.  S p (instance) S(P) = c + S p (instance)

6 Space Complexity: Example 1 1. Algorithm abc (a, b, c) 2. { 3. return a+b+b*c+(a+b-c)/(a+b)+4.0; 4. } For every instance 3 computer words required to store variables: a, b, and c. Therefore S p ()= 3. S(P) = 3.

7 Space Complexity: Example 2 1. Algorithm Sum(a[], n) 2. { 3. s:= 0.0; 4. for i = 1 to n do 5. s := s + a[i]; 6. return s; 7. }

8 Space Complexity: Example 2. Every instance needs to store array a[] & n. –Space needed to store n = 1 word. –Space needed to store a[ ] = n floating point words (or at least n words) –Space needed to store i and s = 2 words S p (n) = (n + 3). Hence S(P) = (n + 3).

9 Time Complexity Time required T(P) to run a program P also consists of two components: –A fixed part: compile time which is independent of the problem instance  c. –A variable part: run time which depends on the problem instance  t p (instance) T(P) = c + t p (instance)

10 Time Complexity How to measure T(P)? –Measure experimentally, using a “stop watch”  T(P) obtained in secs, msecs. –Count program steps  T(P) obtained as a step count. Fixed part is usually ignored; only the variable part t p () is measured.

11 Time Complexity What is a program step? –a+b+b*c+(a+b)/(a-b)  one step; –comments  zero steps; – while ( ) do  step count equal to the number of times is executed. – for i= to do  step count equal to number of times is checked.

12 Time Complexity: Example 1 Statements S/EFreq.Total 1Algorithm Sum(a[],n)0-0 2{0-0 3 S = 0.0;111 4 for i=1 to n do1n+1 5 s = s+a[i];1nn 6 return s;111 7}0-0 2n+3

13 Time Complexity: Example 2 Statements S/EFreq.Total 1Algorithm Sum(a[],n,m) 0-0 2{ for i=1 to n do; 1n+1 4 for j=1 to m do 1n(m+1) 5 s = s+a[i][j]; 1nm 6 return s; 111 7} 0-0 2nm+2n+2

14 Performance Measurement Which is better? –T(P1) = (n+1) or T(P2) = (n 2 + 5). –T(P1) = log (n 2 + 1)/n! or T(P2) = n n (nlogn)/n 2. Complex step count functions are difficult to compare. For comparing, ‘rate of growth’ of time and space complexity functions is easy and sufficient.

15 Big O Notation Big O of a function gives us ‘rate of growth’ of the step count function f(n), in terms of a simple function g(n), which is easy to compare. Definition: [Big O] The function f(n) = O(g(n)) (big ‘oh’ of g of n) iff there exist positive constants c and n 0 such that f(n) =n 0. See graph on next slide. Example: f(n) = 3n+2 is O(n) because 3n+2 = 2. c = 4, n 0 = 2. Here g(n) = n.

16 Big O Notation = n 0

17 From DS and Prog. Design in C++, R. Kruse et al.

18 From DS and Prog. Design in C++, R. Kruse et al.

19 From DS and Prog. Design in C++, R. Kruse et al.

20 Big O Notation Example: f(n) = 10n 2 +4n+2 is O(n 2 ) because 10n 2 +4n+2 =5. Example: f(n) = 6*2 n +n 2 is O(2 n ) because 6*2 n +n 2 =4. Algorithms can be: O(1)  constant; O(log n)  logrithmic; O(nlogn); O(n)  linear; O(n 2 )  quadratic; O(n 3 )  cubic; O(2 n )  exponential.

21 Big O Notation Now it is easy to compare time or space complexities of algorithms. Which algorithm complexity is better? –T(P1) = O(n) or T(P2) = O(n 2 ) –T(P1) = O(1) or T(P2) = O(log n) –T(P1) = O(2 n ) or T(P2) = O(n 10 )

22 Some Results Sum of two functions: If f(n) = f 1 (n) + f 2 (n), and f 1 (n) is O(g 1 (n)) and f 2 (n) is O(g 2 (n)), then f(n) = O(max(|g 1 (n)|, | g 2 (n) | )). Product of two functions: If f(n) = f 1 (n)* f 2 (n), and f 1 (n) is O(g 1 (n)) and f 2 (n) is O(g 2 (n)), then f(n) = O(g 1 (n)* g 2 (n)).