1. Prof. Amr Goneid, AUC1 Analysis & Design of Algorithms (CSCE 321) Prof. Amr Goneid Department of Computer Science, AUC Part 3. Time Complexity Calculations

2. Prof. Amr Goneid, AUC2 Time Complexity Calculations

3. Prof. Amr Goneid, AUC3 Time Complexity Calculations Performance Measurement & Modeling Performance Measurement Performance Analysis (modeling) Evaluating Number of Operations T(n) Examples of Algorithm Analysis

4. Prof. Amr Goneid, AUC4 1. Performance Measurement and Modeling Algorithm Actual Code Actual or Pseudo code Measurement Runs Math. Model T(n) and Bounds

5. Prof. Amr Goneid, AUC5 2. Performance Measurement We measure the performance of an algorithm by time measurement or by counting number of operations. Usually called Amortized Analysis: Repeat measuring the running time m times then divided by m. Important issues are: Choice of Bound Clocking or counting domains Choice of Data Sizes Choice of Data Set for the Bound

6. Prof. Amr Goneid, AUC6 Time Measurement Example Set repetition count m; Set problem size n; t acc = 0; Repeat on m { start time t1 = GetTime(); Invoke A(n); end time t2 = GetTime(); t acc = t acc + (t2 - t1); } t av = t acc / m;

7. Prof. Amr Goneid, AUC7 Counting Example 1 int count = 0; float s = 0.0; float A[n][n]; for (i=1; i<=n; i++) for (j = 1, j<=n; j++) { s = s + A[i][j]; count++;} The above code finds the no. of floating point additions T(n) = count as a function of n.

8. Prof. Amr Goneid, AUC8 Counting Example 2: Insertion Sort Function void insertion (int a[ ], int n) { int i, v, j ; for (i =1; i < n; i++) { v = a[i]; j = i; while(j > 0 && a[j-1] > v) { a[j] = a[j-1]; j--; count++;}; a[j] = v; }

9. Prof. Amr Goneid, AUC9 Insertion Sort (Performance) T1 ordered, T2 Random, T3 Inversely ordered n 5102050100200 T1(n)000000 T2(n)318785862609 10,052 T3(n)104519012254950 19,900 n(n-1)/2 104519012254950 19,900

10. Prof. Amr Goneid, AUC10 T(n) vs n for insertsort

11. Prof. Amr Goneid, AUC11 3. Performance Analysis (Modeling) Use a high-level Description (e.g Pseudocode) instead of implementation Identify problem size (n) Identify elementary processes. Assign a number of operations to each. Form a mathematical model of the sum of operations in these processes. Solve to find T(n) independent of HW

12. Prof. Amr Goneid, AUC12 Performance Analysis (Modeling) Example: Find the sum of all elements in a 2-D array of size (nxn) sum ← 0.0; for (i = 0 to n-1) for (j = 0 to n-1) sum ← sum + a ij The no. of floating point additions T(n) as a function of n is: ∑i∑i ∑j∑j 1

13. Prof. Amr Goneid, AUC13 4. Evaluating No. of Operations T(n) Usually we specify a certain type of operations, e.g., additions, array comparisons, etc. Simple Statements: T(n) = no. of specified operations. e.g. for z = 2*x + y T(n) = 2 arithmetic operations Code blocks: T(n) = Sum of sub-block T(n)’s

14. Prof. Amr Goneid, AUC14 No. of Operations T(n) Selection Statements: c = condition, s = statement block if (c) s; T(n) = T c (n) for c false (best case) = T c (n) + T s (n) for c true (worst case) If (c) s1; else s2; Worst case T(n) = T c (n) + max(T s1 (n),T s2 (n))

15. Example: T(n) = no. of comparisons Given that fun1(n) does n+2 comparisons, module1 does n 2 and module2 does n 4. Find worst case T(n) for the segment: if ( c == fun1(n)) call module1; else call module2; Prof. Amr Goneid, AUC15 No. of Operations T(n) ProcessT(n) ==1 fun1(n)n+2 module1n2n2 module2n4n4 Worst case total ?

16. Repetition Statements: For Loop Example: T(n) = No. of multiplications Fun(i,n) does 2i+n multiplications m = n*n for (i =1; i <= m; i++) Fun(i,n); Prof. Amr Goneid, AUC16 No. of Operations T(n) ProcessT(n) Compute m 1 Fun(i,n)2i + n TotalT(n) = ?

17. Prof. Amr Goneid, AUC17 No. of Operations T(n) Repetition Statements: while Loop Example: In the following code: Fun(i,n) does 3i + log n comparisons, find T(n) = no. of comparisons i = 0; while(i <= n) {Fun(i,n); i++;}

18. Prof. Amr Goneid, AUC18 Running Time The number of operations can be used to predict the running time of an algorithm for a given problem size (n) Example: The number of operations of a sorting algorithm is directly proportional to (n log n). Direct time measurement gives 1 ms to sort 1000 items. Find how long it will take to sort 1,000,000 items. Solution:

19. Prof. Amr Goneid, AUC19 5. Examples of Algorithm Analysis (a) Uniqueness Test Check whether all the elements in a given array are distinct Input: An array A[0…n-1] Output: Return “true” if all the elements in A are distinct and “false” otherwise

20. Prof. Amr Goneid, AUC20 Uniqueness Test

21. Prof. Amr Goneid, AUC21 Exercise Prove the formula: either by mathematical induction or by following the insight of a 10year-old school boy named Carl Friedrich Gauss (1777–1855) who grew up to become one of the greatest mathematicians of all times.

22. Prof. Amr Goneid, AUC22 (b) More Nested Loops Find the number of double arithmetic operations done by the following piece of code assuming that n = 2 m : for( int t = n; t > 1; t /= 2 ) { for( int u = 1; u < n; u *= 2 ) { for( int v = 0; v < n; v += 2 ) {... // constant number of double arithmetic //operations }

23. Prof. Amr Goneid, AUC23 More Nested Loops The loop variables can be transformed to: Outermost loop: i = log t = m … 1, Middle loop: j = log u = 0 … m-1 innermost loop: k = v/2 = 0.. n/2-1

24. Prof. Amr Goneid, AUC24 (c) Cosine Function Evaluation The cosine function can be evaluated using a truncated infinite series: The corresponding algorithm is: float cosine (float x, int n) { float y = -x * x; float s = 1.0; for (int i = 1; i <= n; i++) s = s + pow (y,i) / fact (2*i); return s; }

25. Prof. Amr Goneid, AUC25 Cosine Function Evaluation The number of arithmetic operations is evaluated as follows: fact (2*i ) uses one mult. + 2i mult. = 2i+1 pow(y,i) uses (i-1) mult. 1 division and 1 addition inside loop 1 mult. outside loop (-x * x) Show that: i.e., Quadratic algorithm.

26. Prof. Amr Goneid, AUC26 A Faster Cosine Algorithm We express the series in the form where S(x,i) = (-x 2 ) i / (2i)!, and S(x,0) = 1 Expressing S(x,i+1) in terms of S(x,i), then where y = (-x 2 ) and m = 2iwith S(x,0) = 1

27. Prof. Amr Goneid, AUC27 A Faster Cosine Algorithm Show that in this case: This is a linear algorithm that is much faster than the previous quadratic algorithm

28. Prof. Amr Goneid, AUC28 (d) Maximum Subsequence Sum Problem Statement Given a sequence of integers (possibly negative), a 1,a 2,...,a n, find the maximum value of (This is zero if all integers are negative). Example: -2, 11, -4, 13, -5, -2, S max = a 2 +a 3 +a 4 = 20

29. Prof. Amr Goneid, AUC29 Maximum Subsequence Sum Algorithm 1

30. Prof. Amr Goneid, AUC30 Maximum Subsequence Sum Algorithm 1 (Analysis)

31. Prof. Amr Goneid, AUC31 Maximum Subsequence Sum Algorithm 2

32. Prof. Amr Goneid, AUC32 Maximum Subsequence Sum Algorithm 2 (Analysis)

33. Prof. Amr Goneid, AUC33 Maximum Subsequence Sum Algorithm 3

34. Prof. Amr Goneid, AUC34 Maximum Subsequence Sum Algorithm 3 (Analysis)

35. Prof. Amr Goneid, AUC35 (e) Histogram Processing A bad contrast image (F) can be enhanced to an image G by transforming every pixel intensity f ij to another intensity g ij using a method called Histogram Equalization

36. Prof. Amr Goneid, AUC36 Histogram Processing In practice, gray levels are discrete (k = 0.. L-1) A histogram is an array whose element n k = no. of pixels with gray level k (k=0 is black, k=L-1 is white). Algorithm to build histogram for an image with N rows and M columns (N tot = NM pixels):

37. Prof. Amr Goneid, AUC37 Histogram Equalization Algorithm (1) The histogram is used in the following algorithm:

38. Prof. Amr Goneid, AUC38 Histogram Equalization Algorithm (2) A better algorithm first builds a Cumulative array:

39. Prof. Amr Goneid, AUC39 Histogram Equalization Summary of Total Complexity Algorithm (1): Algorithm (2):

40. Prof. Amr Goneid, AUC40 Histogram Equalization Numerical Example L = 256 gray levels Algorithm (1): Algorithm (2):