1. Complexity Analysis (Part I ) Introduction to Algorithms and Complexity Analysis. Motivations for Complexity Analysis. Big-O Notation: An Introduction. Simple Complexity Analysis Examples. Average, Best, and Worst Cases.

2. Introduction to Algorithm & Complexity Analysis Let’s consider the execution time of some of the sorting algorithms.

3. Introduction (Contd.) From a theoretical point of view, the execution time of the tested algorithms is insufficient. So we need to define sound complexity metrics to give us a complete view about the algorithm performance. Computer scientists have defined for such purposes several efficiency indices such as: –Machine efficiency. –User efficiency. –Maintenance efficiency. –Algorithm complexity. –Coding efficiency.

4. Introduction (contd.) We will focus on the algorithm complexity measure. The main concern is to highlight the relation between the size of the processed input data and the algorithm complexity. The curves shown below indicate the number of operations required to implement the algorithm given the size of the data (n).

5. Introduction (contd.) Let's inspect the algorithms with 2 n and 10 n complexities, respectively.

6. Big-O Notation: an Introduction With the above concept of algorithm complexity, we can cast algorithms into different complexity classes. Some common classes are shown below: Big-O NotationComplexity Class O(1)Constant O(log n)Logarithmic O(n)Linear O(n^2)Quadratic O(2^n)Exponential

7. Big-O Notation (contd.) Question: Why do we need this notation? Answer: Let’s consider the following code fragment to perform the sum of the integer elements in an array A. The array A has n elements. 1 public int sum(int[] A) { 2 int sm = 0; 3 for(int i=0; i < A.length; i++) 4 sm += A[i]; 5 return(sm); 6 } The number of operations performed in this method is equal to: 1+1+n+1+n(2+2)+1=5n+4

8. Big-O Notation (contd.) Let’s consider another code fragment to perform the sum of the integer elements in a two-dimensional array B. 1 public int sum_all(int[][] B) { 2 int sm = 0; 3 for(int i=0; i < B.length; i++) 4 for(int j=0; j < B[0].length; j++) 5 sm += B[i][j]; 6 return(sm); 7 } The number of operations performed in this method is equal to: 1+1+n+1+n(1+n+1+n(2+2)+2)+1=5n^2+5n+4

9. Big-O Notation (contd.) Now, we will consider a case of exponential class complexity. We will investigate matrix multiplication operation. Number of columns in matrix A EQUAL to the number of rows in matrix B. B 3100 3102 3-216 4321 8075 2301 5423 x A = 4514823 19827 5691747

10. Big-O Notation (contd.) 1 public int[][] matrixMult(int[][] A, int[][] B) \{ 2 int r_a = A.length, c_a = A[0].length; 3 int r_b = B.length, c_b = B[0].length; 4 if(c_a != r_b) \{ 5 System.out.pritln("ERROR! Matrix dimensions mismatch."); 6 System.exit(-1); 7 } 8 else \{ 9 int[][] C = new int[r_a][c_b]; 9 for(int i = 0; i < r_a; i++) \{ 10 for(int j = 0; j < c_b; j++) \{ 11 C[i][j] = 0; 12 for(int k = 0; k < r_b; k++) 13 C[i][j] += A[i][k] * B[k][j]; 14 } 15 } 16 return(C); 17 } 18 } // End of matrixMult()

11. Average, Worst, and Best Cases An algorithm may run faster on certain data sets than others. Finding the average case can be very difficult, so typically algorithms are measured in the worst case time complexity. Also, in certain application domains (e.g., air traffic control, medical, etc.) knowing the worst case time complexity is of crucial importance.

12. Average, Worst, and Best Cases (contd.) The analysis of the linear search algorithm illustrates the difficulties in defining the algorithm complexity. 2 0 3 12 -4 9 CostCounterKey Value 1 Op12 2 Op2 7 Op79 7 Op & Search {FAILED}711

13. Average, Worst, and Best Cases (contd.) From the previous example, the linear search algorithm can have different complexity cases. The best case requires 1 comparison operation. The worst case requires n comparison operations (with possible failure!). The average case is the average of all the possible cases over the number of possibilities, i.e., O(n) = O(n)= ∑ i = 1+2+…+n = 1+2+…+n n = ∑ i i -1 n n = n+1 n i=1 n(n+1) 2

14. Drill Questions Find the number of operations performed by the following code fragment: sum = 0; for(i = 1; i <= n; i++) sum += A[i][i]; Find the number of operations performed by the following code fragment: for(cnt1 = 0, i = 1; i <= n; i++) for(j = 1; j <= n; j++) cnt1++; Find the number of operations performed by the following code fragment: for(cnt2 = 0, i = 1; i <= n; i *= 2) for(j = 1; j <= n; j++) cnt2++;