1. Lec 5 Feb 10 Goals: analysis of algorithms (continued) O notation summation formulas maximum subsequence sum problem (Chapter 2) three algorithms image processing examples (array processing)

2. How to Measure Algorithm Performance? What metric should be used to judge algorithms? –Length of the program (lines of code) since the personnel cost is related to this. –Ease of programming (bugs, maintenance) –Memory required  Running time Running time is the dominant standard. –Quantifiable and easy to compare –Often the critical bottleneck –Particularly important when real-time response is expected

3. Average, Best, and Worst-Case On which input instances should the algorithm’s performance be judged? Average case: –Real world distributions difficult to predict Best case: –Unrealistic since this may occur very rarely Worst case: –Gives an absolute guarantee –We will use the worst-case measure.

4. Examples Vector addition Z = A+B for (int i=0; i<n; i++) Z[i] = A[i] + B[i]; T(n) = c n Vector multiplication Z=A*B z = 0; for (int i=0; i<n; i++) z = z + A[i]*B[i]; T(n) = c’ + c 1 n

5. Simplifying the Bound T(n) = c k n k + c k-1 n k-1 + c k-2 n k-2 + … + c 1 n + c o –too complicated –too many terms –Difficult to compare two expressions, each with 10 or 20 terms Do we really need all the terms? For approximation, we can drop all but the biggest term. When n is large, the first term is dominant.

6. Simplifications Keep just one term! – the fastest growing term (dominates the runtime) No constant coefficients are kept –Constant coefficients affected by machines, languages, etc. Order of magnitude (as n gets large) is captured well by the leading term. –Example. T(n) = 10 n 3 + n 2 + 40n + 800 If n = 1,000, then T(n) = 10,001,040,800 error is 0.01% if we drop all but the n 3 term

7. O (order) notation Definition: Let f(n) and g(n) be two functions defined on the set of integers. If there is a c > 0 such that f(n) <= c g(n) for all large enough n. Then, we say f(n) = O(g(n)). Example: n 2 + 2n – 15 is O(n 2 ) Rule: When the expression involves a sum, keep only the term with the highest power, drop the rest. You can also drop constant coefficients from this term. (3n 2 + 2 n + 1) (4 n – 5) is O(n 3 )

8. Problem size vs. time taken Assume the computer does 1 billion ops per sec.

9. 2n2n n2n2 n log n n log n n n log n n2n2 n3n3 n3n3 2n2n

10. Exercise: What is the number of operations (additions) by the following program? for (j = 100; j < 200; ++j) if (j % 2 == 0) sum+= A[j]; Answer: The number of iterations of the loop = 100, But the addition is performed in every other iteration so the number of additions performed = 50.

11. Max Subsequence Problem w Given a sequence of integers A1, A2, …, An, find the maximum possible value of a subsequence Ai, …, Aj. w Numbers can be negative. w You want a contiguous chunk with largest sum. w Example: -2, 11, -4, 13, -5, -2 The answer is 20 (subsequence A2 through A4). w We will discuss three different algorithms, with time complexities O(n 3 ), O(n 2 ), and O(n). w With n = 10 6, algorithm 1 may take > 10 years; algorithm 3 will take a fraction of a second!

12. Summation formulas 1 + 2 + … + n = n(n+1)/2 Arithmetic series sum: S = (average of first and last terms) x num of terms 1 k + 2 k + … + n k ~ n k+1 /(k+1) More generally:

13. int maxSum = 0; for( int i = 0; i < a.size( ); i++ ) for( int j = i; j < a.size( ); j++ ) { int thisSum = 0; for( int k = i; k <= j; k++ ) thisSum += a[ k ]; if( thisSum > maxSum ) maxSum = thisSum; } return maxSum; Algorithm 1 for Max Subsequence Sum w Given A 1,…,A n, find the maximum value of A i +A i+1 +···+A j 0 if the max value is negative n Time complexity: O(n 3 )

14. Algorithm 2 w Idea: Given sum from i to j-1, we can compute the sum from i to j in constant time. w This eliminates one nested loop, and reduces the running time to O(n 2 ). into maxSum = 0; for( int i = 0; i < a.size( ); i++ ) int thisSum = 0; for( int j = i; j < a.size( ); j++ ) { thisSum += a[ j ]; if( thisSum > maxSum ) maxSum = thisSum; } return maxSum;

15. Algorithm 3 int maxSum = 0, thisSum = 0; for( int j = 0; j < a.size( ); j++ ) { thisSum += a[ j ]; if ( thisSum > maxSum ) maxSum = thisSum; else if ( thisSum < 0 ) thisSum = 0; } return maxSum n The algorithm resets whenever prefix is < 0. Otherwise, it forms new sums and updates maxSum in one pass.

16. 16 Array representation of images Image: A pixel is a square region on a display device that can be illuminated with one of the color combinations. A wide range of colors can be specified using 3 bytes – one for each color R, G and B. R = 255, G = 255, B = 255 represents White. R = 0, G = 0, B = 0 represents Black. Bitmap format: uncompressed, each pixel information stored. Header + each pixel description

17. 17 An image filtering problem A small percentage (~ 5%) of pixels have become corrupted – randomly changed to arbitrary value. (salt and pepper noise). How should we remove this noise? Original image image with noise after filtering The image has become blurred, but this can be corrected. (We will not discuss this part.)

18. 18 Mean filtering For each pixel, consider its eight neighbors. Replace its color value by the average of the 9 color values, itself and the 8 neighbors. Example: Suppose all the neighbors were blue pixels (0, 0, 150), but the center was red (noise), say (200, 0, 0). Then, the average value = (22, 0, 133). This color is much closer to blue, so the noise has been removed.

19. 19 Algorithm for mean filtering I = input image; O = output image; w = width of the image; h = height of the image; for j from 1 to w-2 do for k from 1 to h-2 do O(j,k)->Blue = (I(j,k)->Blue + I(j-1,k)->Blue + I(j+1,k)->Blue + I(j,k-1)->Blue + I(j-1,k-1)->Blue+ I(j+1,k-1)->Blue +I(j,k+1)->Blue + I(j-1,k+1)->Blue+ I(j+k+1)->Blue)/9;.... // similarly for other colors end do; On a 1024 x 1024 pixel image, how many operations does this perform? More generally, on an n x n image? Answer: O(n 2 ) which is linear since the size of the input is O(n 2 ). More precisely, 30 n 2 operations.

20. 20 Median filter A problem with mean filter is that the image loses its sharpness. In median filter, we examine the neighborhood pixel values and sort them, replace the current pixel value by the median value. Example: 374139 40 234 38 423844 Sorted sequence: 37, 38, 38, 39, 40, 41, 42, 44, 234 A good choice to find the median is insertion sorting. Better than selection sorting. Why?