1 Analysis of Algorithms CS 105 Introduction to Data Structures and Algorithms

2 What is a good algorithm? It must be correct It must be efficient Implementations of an algorithm must run as fast as possible How is this measured? Running time

3 Running time of a program Amount of time required for a program to execute for a given input If measured experimentally, Dependent on hardware, operating system and software Answer will be in milliseconds, seconds, minutes, hours, …

4 Running time of an algorithm Answer will not be in seconds or minutes Instead, we count the number of operations carried out Result will be a formula in terms of some input size, n Takes into account all possible inputs Example statement on running time: Running time of algorithm A is T(n) = 2n + 3

5 Describing algorithms First, we need to agree on how to describe or specify an algorithm Note: algorithms are intended for humans (programs are intended for computers) Descriptions should be high-level explanations that combine natural language and familiar programming structures: Pseudo-code

6 Pseudo-code example Algorithm arrayMax(A,n): Input: An array A storing n integers. Output: The maximum element in A. currentMax  A[0] for i  1 to n - 1 do if currentMax < A[i] then currentMax  A[i] return currentMax

7 Pseudo-Code conventions General Algorithm Structure Statements Expressions Control Structures

8 Algorithm Structure Algorithm heading Algorithm name(param1, param2,...): Input : input elements Output : output elements statements…

9 Statements Assignment: use  instead of = Method or function call: object.method(arguments) method(arguments) Return statement: return expression Control structures

10 Control Structures decision structures while loops repeat loops for loop n if... then... [else...] n while... do n repeat... until... n for... do

11 Expressions Standard math symbols + - * / ( ) Relational operators = > = != Boolean operators and or not Assignment operator  Array indexing A[i]

12 General rules on pseudo-code Should communicate high-level ideas and not implementation details Syntax is not as tight (e.g., indentation and line-breaks take the place of ; and {} in Java) Clear and informative

13 Back to running time Once an algorithm has been described in pseudo-code, we can now count the primitive operations carried out by the algorithm Primitive operations: assignment, calls, arithmetic operations, comparisons, array accesses, return statements, …

14 Back to arrayMax example Suppose the input array is A = {42, 6, 88, 53}, (n = 4) How many primitive operations are carried out? Some notes The for statement implies assignments, comparisons, subtractions, and increments the statement currentMax  A[i] will sometimes not be carried out

15 What to count currentMax  A[0]: assignment, array access for i  1 to n - 1 do: assignment,comparison,subtraction,increment (2 ops) if currentMax < A[i] then: comparison, access currentMax  A[i]: assignment, access return currentMax: return

16 Counting operations currentMax  A[0]2 for i  1 to n - 1 do15 if currentMax < A[i] then6 currentMax  A[i]2 return currentMax1 OPERATIONS CARRIED OUT26

17 Handling all possible inputs In the example just carried out, running time = 26 for the given input A more useful answer is to provide a formula that applies in general Formula is in terms of n, the input or array size Since there may be statements that do not always execute, depending on input values, there are two approaches Assume worst-case Provide a range

18 Measuring running time currentMax  A[0]2 for i  1 to n - 1 do1+2n+2(n-1) if currentMax < A[i] then2(n-1) currentMax  A[i]0.. 2(n-1) return currentMax1 RUNNING TIME (worst-case)8n - 2 increment comparison & subtraction

19 Nested loops // prints all pairs from the set {1,2,…n} for i  1 to n do for j  1 to n do print i, j How many times does the print statement execute?

20 Nested loops // prints all pairs from the set {1,2,…n} for i  1 to n do for j  1 to n do print i, jn 2 times How about the operations implied in the for statement headers?

21 for statement revisited for i  1 to n do … 1 assignmenti  1 n+1 comparisonsi <= n for each value of i in {1, 2, …, n+1} n increments = n assignments + n adds => 3n+2 operations

22 Nested loops running time for i  1 to n do3n+2 for j  1 to n don (3n+2) print i, jn 2 TOTAL:4n 2 + 5n + 2

23 Nested loops example 2 // prints all pairs from the set {1,2,…n} // excluding redundancies (i.e., 1,1 is not a // real pair, 1,2 and 2,1 are the same pair) for i  1 to n-1 do for j  i+1 to n do print i, j How many times does the print statement execute?

24 Nested loops example 2 for i  1 to n-1 do for j  i+1 to n do print i, j when i = 1,n-1 prints when i = 2,n-2 prints … when i = n-2,2 prints when i = n-1,1 print

25 Nested loops example 2 Number of executions of the print statement: … + n-1 = n(n-1)/2 = ½n 2 – ½n

26 Nested loops example 2 for i  1 to n-1 do for j  i+1 to n do print i, j Take-home exercise: count the operations implied by the for statement headers

27 Running time function considerations In determining the running time of an algorithm, an exact formula is possible only when the operations to be counted are explicitly specified Different answers will result if we choose not to count particular operations Problem: variations in the answers appear arbitrary

28 Classifying running time functions We want to classify running times under particular function categories Goal: some assessment of the running time of the algorithm that eliminates the arbitrariness of counting operations Example: arrayMax runs in time proportional to n

29 Some function categories The constant function:f(n) = c The linear function:f(n) = n The quadratic function:f(n) = n 2 The cubic function:f(n) = n 3 The exponential function:f(n) = b n The logarithm function:f(n) = log n The n log n function:f(n) = n log n

30 Big-Oh notation Consider the following (running time) functions f 1 (n) = 2n + 3 f 2 (n) = 4n 2 + 5n + 2 f 3 (n) = 5n We place f 1 and f 3 under the same category (both are proportional to n) and f 2 under a separate category (n 2 ) We say: 2n+3 is O(n), and 5n is O(n), while 4n 2 + 5n + 2 is O(n 2 ) 2n+3 and 5n are linear functions, while 4n 2 + 5n + 2 is a quadratic function

31 Significance of Big-Oh Example for i  0 to n-1 do A[i]  0 Running time is 2n, if we count assignments & array accesses 4n+2, if we include implicit loop assignments/adds 5n+3, if we include implicit loop comparisons Regardless, running time is O(n)

32 Prefix Averages Problem Given an array X storing n numbers, compute prefix averages or running averages of the sequence of numbers A[i] = average of X[0],X[1],…,X[i] Example Input:X = {1,3,8,2} Output:A = {1,2,4,3.5} Algorithm?

33 Algorithm 1 Algorithm prefixAverages1(A,n): Input: An array X of n numbers. Output: An array A containing prefix averages for i  0 to n - 1 do sum  0 for j  0 to i do sum  sum + X[j] A[i]  sum/(i+1) return array A

34 Running time of Algorithm 1 Exercise: count primitive operations carried out by Algorithm 1 Regardless of what you decide to count, the answer will be of the form: a n 2 + b n + c Observe that the answer is O(n 2 )

35 Algorithm 2 Algorithm prefixAverages2(A,n): Input: An array X of n numbers. Output: An array A containing prefix averages sum  0 for i  0 to n - 1 do sum  sum + X[j] A[i]  sum/(i+1) return array A

36 Running time of Algorithm 2 Running time of Algorithm 2 will be of the form: a n + b Observe that the answer is O(n) Assessment: Algorithm 2 is more efficient than Algorithm 1, particularly for large input (observe the growth rate of n and n 2 )

37 Summary Algorithms are expressed in pseudo-code The running time of an algorithm is the number of operations it carries out in terms of input size Answer depends on which operation(s) we decide to count Big-Oh notation allows us to categorize running times according to function categories Next: formal definition of Big-Oh notation