1. CS 3610/5610N data structures Lecture: complexity analysis Data Structures Using C++ 2E1

2. Time matters Data Structures Using C++ 2E2

3. To understand a data structure Data Structures Using C++ 2E3

4. 4 Algorithm Analysis Example –50 packages delivered to 50 different houses –50 houses one mile apart, in the same area FIGURE 1-1 Gift shop and each dot representing a house

5. Algorithm Analysis (cont’d.) Example (cont’d.) –Driver picks up all 50 packages –Drives one mile to first house, delivers first package –Drives another mile, delivers second package –Drives another mile, delivers third package, and so on –Distance driven to deliver packages 1+1+1+… +1 = 50 miles –Total distance traveled: 50 + 50 = 100 miles Data Structures Using C++ 2E5 FIGURE 1-2 Package delivering scheme

6. Algorithm Analysis (cont’d.) Example (cont’d.) –Similar route to deliver another set of 50 packages Driver picks up first package, drives one mile to the first house, delivers package, returns to the shop Driver picks up second package, drives two miles, delivers second package, returns to the shop –Total distance traveled 2 * (1+2+3+…+50) = 2550 miles Data Structures Using C++ 2E6 FIGURE 1-3 Another package delivery scheme

7. Data Structures Using C++ 2E7 Algorithm Analysis (cont’d.) Example (cont’d.) –n packages to deliver to n houses, each one mile apart –First scheme: total distance traveled 1+1+1+… +n = 2n miles Function of n –Second scheme: total distance traveled 2 * (1+2+3+…+n) = 2*(n(n+1) / 2) = n 2 +n Function of n 2

8. Data Structures Using C++ 2E8 Algorithm Analysis (cont’d.) Analyzing an algorithm –Count number of operations performed Not affected by computer speed TABLE 1-1 Various values of n, 2n, n 2, and n 2 + n

9. Data Structures Using C++ 2E9 Algorithm Analysis (cont’d.) Example 1-1 –Illustrates fixed number of executed operations

10. Analysis of Example 1-1 –Line 1 has one operation, <<; –Line 2 has two operations; –Line 3 has one operation, >=; –Line 4 has one operation, =; –Line 6 has one operation; and –Line 7 has three operations. –Either Line 4 or Line 6 executes. Therefore, the total number of operations executed in the preceding code is 1 + 2 + 1 + 1 + 3 = 8. In this algorithm, the number of operations executed is fixed. Data Structures Using C++ 2E10

11. Data Structures Using C++ 2E11 Algorithm Analysis (cont’d.) Example 1-2 –Illustrates dominant operations

12. Analysis of Example 1-2 –Before the while loop: 5 operations (lines 1 – 4) –After the while loop: 9 or 8 operations (depends on whether line 11 or line 13 executes) –The while loop: 1 operation (line 5) 4 operations (lines 6-8) Note: the loop head (line 5) will be executed one more time than the body to terminate the loop. Therefore, if the loop runs k times, the total number of operations is 5 + (k*5 + 1) + 9 or 5 + (k*5 + 1) + 8. 12

13. Data Structures Using C++ 2E13 Algorithm Analysis (cont’d.) Another example: Search algorithm –n: represents list size –f(n): count function Number of comparisons in search algorithm –c: units of computer time to execute one operation –cf(n): computer time to execute f(n) operations –Constant c depends computer speed (varies) –f(n): number of basic operations (constant) –Determine algorithm efficiency Knowing how function f(n) grows as problem size grows

14. Data Structures Using C++ 2E14 Algorithm Analysis (cont’d.) TABLE 1-2 Growth rates of various functions

15. Data Structures Using C++ 2E15 Algorithm Analysis (cont’d.) Figure 1-4 Growth rate of functions in Table 1-3 TABLE 1-3 Time for f(n) instructions on a computer that executes 1 billion instructions per second Picture from the textbook; the curve 2^n is wrong

16. Data Structures Using C++ 2E16 The Big-O Notation Notation useful in describing algorithm behavior –Shows how a function f(n) grows as n increases without bound Asymptotic –Study of the function f as n becomes larger and larger without bound –Examples of functions g(n)=n 2 (no linear term) f(n)=n 2 + 4n + 20

17. Data Structures Using C++ 2E17 Algorithm Analysis: The Big-O Notation (cont’d.) As n becomes larger and larger –Term 4n + 20 in f(n) becomes insignificant –Term n 2 becomes dominant term TABLE 1-4 Growth rate of n 2 and n 2 + 4n + 20n

18. Data Structures Using C++ 2E18 Algorithm Analysis: The Big-O Notation (cont’d.) Algorithm analysis –If function complexity can be described by complexity of a quadratic function without the linear term We say the function is of O(n 2 ) or Big-O of n 2 Let f and g be real-valued functions –Assume f and g nonnegative For all real numbers n, f(n) >= 0 and g(n) >= 0 f(n) is Big-O of g(n): written f(n) = O(g(n)) –If there exists positive constants c and n 0 such that f(n) = n 0

19. Data Structures Using C++ 2E19 Algorithm Analysis: The Big-O Notation (cont’d.) TABLE 1-5 Some Big-O functions that appear in algorithm analysis

20. Three time complexities Data Structures Using C++ 2E20

21. Running time, cont’d Data Structures Using C++ 2E21

22. Running time of iterative functions The routine “Example 1-2” is an iterative example The number of operations can usually be calculated through the summation of the operations within individual steps/stages. Data Structures Using C++ 2E22

23. Recursive example: merge sort 23

24. Merge() Data Structures Using C++ 2E24

25. Time complexity of Merge Sort Data Structures Using C++ 2E25 A constant-time step A linear-time step

26. Time complexity 26

27. Big-O notation: revisit Data Structures Using C++ 2E27

28. Asymptotic notation Data Structures Using C++ 2E28

29. Big-O, Big-Omega, Big-Theta Data Structures Using C++ 2E29

30. Example Data Structures Using C++ 2E30

31. Example, cont’d Data Structures Using C++ 2E31

32. Logarithms Data Structures Using C++ 2E32

33. Some rules about Big-O notations Data Structures Using C++ 2E33

34. Some rules about Big-O notations, cont’d Data Structures Using C++ 2E34

35. Some rules about Big-O notations, cont’d Data Structures Using C++ 2E35

36. Important notation Data Structures Using C++ 2E36

37. Big Omega notation Data Structures Using C++ 2E37

38. Big Theta notation 38

39. Big-O, Big-Omega, Big-Theta Data Structures Using C++ 2E39

40. Amortized complexity (won’t show up in homework/exam) Sometimes, worst case performance is a poor indicator of the running time of a given algorithm because infrequent executions of the algorithm may take a long time (worst case), but, in aggregate, the algorithm actually performs very well. Since algorithm designers usually use basic algorithms over and over again in larger tasks, these infrequent worst case executions can be “smoothed out.” Data Structures Using C++ 2E40

41. Amortized complexity, Cont’d The technique used to describe this behavior is referred to as “amortized complexity” because it consider the running time of an algorithm over a collection of invocations. Amortized complexity is still worst case complexity —- however, it provides a way of describing the worst case complexity of a collection of operations. Data Structures Using C++ 2E41

42. Amortized complexity An example: your daily expenditure in college –Regular days: meals, transportations, laundry, etc $25 –Once a while: rental $600 –Once a semester: tuition due $5,000 Single-day worst case: $5,625 Amortized complexity: much better than the worst case. In algorithm/data structure designs, we also often consider the amortized complexity --- looking at certain solution from a longitudinal perspective. Data Structures Using C++ 2E42