CS 3610/5610N data structures Lecture: complexity analysis Data Structures Using C++ 2E1

Time matters Data Structures Using C++ 2E2

To understand a data structure Data Structures Using C++ 2E3

4 Algorithm Analysis Example –50 packages delivered to 50 different houses –50 houses one mile apart, in the same area FIGURE 1-1 Gift shop and each dot representing a house

Algorithm Analysis (cont’d.) Example (cont’d.) –Driver picks up all 50 packages –Drives one mile to first house, delivers first package –Drives another mile, delivers second package –Drives another mile, delivers third package, and so on –Distance driven to deliver packages … +1 = 50 miles –Total distance traveled: = 100 miles Data Structures Using C++ 2E5 FIGURE 1-2 Package delivering scheme

Algorithm Analysis (cont’d.) Example (cont’d.) –Similar route to deliver another set of 50 packages Driver picks up first package, drives one mile to the first house, delivers package, returns to the shop Driver picks up second package, drives two miles, delivers second package, returns to the shop –Total distance traveled 2 * (1+2+3+…+50) = 2550 miles Data Structures Using C++ 2E6 FIGURE 1-3 Another package delivery scheme

Data Structures Using C++ 2E7 Algorithm Analysis (cont’d.) Example (cont’d.) –n packages to deliver to n houses, each one mile apart –First scheme: total distance traveled … +n = 2n miles Function of n –Second scheme: total distance traveled 2 * (1+2+3+…+n) = 2*(n(n+1) / 2) = n 2 +n Function of n 2

Data Structures Using C++ 2E8 Algorithm Analysis (cont’d.) Analyzing an algorithm –Count number of operations performed Not affected by computer speed TABLE 1-1 Various values of n, 2n, n 2, and n 2 + n

Data Structures Using C++ 2E9 Algorithm Analysis (cont’d.) Example 1-1 –Illustrates fixed number of executed operations

Analysis of Example 1-1 –Line 1 has one operation, <<; –Line 2 has two operations; –Line 3 has one operation, >=; –Line 4 has one operation, =; –Line 6 has one operation; and –Line 7 has three operations. –Either Line 4 or Line 6 executes. Therefore, the total number of operations executed in the preceding code is = 8. In this algorithm, the number of operations executed is fixed. Data Structures Using C++ 2E10

Data Structures Using C++ 2E11 Algorithm Analysis (cont’d.) Example 1-2 –Illustrates dominant operations

Analysis of Example 1-2 –Before the while loop: 5 operations (lines 1 – 4) –After the while loop: 9 or 8 operations (depends on whether line 11 or line 13 executes) –The while loop: 1 operation (line 5) 4 operations (lines 6-8) Note: the loop head (line 5) will be executed one more time than the body to terminate the loop. Therefore, if the loop runs k times, the total number of operations is 5 + (k*5 + 1) + 9 or 5 + (k*5 + 1)

Data Structures Using C++ 2E13 Algorithm Analysis (cont’d.) Another example: Search algorithm –n: represents list size –f(n): count function Number of comparisons in search algorithm –c: units of computer time to execute one operation –cf(n): computer time to execute f(n) operations –Constant c depends computer speed (varies) –f(n): number of basic operations (constant) –Determine algorithm efficiency Knowing how function f(n) grows as problem size grows

Data Structures Using C++ 2E14 Algorithm Analysis (cont’d.) TABLE 1-2 Growth rates of various functions

Data Structures Using C++ 2E15 Algorithm Analysis (cont’d.) Figure 1-4 Growth rate of functions in Table 1-3 TABLE 1-3 Time for f(n) instructions on a computer that executes 1 billion instructions per second Picture from the textbook; the curve 2^n is wrong

Data Structures Using C++ 2E16 The Big-O Notation Notation useful in describing algorithm behavior –Shows how a function f(n) grows as n increases without bound Asymptotic –Study of the function f as n becomes larger and larger without bound –Examples of functions g(n)=n 2 (no linear term) f(n)=n 2 + 4n + 20

Data Structures Using C++ 2E17 Algorithm Analysis: The Big-O Notation (cont’d.) As n becomes larger and larger –Term 4n + 20 in f(n) becomes insignificant –Term n 2 becomes dominant term TABLE 1-4 Growth rate of n 2 and n 2 + 4n + 20n

Data Structures Using C++ 2E18 Algorithm Analysis: The Big-O Notation (cont’d.) Algorithm analysis –If function complexity can be described by complexity of a quadratic function without the linear term We say the function is of O(n 2 ) or Big-O of n 2 Let f and g be real-valued functions –Assume f and g nonnegative For all real numbers n, f(n) >= 0 and g(n) >= 0 f(n) is Big-O of g(n): written f(n) = O(g(n)) –If there exists positive constants c and n 0 such that f(n) = n 0

Data Structures Using C++ 2E19 Algorithm Analysis: The Big-O Notation (cont’d.) TABLE 1-5 Some Big-O functions that appear in algorithm analysis

Three time complexities Data Structures Using C++ 2E20

Running time, cont’d Data Structures Using C++ 2E21

Running time of iterative functions The routine “Example 1-2” is an iterative example The number of operations can usually be calculated through the summation of the operations within individual steps/stages. Data Structures Using C++ 2E22

Recursive example: merge sort 23

Merge() Data Structures Using C++ 2E24

Time complexity of Merge Sort Data Structures Using C++ 2E25 A constant-time step A linear-time step

Time complexity 26

Big-O notation: revisit Data Structures Using C++ 2E27

Asymptotic notation Data Structures Using C++ 2E28

Big-O, Big-Omega, Big-Theta Data Structures Using C++ 2E29

Example Data Structures Using C++ 2E30

Example, cont’d Data Structures Using C++ 2E31

Logarithms Data Structures Using C++ 2E32

Some rules about Big-O notations Data Structures Using C++ 2E33

Some rules about Big-O notations, cont’d Data Structures Using C++ 2E34

Some rules about Big-O notations, cont’d Data Structures Using C++ 2E35

Important notation Data Structures Using C++ 2E36

Big Omega notation Data Structures Using C++ 2E37

Big Theta notation 38

Big-O, Big-Omega, Big-Theta Data Structures Using C++ 2E39

Amortized complexity (won’t show up in homework/exam) Sometimes, worst case performance is a poor indicator of the running time of a given algorithm because infrequent executions of the algorithm may take a long time (worst case), but, in aggregate, the algorithm actually performs very well. Since algorithm designers usually use basic algorithms over and over again in larger tasks, these infrequent worst case executions can be “smoothed out.” Data Structures Using C++ 2E40

Amortized complexity, Cont’d The technique used to describe this behavior is referred to as “amortized complexity” because it consider the running time of an algorithm over a collection of invocations. Amortized complexity is still worst case complexity —- however, it provides a way of describing the worst case complexity of a collection of operations. Data Structures Using C++ 2E41

Amortized complexity An example: your daily expenditure in college –Regular days: meals, transportations, laundry, etc $25 –Once a while: rental $600 –Once a semester: tuition due $5,000 Single-day worst case: $5,625 Amortized complexity: much better than the worst case. In algorithm/data structure designs, we also often consider the amortized complexity --- looking at certain solution from a longitudinal perspective. Data Structures Using C++ 2E42