CISC220 Spring 2010 James Atlas Lecture 07: Big O Notation

Objectives for Today Big-O notation Reading - K+W Chap , 2.6

Big-O notation Describes the relationship between input size and execution time If we double the number of inputs, n, and the execution time is approximately doubled –Linear growth rate –Growth rate has an order of n –O(n)

Example 2.14/2.16 Analysis 2.14 –execution time proportional to x_length –O(n) 2.16 –execution time proportional to (x_length) 2 –O(n 2 )

Let’s count individual instructions for (int i = 0; i < n; i++) { for (int j = 0; j < n; j++) { Simple Statement } for (int k = 0; k < n; k++) { Simple Statement 1 Simple Statement 2 Simple Statement 3 Simple Statement 4 Simple Statement 5 } Simple Statement 1 Simple Statement 2... Simple Statement 25

Let’s count individual instructions for (int i = 0; i < n; i++) { for (int j = 0; j < n; j++) { Simple Statement } n 2 executions

Let’s count individual instructions for (int i = 0; i < n; i++) { for (int j = 0; j < n; j++) { Simple Statement } for (int k = 0; k < n; k++) { Simple Statement 1 Simple Statement 2 Simple Statement 3 Simple Statement 4 Simple Statement 5 } n executions, 5 statements per = 5n

Let’s count individual instructions for (int i = 0; i < n; i++) { for (int j = 0; j < n; j++) { Simple Statement } for (int k = 0; k < n; k++) { Simple Statement 1 Simple Statement 2 Simple Statement 3 Simple Statement 4 Simple Statement 5 } Simple Statement 1 Simple Statement 2... Simple Statement 25 1 execution, 25 statements = 25

Let’s count individual instructions for (int i = 0; i < n; i++) { for (int j = 0; j < n; j++) { Simple Statement } for (int k = 0; k < n; k++) { Simple Statement 1 Simple Statement 2 Simple Statement 3 Simple Statement 4 Simple Statement 5 } Simple Statement 1 Simple Statement 2... Simple Statement 25 T(n) = total execution time as a function of n T(n) = n 2 + 5n + 25

Formal Big-O Definition T(n) = n 2 + 5n + 25 T(n) = O(f(n)) means that there exists a function, f(n), that for sufficiently large n and some constant c: cf(n)  T(n)

n n + 25 vs. 3n 2

Common Big-O Runtimes Constant -- O(1) Logarithmic -- O(log n) Fractional -- O(sqrt n) Linear -- O(n) Log-Linear-- O(n log n) Quadratic -- O(n 2 ) Cubic -- O(n 3 ) Exponential -- O(2 n ) Factorial -- O(n!)

Various Runtimes

Powers of n (Order of the polynomial)

Multiplicative Constants

Multiplicative Constants (cont’)

Dominant Terms (1)

Dominant Terms (2)

Dominant Terms (3)

Dominant Terms Comparison

Dominant Terms Comparison (cont’)

Class Exercise Implement last on our Linked List

Group Exercise Implement last in O(1)

The big-O notation can be derived from T(n) using the following steps: 1) Set the coefficient of each term to 1 2) Keep the largest (least efficient) term in the function and discard the others.

Standard Measures of Efficiency To get a feel for how much of a difference an algorithm’s efficiency makes, check out the table on the next slide. The table assumes an instruction speed of 1 microsecond, 10 instructions per loop, and n=10,000

Standard Measures of Efficiency Efficiency Big-O IterationsEstimated Time Logarithmic O(log 2 n)14Microseconds Linear O(n)10, seconds Linear Logarithmic O(nlog 2 n)140,0002 seconds Quadratic O(n 2 )10,000 2 ~17 minutes Polynomial O(n k )10,000 k Hours Exponential O(c n )2 10,000 Intractable Factorial O(n!)10,000!Intractable

Standard Measures of Efficiency

Calculating Big-O 1.T(n) = 6n 4 − 2n T(n) = 9 log n + 5 (log n) 3 + 3n 2 + 2n 3 3.T(n) = 3n 2 + O(log n) + O(n) + O(n log n)

Calculating Big-O 4.Binary Search on an array: T(n) = T(n/2) + O(1) 5.T(1) = 2 T(n) = 2T(n - 1) + O(1) for n>1 6.T(1) = 1 T(n) = 2T(n/2) + O(1) for n>1