Algorithms & Complexity Dr. Ranette Halverson CMPS 2433 – Discrete Systems & Analysis Chapter 1: Introduction, 1.4

Solving a Problem Existence of a Solution How many Solutions Optimal Solution Sample problems on page 1 of book.

Algorithm Step-by-Step instructions to accomplish a task or solve a problem Computer Program Assembly instructions Driving Directions Recipe Format & Detail level depends on user

Efficiency of Algorithms Different algorithms for a single task may have different efficiency Driving Directions – short route vs. long route Searching Algorithms Sorting Algorithms Must be able to evaluate efficiency (speed) of computer algorithms

Comparing Algorithms Want to choose the “best” algorithm for tasks Generally, best = fastest But there are other considerations Hardware (e.g. LaTex) Size of data set Data Structure Need a standard measurement

Complexity = Speed = Efficiency NOT Really – but sort of *Complexity is the number of basic operations required by an algorithm Will 2 algorithms with same complexity take the same actual amount of time to run?? Why or Why Not? * Means memorize

Complexity Examples Read x, y, z x = y + z Print x How many operations? Read x, y, z x = y / z Print x Are these algorithms the same complexity? The same speed?

Complexity Examples Do 10 times Read x, y, z x = y + z Print x

Complexity Examples Do n times Read x, y, z x = y + z Print x Do 10 times Read x, y, z x = y + z Print x

Big Oh Notation O(f(n)) – “Big Oh of f of n” n represents size of data set Examples O(1) or O(c) O(n) O(n2) Big Oh is an upper bound.

Constant Complexity An algorithm with the same number of operations regardless of the data set is said to have CONSTANT COMPLEXITY O(1) or 0(c) See previous algorithms Most significant algorithms are NOT constant complexity

Complexity Examples Do n times Read x, y, z x = y + z Print x Complexity? Depends on value of n! NOT Constant 6 * n basic operations O(6n)  O(n) Do 10 times Read x, y, z x = y + z Print x Complexity? O(c) or O(1) - constant

Algorithm Complexity Number of operations in terms of the data set size Do 10 times Read x, y, z x = y + z Print x Constant Time Complexity – O(1) or O(c)

Algorithm Complexity Do n times Read x, y, z x = y + z Print x Time Complexity is dependent upon n 6n Operations - O(6n)  O(n)

*Big Oh (yes, memorize) O(f(n)) – Big Oh of f of n A function g(n) = O(f(n)) if there exist 2 constants K and n0 such that |g(n)| <= K|f(n)| for all n >=n0 Big Oh is an upper bound.

Evaluating xn Operations??? P = x k = 1 While k < n P = P * x k = k + 1 Print P Operations???

Evaluating xn Total =3 + n + 2(n-1) + 2(n-1) 3 + n + 2n -2 + 2n – 2 P = x k = 1 While k < n P = P * x k = k + 1 Print P Operations 1 n 2(n-1) Total =3 + n + 2(n-1) + 2(n-1) 3 + n + 2n -2 + 2n – 2 5n – 1 = O(5n -1)  O(n)

Trace: xn  54  P = x k = 1 While k < n P = P * x k = k + 1 Print P (n=4, x=5) P = 5, k = 1 P = 5*5 (25); k= 2 P = 25*5 (125); k = 3 P = 125*5 (625); k=4 Print 625

Complexity & Rate of Growth Complexity measures growth rate of algorithm time in terms of data size O(1) Constant O(n) Linear O(n2) Polynomial (any constant power) O(5n) Exponential (any constant base) What do these functions look like when graphed? How big are real world data sets? (name some)

Polynomial Evaluation P(x) = anxn + an-1xn-1 + … +a1x + a0 Given values for x, a0, a1,…an How many ops for anxn ? Operations to calculate each term in poly? n + n-1 + n-2 +…+ 1 + 0 = (n*(n+1))/2 Additions to combine the n+1 terms = n Total Ops = (n2+n)/2 + n  O(n2) What did we omit from analysis???

Polynomial Evaluation (p.26) P(x) = anxn + an-1xn-1 + … +a1x + a0 Given values for x, a0, a1,…an S = a0 , k = 1 while k <=1 S = S + ak xk k = k+ 1 Print S Can you spot any inefficiencies?

Polynomial Evaluation Algorithm (p Polynomial Evaluation Algorithm (p.26) P(x) = anxn + an-1xn-1 + … +a1x + a0 Given values for n, x, a0, a1,…an S = a0 , k = 1 while k <=n S = S + ak xk k = k+ 1 Print S *Slide 17: xk = 5k + 1 2 n + 1 #n(2+5(k+1)+1) 2n 1 #Total = 2n +5kn + 5n + n 5kn + 8n

Complexity of Poly. Evaluation Total = (by line on previous slide) =2 + (n + 1) + (5kn + 8n) + 2n + 1 = 5kn + 11n + 4 Worst case for k?? = 5n2 + 11 n + 4 O(n2) Note: this is for inserting code for xk Function call adds additional overhead.

Analysis of Polynomial Compare Slides 20 and 22/23 Same Big Oh  O(n2) “Different details” but same results

Horner’s Rule (Polynomial Evaluation) P(x) = anxn + an-1xn-1 + … +a1x + a0 S = an, k = 1 While k <= n S = Sx + an-k k = k = 1 Print s 2 n + 1 n * 3 n *2 1 Total = 2 + n+1 + 3n + n2 + 1 = 6n + 4  O(n)

Summary & Homework See table on page 31 In general, an algorithm is considered “good” if its complexity is no more than some polynomial in n For small n, some non-polynomial algorithms may be “acceptable” Homework: Page 33+ 1 – 10, 23 – 26 – will discuss in class 27 – 30 (detail for each step as on slides, state Total & Big Oh) – Turn in 27-30 for grading