1. Algorithms & Complexity
Dr. Ranette Halverson
CMPS 2433 – Discrete Systems & Analysis
Chapter 1: Introduction, 1.4

2. Solving a Problem Existence of a Solution How many Solutions
Optimal Solution
Sample problems on page 1 of book.

3. Algorithm
Step-by-Step instructions to accomplish a task or solve a problem
Computer Program
Assembly instructions
Driving Directions
Recipe
Format & Detail level depends on user

4. Efficiency of Algorithms
Different algorithms for a single task may have different efficiency
Driving Directions – short route vs. long route
Searching Algorithms
Sorting Algorithms
Must be able to evaluate efficiency (speed) of computer algorithms

5. Comparing Algorithms Want to choose the “best” algorithm for tasks
Generally, best = fastest
But there are other considerations
Hardware (e.g. LaTex)
Size of data set
Data Structure
Need a standard measurement

6. Complexity = Speed = Efficiency
NOT Really – but sort of
*Complexity is the number of basic operations required by an algorithm
Will 2 algorithms with same complexity take the same actual amount of time to run??
Why or Why Not?
* Means memorize

7. Complexity Examples
Read x, y, z x = y + z Print x How many operations?
Read x, y, z x = y / z Print x Are these algorithms the same complexity? The same speed?

8. Complexity Examples Do 10 times Read x, y, z x = y + z Print x

9. Complexity Examples Do n times Read x, y, z x = y + z Print x
Do 10 times Read x, y, z x = y + z Print x

10. Big Oh Notation O(f(n)) – “Big Oh of f of n”
n represents size of data set
Examples
O(1) or O(c)
O(n)
O(n2)
Big Oh is an upper bound.

11. Constant Complexity
An algorithm with the same number of operations regardless of the data set is said to have CONSTANT COMPLEXITY
O(1) or 0(c)
See previous algorithms
Most significant algorithms are NOT constant complexity

12. Complexity Examples
Do n times Read x, y, z x = y + z Print x Complexity? Depends on value of n! NOT Constant 6 * n basic operations O(6n)  O(n)
Do 10 times Read x, y, z x = y + z Print x Complexity? O(c) or O(1) - constant

13. Algorithm Complexity
Number of operations in terms of the data set size
Do 10 times
Read x, y, z
x = y + z
Print x
Constant Time Complexity
– O(1) or O(c)

14. Algorithm Complexity
Do n times Read x, y, z x = y + z Print x Time Complexity is dependent upon n 6n Operations - O(6n)  O(n)

15. *Big Oh (yes, memorize) O(f(n)) – Big Oh of f of n
A function g(n) = O(f(n)) if there exist 2 constants K and n0 such that
|g(n)| <= K|f(n)| for all n >=n0
Big Oh is an upper bound.

16. Evaluating xn Operations???
P = x k = 1 While k < n P = P * x k = k + 1 Print P
Operations???

17. Evaluating xn Total =3 + n + 2(n-1) + 2(n-1) 3 + n + 2n -2 + 2n – 2
P = x k = 1 While k < n P = P * x k = k + 1 Print P
Operations 1 n
2(n-1)
Total =3 + n + 2(n-1) + 2(n-1)
3 + n + 2n n – 2
5n – 1 = O(5n -1)  O(n)

18. Trace: xn  54 
P = x k = 1 While k < n P = P * x k = k + 1 Print P
(n=4, x=5) P = 5, k = 1 P = 5*5 (25); k= 2 P = 25*5 (125); k = 3 P = 125*5 (625); k=4 Print 625

19. Complexity & Rate of Growth
Complexity measures growth rate of algorithm time in terms of data size
O(1) Constant
O(n) Linear
O(n2) Polynomial (any constant power)
O(5n) Exponential (any constant base)
What do these functions look like when graphed?
How big are real world data sets? (name some)

20. Polynomial Evaluation P(x) = anxn + an-1xn-1 + … +a1x + a0
Given values for x, a0, a1,…an How many ops for anxn ? Operations to calculate each term in poly? n + n-1 + n-2 +… = (n*(n+1))/2 Additions to combine the n+1 terms = n Total Ops = (n2+n)/2 + n  O(n2) What did we omit from analysis???

21. Polynomial Evaluation (p.26) P(x) = anxn + an-1xn-1 + … +a1x + a0
Given values for x, a0, a1,…an S = a0 , k = 1 while k <=1 S = S + ak xk k = k+ 1 Print S
Can you spot any inefficiencies?

22. Polynomial Evaluation Algorithm (p
Polynomial Evaluation Algorithm (p.26) P(x) = anxn + an-1xn-1 + … +a1x + a0
Given values for
n, x, a0, a1,…an
S = a0 , k = 1
while k <=n
S = S + ak xk
k = k+ 1
Print S
*Slide 17: xk = 5k + 1 2
n + 1
#n(2+5(k+1)+1) 2n 1
#Total = 2n +5kn + 5n + n
5kn + 8n

23. Complexity of Poly. Evaluation
Total = (by line on previous slide)
=2 + (n + 1) + (5kn + 8n) + 2n + 1
= 5kn + 11n + 4
Worst case for k??
= 5n n + 4 O(n2)
Note: this is for inserting code for xk
Function call adds additional overhead.

24. Analysis of Polynomial
Compare Slides 20 and 22/23
Same Big Oh  O(n2)
“Different details” but same results

25. Horner’s Rule (Polynomial Evaluation) P(x) = anxn + an-1xn-1 + … +a1x + a0
S = an, k = 1 While k <= n S = Sx + an-k k = k = 1 Print s
2 n + 1 n * 3 n *2 1
Total = 2 + n+1 + 3n + n2 + 1 = 6n + 4
 O(n)

26. Summary & Homework See table on page 31
In general, an algorithm is considered “good” if its complexity is no more than some polynomial in n
For small n, some non-polynomial algorithms may be “acceptable”
Homework: Page 33+
1 – 10, 23 – 26 – will discuss in class
27 – 30 (detail for each step as on slides, state Total & Big Oh) – Turn in for grading