1. Prof. Amr Goneid, AUC1 Analysis & Design of Algorithms (CSCE 321) Prof. Amr Goneid Department of Computer Science, AUC Part 2. Types of Complexities

2. Prof. Amr Goneid, AUC2 Types of Complexities

3. Prof. Amr Goneid, AUC3 Types of Complexities Rules for Upper Bound Comparing Complexities Types of Complexities

4. Prof. Amr Goneid, AUC4 1. Rules for Upper Bound If (k) is a constant, then: O(k)  O(n) O(k f(n)) = O(f(n)) T(n) n O(n) O(k) O(kn)

5. Prof. Amr Goneid, AUC5 Rules for Big O O(f(n)) + O(g(n)) = max(O(f(n)),O(g(n))) e.g. f(n) = 2n = O(n), g(n) = 0.1 n 3 = O(n 3 ) T(n) = max(O(n), O(n 3 )) = O(n 3 ) 2n 0.1 n 3 O(n 3 )

6. Prof. Amr Goneid, AUC6 Rules for Big O O(f(n)) * O(g(n)) = O(f(n) * g(n)) e.g. f(n) = n, g(n) = n 2 T(n) = n * n 2 = O(n 3 ) O(n 2 ) Repeat n times O(n 3 )

7. Prof. Amr Goneid, AUC7 Rules for Big O Logarithmic Complexity < Linear Complexity O(log n)  O(n) Claim: for all n ≥ 1, log n ≤ n. Prove by induction on n.

8. Prof. Amr Goneid, AUC8 Rules for Big O For a polynomial of degree m, Prove! O(n m-1 )  O(n m ) follows from above

9. Prof. Amr Goneid, AUC9 Summary of Rules for Big-O RuleExample For constant k, O(k) < O(n) O(7) = O(1) < O(n) For constant k, O(kf) = O(f) O(2n) = O(n) O(|f|+|g|) = O(|f|) + O(|g|) = Max (O(|f|), O(|g|) O(6n 2 +n)=O(6n 2 )+O(n) = O(n 2 ) Nesting of loop O(g) within a loop O(f) gives O(f*g) O(n 4 *n 2 )=O(n 6 ) O(n m-1 ) < O(n m )O(n 2 ) < O(n 3 ) O((log n) k )  O(n) O(log n) < O(n)

10. Prof. Amr Goneid, AUC10 2. Comparing Complexities Dominance: If lim (n->  ) f(n)/g(n) =  then f(n) dominates (i.e. grows faster), but if lim (n->  ) f(n)/g(n) = 0 then g(n) dominates. In the latter case, we say that f(n) = o(g(n)) little oh

11. Prof. Amr Goneid, AUC11 Comparing Complexities Examples: if a > b then n a dominates n b Why? n 2 dominates (3n+2) Why? n 2 dominates (n log n) Why?

12. Prof. Amr Goneid, AUC12 Using l’Hopital’s Rule (1696) Example: Show that f(n) = n (k+α) + n k (log n) 2 and g(n) = k n (k+α) grow at the same rate.

13. Prof. Amr Goneid, AUC13 Comparing Complexities Which grows faster: f(n) = n 3 or g(n) = n log n f(n) = n 0.001 org(n) = log n f(n) = 2 n+1 org(n) = 2 n f(n) = 2 n org(n) = 2 2n

14. Prof. Amr Goneid, AUC14 Exercises Which function has smaller complexity ? f = 100 n 4 g = n 5 f = log(log n 3 )g = log n f = n 2 g = n log n f = 50 n 5 + n 2 + ng = n 5 f = e n g = n!

15. Prof. Amr Goneid, AUC15 3. Types of Complexities Constant Complexity T(n) = constant independent of (n) Runs in constant amount of time  O(1) Example: cout << a[0][0]

16. Prof. Amr Goneid, AUC16 Types of Complexities Logarithmic Complexity Log 2 n=m is equivalent to n=2 m Reduces the problem to half  O(log 2 n) Example: Binary Search T(n) = O(log 2 n) Much faster than Linear Search which has T(n) = O(n)

17. Prof. Amr Goneid, AUC17 Linear vs Logarithmic Complexities n T(n) O(log 2 n) O(n)

18. Prof. Amr Goneid, AUC18 Types of Complexities Polynomial Complexity T(n)=a m n m +…+ a 2 n 2 + a 1 n 1 + a 0 If m=1, then O(a 1 n+a 0 )  O(n) If m > 1, then  O(n m ) as n m dominates

19. Prof. Amr Goneid, AUC19 Polynomials Complexities O(n) O(n 2 ) O(n 3 ) n Log T(n)

20. Prof. Amr Goneid, AUC20 Types of Complexities Exponential Example: List all the subsets of a set of n elements {a,b,c} {a,b,c}, {a,b},{a,c},{b,c},{a},{b},{c},{} Number of operations T(n) = O(2 n ) Exponential expansion of the problem  O(a n ) where a is a constant greater than 1

21. Prof. Amr Goneid, AUC21 Exponential Vs Polynomial Log T(n) n O(n 3 ) O(n) O(2 n )

22. Prof. Amr Goneid, AUC22 Types of Complexities Factorial time Algorithms Example: Traveling salesperson problem (TSP): Find the best route to take in visiting n cities away from home. What are the number of possible routes? For 3 cities: (A,B,C)

23. Prof. Amr Goneid, AUC23 Possible routes in a TSP –Number of operations = 3!=6, Hence T(n) = n! –Expansion of the problem  O(n!)

24. Prof. Amr Goneid, AUC24 Exponential Vs Factorial Log T(n) n O(2 n ) O(n!) O(n n )

25. Prof. Amr Goneid, AUC25 Execution Time Example Example: For the exponential algorithm of listing all subsets of a given set, assume the set size to be of 1024 elements Number of operations is 2 1024 about 1.8*10 308 If we can list a subset every nanosecond the process will take 5.7 * 10 291 yr!!!

26. Prof. Amr Goneid, AUC26 P and NP – Times P (Polynomial) Times: O(1), O(log n), O(log n) 2, O(n), O(n log n), O(n 2 ), O(n 3 ), …. NP (Non-Polynomial) Times: O(2 n ), O(e n ), O(n!), O(n n ), …..

27. Prof. Amr Goneid, AUC27 P and NP – Times Polynomial time is GOOD Try to reduce the polynomial power NP (e.g. Exponential) Time is BAD