1. Introduction to Analysis of Algorithms Manolis Koubarakis Data Structures and Programming Techniques 1

2. Outline How can we measure and compare algorithms meaningfully? O notation. Analysis of a few interesting algorithms. Data Structures and Programming Techniques 2

3. Introduction How do we measure and compare algorithms meaningfully given that the same algorithm will run at different speeds and will require different amounts of space when run on different computers or when implemented in different programming languages? Example: let us consider a sorting algorithm for sorting an array A[0:n-1]. Data Structures and Programming Techniques 3

4. Selection Sorting Algorithm void SelectionSort(InputArray A) { int MinPosition, temp, i, j; for (i=n-1; i>0; --i){ MinPosition=i; for (j=0; j<i; ++i){ if (A[j] < A[MinPosition]){ MinPosition=j; } temp=A[i]; A[i]=A[MinPosition]; A[MinPosition]=temp; } Data Structures and Programming Techniques 4

5. Running Times in Seconds to Sort an Array of 2000 Integers Computers A, B, etc. up to E are progressively faster. The algorithm runs faster on faster computers. Type of ComputerTime Computer A51.915 Computer B11.508 Computer C2.382 Computer D0.431 Computer E0.087 Data Structures and Programming Techniques 5

6. More Measurements In addition to trying different computers, we should try different programming languages and different compilers. Shall we take all these measurements to decide whether an algorithm is better than another one? Data Structures and Programming Techniques 6

7. A More Meaningful Criterion Data Structures and Programming Techniques 7

8. SelectionSort Running Times in Milliseconds on Two Types of Computers Array Size nHome ComputerDesktop Computer 12512.52.8 25049.311.0 500195.843.4 1000780.3172.9 20003114.9690.5 Data Structures and Programming Techniques 8

9. Two Curves Fitting the Previous Data Data Structures and Programming Techniques 9

10. Discussion Data Structures and Programming Techniques 10

11. Complexity Classes The running times of various algorithms belong to different complexity classes. Each complexity class is characterized by a different family of curves. All of the curves in a given complexity class share the same basic shape. The shape is characterized by an equation that gives running times as a function of problem size. Data Structures and Programming Techniques 11

12. Data Structures and Programming Techniques 12

13. Data Structures and Programming Techniques 13 1252.82.794.7 25011.010.898.2 50043.443.199.3 1000172.9172.499.7 2000690.5689.699.9

14. Data Structures and Programming Techniques 14

15. Some Common Complexity Classes Adjective Name Constant Logarithmic n log n Linear Quadratic Cubic Exponential Data Structures and Programming Techniques 15

16. Comparing Complexity Classes Data Structures and Programming Techniques 16

17. Running Times for Algorithm A 1 μsec 4 μsec8 μsec10 μsec 2 μsec16 μsec256 μsec1.02 ms 2 μsec64 μsec2.05 ms10.2 ms 4 μsec25.6 μsec65.5 ms1.05 secs 8 μsec4.1 ms16.8 ms17.9 min 4 μsec65.5 ms Data Structures and Programming Techniques 17

18. Number of steps isT = 1 minT = 1hr 2531 79 Data Structures and Programming Techniques 18

19. Time Complexity Discussion Data Structures and Programming Techniques 19

20. Time Complexity Discussion (cont’d) Data Structures and Programming Techniques 20

21. Time Complexity We have to consider the following cases: – Worst case – Best case – Average case Data Structures and Programming Techniques 21

22. Examples of Well-Known Algorithms and Their Time Complexity Data Structures and Programming Techniques 22

23. Data Structures and Programming Techniques 23

24. Three Ways of Saying it in Words Data Structures and Programming Techniques 24

25. Example of a Formal Proof Data Structures and Programming Techniques 25

26. Proof (cont’d) Data Structures and Programming Techniques 26

27. Proof (cont’d) Data Structures and Programming Techniques 27

28. Data Structures and Programming Techniques 28

29. Data Structures and Programming Techniques 29

30. Example Data Structures and Programming Techniques 30

31. Ignoring Bases of Logarithms Data Structures and Programming Techniques 31

32. Data Structures and Programming Techniques 32

33. Some Algorithms and Their Complexity Sequential searching Selection sort Recursive selection sort Towers of Hanoi Data Structures and Programming Techniques 33

34. Analysis of Sequential Searching Data Structures and Programming Techniques 34

35. An Algorithm for Sequential Searching #define n 100 typedef int Key; typedef Key SearchArray[n]; int SequentialSearch(Key K, SearchArray A) { int i; for (i=0; i<n; ++i){ if (K==A[i]) return i; } return(-1); } Data Structures and Programming Techniques 35

36. Complexity Analysis Data Structures and Programming Techniques 36

37. Complexity Analysis (cont’d) Data Structures and Programming Techniques 37

38. Complexity Analysis (cont’d) Data Structures and Programming Techniques 38

39. Selection Sorting Algorithm void SelectionSort(InputArray A) { int MinPosition, temp, i, j; for (i=n-1; i>0; --i){ MinPosition=i; for (j=0; j<i; ++i){ if (A[j] < A[MinPosition]){ MinPosition=j; } temp=A[i]; A[i]=A[MinPosition]; A[MinPosition]=temp; } Data Structures and Programming Techniques 39

40. Complexity Analysis of SelectionSort Data Structures and Programming Techniques 40

41. Complexity Analysis (cont’d) Data Structures and Programming Techniques 41

42. Recursive SelectionSort /* FindMin is an auxiliary function used by the Selection sort below */ int FindMin(InputArray A, int n) { int i,j=n; for (i=0; i<n; ++i) if (A[i]<A[j]) j=i; return j; } void SelectionSort(InputArray A, int n) { int MinPosition, temp; if (n>0){ MinPosition=FindMin(A,n); temp=A[n]; A[n]=A[MinPosition]; A[MinPosition]=temp; SelectionSort(A, n-1) } Data Structures and Programming Techniques 42

43. Analysis of Recursive SelectionSort Data Structures and Programming Techniques 43

44. Analysis of Recursive SelectionSort (cont’d) Data Structures and Programming Techniques 44

45. Analysis of Recursive SelectionSort (cont’d) Data Structures and Programming Techniques 45

46. Analysis of Recursive SelectionSort (cont’d) Data Structures and Programming Techniques 46

47. Analysis of Recursive SelectionSort (cont’d) Data Structures and Programming Techniques 47

48. Analysis of Recursive SelectionSort (cont’d) Data Structures and Programming Techniques 48

49. A Recursive Solution to the Towers of Hanoi void MoveTowers(int n, int start, int finish, int spare) { if (n==1){ printf(“Move a disk from peg %1d to peg %1d\n”, start, finish); } else { MoveTowers(n-1, start, spare, finish); printf(“Move a disk from peg %1d to peg %1d\n”, start, finish); MoveTowers(n-1, spare, finish, start); } Data Structures and Programming Techniques 49

50. Analysis of Towers of Hanoi Data Structures and Programming Techniques 50

51. Analysis of Towers of Hanoi (cont’d) Data Structures and Programming Techniques 51

52. Analysis of Towers of Hanoi (cont’d) Data Structures and Programming Techniques 52

53. Analysis of Towers of Hanoi (cont’d) Data Structures and Programming Techniques 53

54. Analysis of Towers of Hanoi (cont’d) Data Structures and Programming Techniques 54

55. Data Structures and Programming Techniques 55

56. Space Complexity In a similar way, we can measure the space complexity of an algorithm. Data Structures and Programming Techniques 56

57. Other notations Data Structures and Programming Techniques 57

58. Readings T. A. Standish. Data Structures, Algorithms and Software Principles in C. Chapter 6. Data Structures and Programming Techniques 58