Introduction to Analysis of Algorithms Manolis Koubarakis Data Structures and Programming Techniques 1

Outline How can we measure and compare algorithms meaningfully? O notation. Analysis of a few interesting algorithms. Data Structures and Programming Techniques 2

Introduction How do we measure and compare algorithms meaningfully given that the same algorithm will run at different speeds and will require different amounts of space when run on different computers or when implemented in different programming languages? Example: let us consider a sorting algorithm for sorting an array A[0:n-1]. Data Structures and Programming Techniques 3

Selection Sorting Algorithm void SelectionSort(InputArray A) { int MinPosition, temp, i, j; for (i=n-1; i>0; --i){ MinPosition=i; for (j=0; j<i; ++i){ if (A[j] < A[MinPosition]){ MinPosition=j; } temp=A[i]; A[i]=A[MinPosition]; A[MinPosition]=temp; } Data Structures and Programming Techniques 4

Running Times in Seconds to Sort an Array of 2000 Integers Computers A, B, etc. up to E are progressively faster. The algorithm runs faster on faster computers. Type of ComputerTime Computer A Computer B Computer C2.382 Computer D0.431 Computer E0.087 Data Structures and Programming Techniques 5

More Measurements In addition to trying different computers, we should try different programming languages and different compilers. Shall we take all these measurements to decide whether an algorithm is better than another one? Data Structures and Programming Techniques 6

A More Meaningful Criterion Data Structures and Programming Techniques 7

SelectionSort Running Times in Milliseconds on Two Types of Computers Array Size nHome ComputerDesktop Computer Data Structures and Programming Techniques 8

Two Curves Fitting the Previous Data Data Structures and Programming Techniques 9

Discussion Data Structures and Programming Techniques 10

Complexity Classes The running times of various algorithms belong to different complexity classes. Each complexity class is characterized by a different family of curves. All of the curves in a given complexity class share the same basic shape. The shape is characterized by an equation that gives running times as a function of problem size. Data Structures and Programming Techniques 11

Data Structures and Programming Techniques 12

Data Structures and Programming Techniques

Data Structures and Programming Techniques 14

Some Common Complexity Classes Adjective Name Constant Logarithmic n log n Linear Quadratic Cubic Exponential Data Structures and Programming Techniques 15

Comparing Complexity Classes Data Structures and Programming Techniques 16

Running Times for Algorithm A 1 μsec 4 μsec8 μsec10 μsec 2 μsec16 μsec256 μsec1.02 ms 2 μsec64 μsec2.05 ms10.2 ms 4 μsec25.6 μsec65.5 ms1.05 secs 8 μsec4.1 ms16.8 ms17.9 min 4 μsec65.5 ms Data Structures and Programming Techniques 17

Number of steps isT = 1 minT = 1hr Data Structures and Programming Techniques 18

Time Complexity Discussion Data Structures and Programming Techniques 19

Time Complexity Discussion (cont’d) Data Structures and Programming Techniques 20

Time Complexity We have to consider the following cases: – Worst case – Best case – Average case Data Structures and Programming Techniques 21

Examples of Well-Known Algorithms and Their Time Complexity Data Structures and Programming Techniques 22

Data Structures and Programming Techniques 23

Three Ways of Saying it in Words Data Structures and Programming Techniques 24

Example of a Formal Proof Data Structures and Programming Techniques 25

Proof (cont’d) Data Structures and Programming Techniques 26

Proof (cont’d) Data Structures and Programming Techniques 27

Data Structures and Programming Techniques 28

Data Structures and Programming Techniques 29

Example Data Structures and Programming Techniques 30

Ignoring Bases of Logarithms Data Structures and Programming Techniques 31

Data Structures and Programming Techniques 32

Some Algorithms and Their Complexity Sequential searching Selection sort Recursive selection sort Towers of Hanoi Data Structures and Programming Techniques 33

Analysis of Sequential Searching Data Structures and Programming Techniques 34

An Algorithm for Sequential Searching #define n 100 typedef int Key; typedef Key SearchArray[n]; int SequentialSearch(Key K, SearchArray A) { int i; for (i=0; i<n; ++i){ if (K==A[i]) return i; } return(-1); } Data Structures and Programming Techniques 35

Complexity Analysis Data Structures and Programming Techniques 36

Complexity Analysis (cont’d) Data Structures and Programming Techniques 37

Complexity Analysis (cont’d) Data Structures and Programming Techniques 38

Selection Sorting Algorithm void SelectionSort(InputArray A) { int MinPosition, temp, i, j; for (i=n-1; i>0; --i){ MinPosition=i; for (j=0; j<i; ++i){ if (A[j] < A[MinPosition]){ MinPosition=j; } temp=A[i]; A[i]=A[MinPosition]; A[MinPosition]=temp; } Data Structures and Programming Techniques 39

Complexity Analysis of SelectionSort Data Structures and Programming Techniques 40

Complexity Analysis (cont’d) Data Structures and Programming Techniques 41

Recursive SelectionSort /* FindMin is an auxiliary function used by the Selection sort below */ int FindMin(InputArray A, int n) { int i,j=n; for (i=0; i<n; ++i) if (A[i]<A[j]) j=i; return j; } void SelectionSort(InputArray A, int n) { int MinPosition, temp; if (n>0){ MinPosition=FindMin(A,n); temp=A[n]; A[n]=A[MinPosition]; A[MinPosition]=temp; SelectionSort(A, n-1) } Data Structures and Programming Techniques 42

Analysis of Recursive SelectionSort Data Structures and Programming Techniques 43

Analysis of Recursive SelectionSort (cont’d) Data Structures and Programming Techniques 44

Analysis of Recursive SelectionSort (cont’d) Data Structures and Programming Techniques 45

Analysis of Recursive SelectionSort (cont’d) Data Structures and Programming Techniques 46

Analysis of Recursive SelectionSort (cont’d) Data Structures and Programming Techniques 47

Analysis of Recursive SelectionSort (cont’d) Data Structures and Programming Techniques 48

A Recursive Solution to the Towers of Hanoi void MoveTowers(int n, int start, int finish, int spare) { if (n==1){ printf(“Move a disk from peg %1d to peg %1d\n”, start, finish); } else { MoveTowers(n-1, start, spare, finish); printf(“Move a disk from peg %1d to peg %1d\n”, start, finish); MoveTowers(n-1, spare, finish, start); } Data Structures and Programming Techniques 49

Analysis of Towers of Hanoi Data Structures and Programming Techniques 50

Analysis of Towers of Hanoi (cont’d) Data Structures and Programming Techniques 51

Analysis of Towers of Hanoi (cont’d) Data Structures and Programming Techniques 52

Analysis of Towers of Hanoi (cont’d) Data Structures and Programming Techniques 53

Analysis of Towers of Hanoi (cont’d) Data Structures and Programming Techniques 54

Data Structures and Programming Techniques 55

Space Complexity In a similar way, we can measure the space complexity of an algorithm. Data Structures and Programming Techniques 56

Other notations Data Structures and Programming Techniques 57

Readings T. A. Standish. Data Structures, Algorithms and Software Principles in C. Chapter 6. Data Structures and Programming Techniques 58