MATH 224 – Discrete Mathematics Algorithms and Complexity An algorithm is a precise recipe or set of instruction for solving a problem. In addition, to be considered an algorithm the set of instructions must solve the problem with a finite number of steps. In other words, if the algorithm is implemented on a computer, it must terminate with a solution. An algorithm should leave nothing to chance and must not have any infinite loops. (There are some algorithms that do use random or probabilistic techniques, and therefore may leave some things to chance.) Algorithms solve problems such as sorting, find an object, finding the shortest path between two points, determining mathematical functions such as logarithms among others. Specifying an Algorithms Informal English description Formal English description Pseudo-Code Code in programming language (e.g., C++) 4/10/2017

MATH 224 – Discrete Mathematics Describing an Algorithm – Sorting an Array 0..n−1 Informal description of selection sort Find the smallest number and put it in the first position Find the second smallest number and put it in the second position Continue as above until the array is sorted Formal Description Set i = 0 Find the smallest value between positions i and n−1 Swap the value at i with the smallest value Set i = i+1 If i < n −1 go back to Step 2, otherwise quit 4/10/2017

MATH 224 – Discrete Mathematics Describing an Algorithm – Sorting an Array 0.. n−1 Pseudo-Code void function sort(Array A[n]) for i in [0..n-2] → pos := i for j in [i+1.. n-1] → if A[j] < A[pos] → pos := j fi rof swap(A[i], A[pos]) end sort C++ Version void function sort (A_type A[ ], int n) { for (int i = 0; i < n-1; i++) { pos = i; for (int j = i+1; j < n; j++) if (A[j] < A[pos]) pos = j; // fi // rof swap(A[i], A[pos]); } // rof } // end sort Note the use of “:=” for assignment as do Pascal and Ada. 4/10/2017

MATH 224 – Discrete Mathematics Describing an Algorithm – Insertion Sort Array 0.. n−1 Pseudo-Code procedure insert_sort (A_type A[ ], int n) for i1 in [1..n-1] → i2 := i1–1 item := A[i1] do (i2 ≥ 0) and (item < A[i2]) → A[i2+1] := A[i2]; i2 := i2 –1 od A[i2+1] := item rof end insert_sort C++ Version void function insert_sort (A_type A[ ], int n) { int i1. i2; A_type item; for (int i1 = 1; i < n; i++) { i2 = i1 –1 ; item = A[i1]; while (i2 >= 0 && item < A[i2]) { A[i2+1] = A[i2]; i2 – – ; } // end while A[i2 + 1] = item; } // rof } // end insert_sort 4/10/2017

MATH 224 – Discrete Mathematics Binary search of an Array 1.. n Pseudo-Code – Algorithm 3 from the textbook (Page 172) procedure binary_search(x: integer a[1..n] : integer) i := 1; j := n while i < j m := └(i+j)/2┘ // just integer division in C++ if x > a[m] i := m+1 else j := m fi end while if x = a[i] location is i // here = corresponds to else x is not in the array // C++ = = end binary_search Note the text uses “:=” for assignment as do Pascal and Ada. 4/10/2017

MATH 224 – Discrete Mathematics Exponentiation ap Incremental version int function exp(int a, int p) if p ≤ 0 → return 1 | p > 0 → val := a for i in [1..p-1] → val := val*a rof fi return val end exp How times does the code inside the for statement execute? 4/10/2017

MATH 224 – Discrete Mathematics Exponentiation ap Divide-and-Conquer version (using recursion) int function exp(int a, int p) if p ≤ 0 → return 1 | p > 0 → val := exp(a, p/2) if p is even → return val*val | p is odd → return val*val*a fi end exp How many times does the if statement execute? 4/10/2017

MATH 224 – Discrete Mathematics Exponentiation ap The Program (Run Time) Stack MAIN a=3, p=11 exp(3, 5) val = 9 exp(3, 2) val = 3 exp(3, 1) val = 1 exp(3, 0) Return 1 int function exp(int a, int p) if p ≤ 0 → return 1 | p > 0 → val := exp(a, p/2) if p is even → return val*val | p is odd → return val*val*a fi end exp 1 3 9 243 exp(3, 11) val = 243 177,147 4/10/2017

MATH 224 – Discrete Mathematics Greedy Algorithm for Change Algorithm 6 from the textbook (Page 175) procedure change(n : integer c[1..r] : integer) // C is an array of coins sorted from for i := 1 to r // largest to smallest, and n is the while n > c[i] // the total amount of change. add c[i] to change n = n – c[i] end while rof end change 4/10/2017

MATH 224 – Discrete Mathematics Asymptotic Growth of Functions Big-0 – an Upper Bound Omega – a Lower Bound Theta – a Tight Bound 4/10/2017

MATH 224 – Discrete Mathematics Examples Functions and their Asymptotic Growth 4/10/2017

MATH 224 – Discrete Mathematics Polynomial Functions O(N2) T(N) n0 = 10 for O n0 = 0 for  (N2) 1 10 20 3N2 4N2 3N2 +10N +3 4/10/2017

MATH 224 – Discrete Mathematics Log Function n0 = 5.7*106 N 0.2 Lg(N) N0.2 X axis * 105 4/10/2017

MATH 224 – Discrete Mathematics Evaluating Algorithms and Complexity Execute the algorithm on a standard computer with standard tests Problems: depends on the input and the computer Count the number of steps Problem: depends on the input and can be difficult to do Use big-O (asymptotic) evaluation Problem: provides an approximation and does not give the full picture 4/10/2017

MATH 224 – Discrete Mathematics Searching an Array -4 -1 7 12 21 23 31 45 72 83 86 92 VALUE INDEX 9 10 11 1 2 4/10/2017

MATH 224 – Discrete Mathematics Linear Search of an Array -4 -1 7 12 21 23 31 45 72 83 86 92 VALUE INDEX 1 2 3 4 5 6 7 8 9 10 11 Inspects 11 out of 12 elements O(N) 4/10/2017

MATH 224 – Discrete Mathematics Binary Search of an Array -4 -1 7 12 21 23 31 45 72 83 86 92 VALUE INDEX 1 2 5 8 9 10 11 1 2 3 4 Searches O(log n) locations (4 in this example) Divides segments in half until a value is found 4/10/2017

MATH 224 – Discrete Mathematics Complexity of Algorithms The complexity of an algorithm refers to how long it takes for an algorithm to solve a problem (time complexity) or how much memory the algorithm takes (space complexity). Complexity normally expressed as a function of one or more variables. For example, in the previous slides the binary search algorithm is said to take lg(n) steps, where n is the number of elements in the array being searched. Complexity is normally express using O, Ω, or Θ notation since the precise running time is often difficult to calculate and will depend on the computer that is used. 4/10/2017

MATH 224 – Discrete Mathematics Types of Complexity Most often when considering an algorithm either the worst-case or average-case complexity is analyzed. More rarely the best-case complexity may be considered. Worst-case refers to the maximum for a given size problem. So for example the worst-case time complexity for binary search is Θ(lg(x)) for an array of size x. Average-case refers to the average for an algorithm of a given size. It turns out that the average case is also Θ(lg(x)) for binary search since the average case takes one less iteration than does the worst-case. Best-case refers to the best an algorithm can do in the ideal case. So, for example, if the item being searched is in the exact middle of an array, binary search will find it on the first iteration. What would be the best-case time complexity for binary search? Best case is rarely used. 4/10/2017

MATH 224 – Discrete Mathematics Worst-Case and Big-O Because it is the easiest to calculate, worst-case analysis is most often used. In addition, it is often the most useful since it is often necessary to minimize the worst-case time or memory usage. Also Big-O is most often used when stating the complexity of an algorithm since it is often easier to calculate than Θ. In most cases, people are in the habit of using Big-O when they really mean Θ. Do you have any idea why Big-O is more common than Θ? 4/10/2017

MATH 224 – Discrete Mathematics Complexity of Selection Sort void function sort(Array A[n]) for i in [0..n-2] → pos := i for j in [i+1.. n−1] → if A[j] < A[pos] → pos := j fi rof swap(A[i], A[pos]) end sort How many times does the outer for-loop execute? The inner loop is more complex since the starting value keeps increasing. When i = 0, the inner loop executes n − 1 times, but when i = n − 2 it only executes once. So the number of steps is indicated by a sum of n − 1 + n − 2, n − 3, … 1. How is this sum written using the summation symbol Σ? 4/10/2017

MATH 224 – Discrete Mathematics Complexity of Selection Sort Thus the total number of steps in the worst-case is something like Σn−1 (2i ) + 2(n−1) +1 i=1 What does this look like in simplified form? n(n–1) + 2(n−1) + 1 = n2 + n − 1 What is this in Big-O or Θ? Note that Σn (i) = n(n+1)/2 void function sort(Array A[n]) for i in [0..n-2] → pos := i for j in [i+1.. n−1] → if A[j] < A[pos] → pos := j fi rof swap(A[i], A[pos]) end sort 4/10/2017