1. MATH 224 – Discrete Mathematics
Algorithms and Complexity
An algorithm is a precise recipe or set of instruction for solving a problem. In addition, to be considered an algorithm the set of instructions must solve the problem with a finite number of steps. In other words, if the algorithm is implemented on a computer, it must terminate with a solution. An algorithm should leave nothing to chance and must not have any infinite loops. (There are some algorithms that do use random or probabilistic techniques, and therefore may leave some things to chance.)
Algorithms solve problems such as sorting, find an object, finding the shortest path between two points, determining mathematical functions such as logarithms among others.
Specifying an Algorithms
Informal English description
Formal English description
Pseudo-Code
Code in programming language (e.g., C++)
4/10/2017

2. MATH 224 – Discrete Mathematics
Describing an Algorithm – Sorting an Array 0..n−1
Informal description of selection sort
Find the smallest number and put it in the first position
Find the second smallest number and put it in the second position
Continue as above until the array is sorted
Formal Description
Set i = 0
Find the smallest value between positions i and n−1
Swap the value at i with the smallest value
Set i = i+1
If i < n −1 go back to Step 2, otherwise quit
4/10/2017

3. MATH 224 – Discrete Mathematics
Describing an Algorithm – Sorting an Array 0.. n−1
Pseudo-Code
void function sort(Array A[n])
for i in [0..n-2] →
pos := i
for j in [i+1.. n-1] →
if A[j] < A[pos] →
pos := j fi
rof
swap(A[i], A[pos])
end sort
C++ Version
void function sort (A_type A[ ], int n) {
for (int i = 0; i < n-1; i++) {
pos = i;
for (int j = i+1; j < n; j++)
if (A[j] < A[pos])
pos = j;
// fi
// rof
swap(A[i], A[pos]);
} // rof
} // end sort
Note the use of “:=” for assignment as do Pascal and Ada.
4/10/2017

4. MATH 224 – Discrete Mathematics
Describing an Algorithm – Insertion Sort Array 0.. n−1
Pseudo-Code
procedure insert_sort (A_type A[ ], int n)
for i1 in [1..n-1] →
i2 := i1–1
item := A[i1]
do (i2 ≥ 0) and (item < A[i2]) →
A[i2+1] := A[i2];
i2 := i2 –1 od
A[i2+1] := item
rof
end insert_sort
C++ Version
void function insert_sort (A_type A[ ], int n) {
int i1. i2;
A_type item;
for (int i1 = 1; i < n; i++) {
i2 = i1 –1 ;
item = A[i1];
while (i2 >= 0 && item < A[i2]) {
A[i2+1] = A[i2];
i2 – – ;
} // end while
A[i2 + 1] = item;
} // rof
} // end insert_sort
4/10/2017

5. MATH 224 – Discrete Mathematics
Binary search of an Array 1.. n
Pseudo-Code – Algorithm 3 from the textbook (Page 172)
procedure binary_search(x: integer a[1..n] : integer)
i := 1; j := n
while i < j
m := └(i+j)/2┘ // just integer division in C++
if x > a[m] i := m+1
else j := m fi
end while
if x = a[i] location is i // here = corresponds to
else x is not in the array // C++ = =
end binary_search
Note the text uses “:=” for assignment as do Pascal and Ada.
4/10/2017

6. MATH 224 – Discrete Mathematics
Exponentiation ap
Incremental version
int function exp(int a, int p)
if p ≤ 0 → return 1
| p > 0 →
val := a
for i in [1..p-1] →
val := val*a
rof fi
return val
end exp
How times does the code inside the for statement execute?
4/10/2017

7. MATH 224 – Discrete Mathematics
Exponentiation ap
Divide-and-Conquer version (using recursion)
int function exp(int a, int p)
if p ≤ 0 → return 1
| p > 0 →
val := exp(a, p/2)
if p is even → return val*val
| p is odd →
return val*val*a fi
end exp
How many times does the if statement execute?
4/10/2017

8. MATH 224 – Discrete Mathematics
Exponentiation ap
The Program (Run Time) Stack
MAIN a=3, p=11
exp(3, 5)
val = 9
exp(3, 2)
val = 3
exp(3, 1)
val = 1
exp(3, 0)
Return 1
int function exp(int a, int p)
if p ≤ 0 → return 1
| p > 0 →
val := exp(a, p/2)
if p is even → return val*val
| p is odd →
return val*val*a fi
end exp 1 3 9
243
exp(3, 11)
val = 243
177,147
4/10/2017

9. MATH 224 – Discrete Mathematics
Greedy Algorithm for Change
Algorithm 6 from the textbook (Page 175)
procedure change(n : integer c[1..r] : integer) // C is an array of coins sorted from
for i := 1 to r // largest to smallest, and n is the
while n > c[i] // the total amount of change.
add c[i] to change
n = n – c[i]
end while
rof
end change
4/10/2017

10. MATH 224 – Discrete Mathematics
Asymptotic Growth of Functions
Big-0 – an Upper Bound
Omega – a Lower Bound
Theta – a Tight Bound
4/10/2017

11. MATH 224 – Discrete Mathematics
Examples Functions and their Asymptotic Growth
4/10/2017

12. MATH 224 – Discrete Mathematics
Polynomial Functions
O(N2)
T(N)
n0 = 10 for O
n0 = 0 for 
(N2)
3N2
4N2
3N2
+10N +3
4/10/2017

13. MATH 224 – Discrete Mathematics
Log Function
n0 = 5.7*106
N 0.2
Lg(N)
N0.2
X axis * 105
4/10/2017

14. MATH 224 – Discrete Mathematics
Evaluating Algorithms and Complexity
Execute the algorithm on a standard computer with standard tests
Problems: depends on the input and the computer
Count the number of steps
Problem: depends on the input and can be difficult to do
Use big-O (asymptotic) evaluation
Problem: provides an approximation and does not give the full picture
4/10/2017

15. MATH 224 – Discrete Mathematics
Searching an Array -4 -1 7 12 21 23 31 45 72 83 86 92
VALUE
INDEX 9 10 11 1 2
4/10/2017

16. MATH 224 – Discrete Mathematics
Linear Search of an Array -4 -1 7 12 21 23 31 45 72 83 86 92
VALUE
INDEX 1
Inspects 11 out of 12 elements O(N)
4/10/2017

17. MATH 224 – Discrete Mathematics
Binary Search of an Array -4 -1 7 12 21 23 31 45 72 83 86 92
VALUE
INDEX 1 2 5
Searches O(log n) locations (4 in this example)
Divides segments in half until a value is found
4/10/2017

18. MATH 224 – Discrete Mathematics
Complexity of Algorithms
The complexity of an algorithm refers to how long it takes for an algorithm to solve a problem (time complexity) or how much memory the algorithm takes (space complexity). Complexity normally expressed as a function of one or more variables. For example, in the previous slides the binary search algorithm is said to take lg(n) steps, where n is the number of elements in the array being searched.
Complexity is normally express using O, Ω, or Θ notation since the precise running time is often difficult to calculate and will depend on the computer that is used.
4/10/2017

19. MATH 224 – Discrete Mathematics
Types of Complexity
Most often when considering an algorithm either the worst-case or average-case complexity is analyzed. More rarely the best-case complexity may be considered.
Worst-case refers to the maximum for a given size problem. So for example the worst-case time complexity for binary search is Θ(lg(x)) for an array of size x.
Average-case refers to the average for an algorithm of a given size. It turns out that the average case is also Θ(lg(x)) for binary search since the average case takes one less iteration than does the worst-case.
Best-case refers to the best an algorithm can do in the ideal case. So, for example, if the item being searched is in the exact middle of an array, binary search will find it on the first iteration. What would be the best-case time complexity for binary search? Best case is rarely used.
4/10/2017

20. MATH 224 – Discrete Mathematics
Worst-Case and Big-O
Because it is the easiest to calculate, worst-case analysis is most often used. In addition, it is often the most useful since it is often necessary to minimize the worst-case time or memory usage.
Also Big-O is most often used when stating the complexity of an algorithm since it is often easier to calculate than Θ. In most cases, people are in the habit of using Big-O when they really mean Θ. Do you have any idea why Big-O is more common than Θ?
4/10/2017

21. MATH 224 – Discrete Mathematics
Complexity of Selection Sort
void function sort(Array A[n])
for i in [0..n-2] →
pos := i
for j in [i+1.. n−1] →
if A[j] < A[pos] →
pos := j fi
rof
swap(A[i], A[pos])
end sort
How many times does the outer for-loop execute?
The inner loop is more complex since the starting value keeps increasing. When i = 0, the inner loop executes n − 1 times, but when i = n − 2 it only executes once. So the number of steps is indicated by a sum of n − 1 + n − 2, n − 3, … 1.
How is this sum written using the summation symbol Σ?
4/10/2017

22. MATH 224 – Discrete Mathematics
Complexity of Selection Sort
Thus the total number of steps in the worst-case is something like
Σn−1 (2i ) + 2(n−1) +1
i=1
What does this look like in simplified form?
n(n–1) + 2(n−1) + 1 = n2 + n − 1
What is this in Big-O or Θ?
Note that Σn (i) = n(n+1)/2
void function sort(Array A[n])
for i in [0..n-2] →
pos := i
for j in [i+1.. n−1] →
if A[j] < A[pos] →
pos := j fi
rof
swap(A[i], A[pos])
end sort
4/10/2017