1. Today’s topics Algorithms Reading: Sections 2.1-3.3 Upcoming
Algorithm review
Orders of Growth
Reading: Sections
Upcoming
Integers
The title of this course (at UF) is “Applications of Discrete Structures.” Actually that title is misleading. Although discrete math has many applications, which we’ll survey in a moment, this class is not really about applications so much as it is about basic mathematical skills and concepts. My guess is that the only reason we call the course “Applications of Discrete Structures” rather than just “Discrete Mathematics” so that the people over in the Math department don’t get upset that we’re teaching our own basic math course over here, rather than leaving it to them.
Anyway, that issue aside, you may be wondering, what is a “discrete structure” anyway? Well, by “discrete” we don’t mean something that’s kept secret – that’s “discreet” with a double-e, a homonym, a completely different English word with a completely different meaning. No, our word “discrete” just means something made of distinct, atomic parts, as opposed to something that is smooth and continuous, like the curves you studied in calculus. Later in the course we’ll learn how to formalize this distinction more precisely using the concepts of countable versus uncountable sets. But for now, this intuitive concept suffices.
By “structures” we just mean objects that can be built up from simpler objects according to some definite, regular pattern, as opposed to unstructured, amorphous blobs. In this course, we’ll see how various types of discrete mathematical objects can be built up from other discrete objects in a well-defined way.
The field of discrete mathematics can be summarized as the study of discrete, mathematical (that is, well-defined, conceptual) objects, both primitive objects and more complex structures.
CompSci 102
© Michael Frank

2. §2.1: Algorithms The foundation of computer programming.
Most generally, an algorithm just means a definite procedure for performing some sort of task.
A computer program is simply a description of an algorithm, in a language precise enough for a computer to understand, requiring only operations that the computer already knows how to do.
We say that a program implements (or “is an implementation of”) its algorithm.
CompSci 102
© Michael Frank

3. Algorithm Characteristics
Some important general features of algorithms:
Input. Information or data that comes in.
Output. Information or data that goes out.
Definiteness. Algorithm is precisely defined.
Correctness. Outputs correctly relate to inputs.
Finiteness. Won’t take forever to describe or run.
Effectiveness. Individual steps are all do-able.
Generality. Works for many possible inputs.
Efficiency. Takes little time & memory to run.
CompSci 102
© Michael Frank

4. Our Pseudocode Language: §A2
Declaration
STATEMENTS
procedure name(argument: type)
variable := expression
informal statement
begin statements end
{comment}
if condition then statement [else statement]
for variable := initial value to final value statement
while condition statement
procname(arguments)
Not defined in book:
return expression
CompSci 102
© Michael Frank

5. procedure procname(arg: type)
Declares that the following text defines a procedure named procname that takes inputs (arguments) named arg which are data objects of the type type.
Example: procedure maximum(L: list of integers) [statements defining maximum…]
CompSci 102
© Michael Frank

6. variable := expression
An assignment statement evaluates the expression expression, then reassigns the variable variable to the value that results.
Example assignment statement: v := 3x (If x is 2, changes v to 13.)
In pseudocode (but not real code), the expression might be informally stated:
x := the largest integer in the list L
CompSci 102
© Michael Frank

7. Keep in mind that real programming languages never allow this.
Informal statement
Sometimes we may write a statement as an informal English imperative, if the meaning is still clear and precise: e.g.,“swap x and y”
Keep in mind that real programming languages never allow this.
When we ask for an algorithm to do so-and-so, writing “Do so-and-so” isn’t enough!
Break down algorithm into detailed steps.
CompSci 102
© Michael Frank

8. begin statements end
Groups a sequence of statements together: begin statement statement … statement n end
Allows the sequence to be used just like a single statement.
Might be used:
After a procedure declaration.
In an if statement after then or else.
In the body of a for or while loop.
Curly braces {} are used instead in many languages.
CompSci 102
© Michael Frank

9. Not executed (does nothing).
{comment}
Not executed (does nothing).
Natural-language text explaining some aspect of the procedure to human readers.
Also called a remark in some real programming languages, e.g. BASIC.
Example, might appear in a max program:
{Note that v is the largest integer seen so far.}
CompSci 102
© Michael Frank

10. if condition then statement
Evaluate the propositional expression condition.
If the resulting truth value is True, then execute the statement statement;
otherwise, just skip on ahead to the next statement after the if statement.
Variant: if cond then stmt1 else stmt2
Like before, but iff truth value is False, executes stmt2.
CompSci 102
© Michael Frank

11. while condition statement
Evaluate the propositional (Boolean) expression condition.
If the resulting value is True, then execute statement.
Continue repeating the above two actions over and over until finally the condition evaluates to False; then proceed to the next statement.
CompSci 102
© Michael Frank

12. while condition statement
Also equivalent to infinite nested ifs, like so: if condition begin statement if condition begin statement …(continue infinite nested if’s) end end
CompSci 102
© Michael Frank

13. for var := initial to final stmt
Initial is an integer expression.
Final is another integer expression.
Semantics: Repeatedly execute stmt, first with variable var := initial, then with var := initial+1, then with var := initial+2, etc., then finally with var := final.
Question: What happens if stmt changes the value of var, or the value that initial or final evaluates to?
CompSci 102
© Michael Frank

14. for var := initial to final stmt
For can be exactly defined in terms of while, like so:
begin var := initial while var  final begin stmt var := var end end
CompSci 102
© Michael Frank

15. procedure(argument)
A procedure call statement invokes the named procedure, giving it as its input the value of the argument expression.
Various real programming languages refer to procedures as functions (since the procedure call notation works similarly to function application f(x)), or as subroutines, subprograms, or methods.
CompSci 102
© Michael Frank

16. Max procedure in pseudocode
procedure max(a1, a2, …, an: integers)
v := a1 {largest element so far}
for i := 2 to n {go thru rest of elems}
if ai > v then v := ai {found bigger?}
{at this point v’s value is the same as the largest integer in the list}
return v
CompSci 102
© Michael Frank

17. Inventing an Algorithm
Requires a lot of creativity and intuition
Like writing proofs.
Unfortunately, we can’t give you an algorithm for inventing algorithms.
Just look at lots of examples…
And practice (preferably, on a computer)
And look at more examples…
And practice some more… etc., etc.
CompSci 102
© Michael Frank

18. Algorithm-Inventing Example
Suppose we ask you to write an algorithm to compute the predicate:
IsPrime:N→{T,F}
Computes whether a given natural number is a prime number.
First, start with a correct predicate-logic definition of the desired function:
n: IsPrime(n)  ¬1<d<n: d|n
Means d divides n evenly (without remainder)
CompSci 102
© Michael Frank

19. IsPrime example, cont.
Notice that the negated exponential can be rewritten as a universal:
¬1<d<n: d|n  1<d<n: d | n  2≤ d ≤ n−1: d | n
This universal can then be translated directly into a corresponding for loop:
for d := 2 to n−1 { Try all potential divisors >1 & <n } if d|n then return F { n has divisor d; not prime } return T { no divisors were found; n must be prime}
Means d does not divide n evenly (the remainder is ≠0)
CompSci 102
© Michael Frank

20. Optimizing IsPrime The IsPrime algorithm can be further optimized:
for d := 2 to n1/2 if d|n then return F return T
This works because of this theorem: If n has any (integer) divisors, it must have one less than n1/2.
Proof: Suppose n’s smallest divisor >1 is a, and let b :≡ n/a. Then n = ab, but if a > n1/2 then b > n1/2 (since a is n’s smallest divisor) and so n = ab > (n1/2)2 = n, an absurdity.
Note smaller range of search.
Further optimizations are possible:
- E.g., only try divisors that are primes less than n1/2.
CompSci 102
© Michael Frank

21. Problem of searching an ordered list.
Another example task
Problem of searching an ordered list.
Given a list L of n elements that are sorted into a definite order (e.g., numeric, alphabetical),
And given a particular element x,
Determine whether x appears in the list,
and if so, return its index (position) in the list.
Problem occurs often in many contexts.
Let’s find an efficient algorithm!
CompSci 102
© Michael Frank

22. Search alg. #1: Linear Search
procedure linear search (x: integer, a1, a2, …, an: distinct integers) i := 1 {start at beginning of list} while (i  n  x  ai) {not done, not found} i := i {go to the next position} if i  n then location := i {it was found} else location := 0 {it wasn’t found} return location {index or 0 if not found}
CompSci 102
© Michael Frank

23. Search alg. #2: Binary Search
Basic idea: On each step, look at the middle element of the remaining list to eliminate half of it, and quickly zero in on the desired element.
<x
<x
<x
>x
CompSci 102
© Michael Frank

24. Search alg. #2: Binary Search
procedure binary search (x:integer, a1, a2, …, an: distinct integers) i := 1 {left endpoint of search interval} j := n {right endpoint of search interval} while i<j begin {while interval has >1 item} m := (i+j)/2 {midpoint} if x>am then i := m+1 else j := m end if x = ai then location := i else location := 0 return location
CompSci 102
© Michael Frank

25. Practice exercises
2.1.3: Devise an algorithm that finds the sum of all the integers in a list. [2 min]
procedure sum(a1, a2, …, an: integers) s := {sum of elems so far} for i := 1 to n {go thru all elems} s := s + ai {add current item} {at this point s is the sum of all items} return s
CompSci 102
© Michael Frank

26. Orders of Growth - Motivation
Suppose you are designing a web site to process user data (e.g., financial records).
Suppose database program A takes fA(n)=30n+8 microseconds to process any n records, while program B takes fB(n)=n2+1 microseconds to process the n records.
Which program do you choose, knowing you’ll want to support millions of users? A
CompSci 102
© Michael Frank

27. Visualizing Orders of Growth
On a graph, as you go to the right, the faster- growing func- tion always eventually becomes the larger one...
fA(n)=30n+8
Value of function 
fB(n)=n2+1
Increasing n 
CompSci 102
© Michael Frank

28. Concept of order of growth
We say fA(n)=30n+8 is (at most) order n, or O(n).
It is, at most, roughly proportional to n.
fB(n)=n2+1 is order n2, or O(n2).
It is (at most) roughly proportional to n2.
Any function whose exact (tightest) order is O(n2) is faster-growing than any O(n) function.
Later we will introduce Θ for expressing exact order.
For large numbers of user records, the exactly order n2 function will always take more time.
CompSci 102
© Michael Frank

29. Definition: O(g), at most order g
Let g be any function RR.
Define “at most order g”, written O(g), to be: {f:RR | c,k: x>k: f(x)  cg(x)}.
“Beyond some point k, function f is at most a constant c times g (i.e., proportional to g).”
“f is at most order g”, or “f is O(g)”, or “f=O(g)” all just mean that fO(g).
Often the phrase “at most” is omitted.
CompSci 102
© Michael Frank

30. Points about the definition
Note that f is O(g) so long as any values of c and k exist that satisfy the definition.
But: The particular c, k, values that make the statement true are not unique: Any larger value of c and/or k will also work.
You are not required to find the smallest c and k values that work. (Indeed, in some cases, there may be no smallest values!)
However, you should prove that the values you choose do work.
CompSci 102
© Michael Frank

31. “Big-O” Proof Examples
Show that 30n+8 is O(n).
Show c,k: n>k: 30n+8  cn.
Let c=31, k=8. Assume n>k=8. Then cn = 31n = 30n + n > 30n+8, so 30n+8 < cn.
Show that n2+1 is O(n2).
Show c,k: n>k: n2+1  cn2.
Let c=2, k=1. Assume n>1. Then cn2 = 2n2 = n2+n2 > n2+1, or n2+1< cn2.
CompSci 102
© Michael Frank

32. Big-O example, graphically
Note 30n+8 isn’t less than n anywhere (n>0).
It isn’t even less than 31n everywhere.
But it is less than 31n everywhere to the right of n=8.
cn = 31n
n>k=8 
30n+8
30n+8 O(n)
Value of function  n
Increasing n 
CompSci 102
© Michael Frank

33. Useful Facts about Big O
Big O, as a relation, is transitive: fO(g)  gO(h)  fO(h)
O with constant multiples, roots, and logs...  f (in (1)) & constants a,bR, with b0, af, f 1-b, and (logb f)a are all O(f).
Sums of functions: If gO(f) and hO(f), then g+hO(f).
CompSci 102
© Michael Frank

34. c>0, O(cf)=O(f+c)=O(fc)=O(f) f1O(g1)  f2O(g2) 
More Big-O facts
c>0, O(cf)=O(f+c)=O(fc)=O(f)
f1O(g1)  f2O(g2) 
f1 f2 O(g1g2)
f1+f2 O(g1+g2) = O(max(g1,g2)) = O(g1) if g2O(g1) (Very useful!)
CompSci 102
© Michael Frank

35. Orders of Growth (§1.8) - So Far
For any g:RR, “at most order g”, O(g)  {f:RR | c,k x>k |f(x)|  |cg(x)|}.
Often, one deals only with positive functions and can ignore absolute value symbols.
“fO(g)” often written “f is O(g)” or “f=O(g)”.
The latter form is an instance of a more general convention...
CompSci 102
© Michael Frank

36. Order-of-Growth Expressions
“O(f)” when used as a term in an arithmetic expression means: “some function f such that fO(f)”.
E.g.: “x2+O(x)” means “x2 plus some function that is O(x)”.
Formally, you can think of any such expression as denoting a set of functions: x2+O(x) : {g | fO(x): g(x)= x2+f(x)}
CompSci 102
© Michael Frank

37. Order of Growth Equations
Suppose E1 and E2 are order-of-growth expressions corresponding to the sets of functions S and T, respectively.
Then the “equation” E1=E2 really means fS, gT : f=g or simply ST.
Example: x2 + O(x) = O(x2) means fO(x): gO(x2): x2+f(x)=g(x)
CompSci 102
© Michael Frank

38. Useful Facts about Big O
 f,g & constants a,bR, with b0,
af = O(f); (e.g. 3x2 = O(x2))
f+O(f) = O(f); (e.g. x2+x = O(x2))
Also, if f=(1) (at least order 1), then:
|f|1-b = O(f); (e.g. x1 = O(x))
(logb |f|)a = O(f). (e.g. log x = O(x))
g=O(fg) (e.g. x = O(x log x))
fg  O(g) (e.g. x log x  O(x))
a=O(f) (e.g. 3 = O(x))
CompSci 102
© Michael Frank

39. Definition: (g), exactly order g
If fO(g) and gO(f), then we say “g and f are of the same order” or “f is (exactly) order g” and write f(g).
Another, equivalent definition: (g)  {f:RR | c1c2k>0 x>k: |c1g(x)||f(x)||c2g(x)| }
“Everywhere beyond some point k, f(x) lies in between two multiples of g(x).”
CompSci 102
© Michael Frank

40. Mostly like rules for O( ), except:
 f,g>0 & constants a,bR, with b>0, af  (f), but  Same as with O. f  (fg) unless g=(1)  Unlike O. |f| 1-b  (f), and  Unlike with O. (logb |f|)c  (f)  Unlike with O.
The functions in the latter two cases we say are strictly of lower order than (f).
CompSci 102
© Michael Frank

41.  example Determine whether: Quick solution: CompSci 102
© Michael Frank

42. Other Order-of-Growth Relations
(g) = {f | gO(f)} “The functions that are at least order g.”
o(g) = {f | c>0 k x>k : |f(x)| < |cg(x)|} “The functions that are strictly lower order than g.” o(g)  O(g)  (g).
(g) = {f | c>0 k x>k : |cg(x)| < |f(x)|} “The functions that are strictly higher order than g.” (g)  (g)  (g).
CompSci 102
© Michael Frank

43. Relations Between the Relations
Subset relations between order-of-growth sets.
RR
O( f )
( f )
• f
o( f )
( f )
( f )
CompSci 102
© Michael Frank

44. A function that is O(x), but neither o(x) nor (x):
Why o(f)O(x)(x)
A function that is O(x), but neither o(x) nor (x):
CompSci 102
© Michael Frank

45. Strict Ordering of Functions
Temporarily let’s write fg to mean fo(g), f~g to mean f(g)
Note that:
Let k>1. Then the following are true: 1  log log n  log n ~ logk n  logk n  n1/k  n  n log n  nk  kn  n!  nn …
CompSci 102
© Michael Frank

46. Review: Orders of Growth (§1.8)
Definitions of order-of-growth sets, g:RR
O(g) : {f |  c>0 k x>k |f(x)| < |cg(x)|}
o(g) : {f | c>0 k x>k |f(x)| < |cg(x)|}
(g) : {f | gO(f)}
(g) : {f | go(f)}
(g) : O(g)  (g)
CompSci 102
© Michael Frank