Analysis of Algorithms: Methods and Examples CSE 2320 – Algorithms and Data Structures Vassilis Athitsos Modified by Alexandra Stefan University of Texas at Arlington 1

Counting instructions Count in detail the total number of instructions executed by the each of the following pieces of code: // Example A. Notice the ; at the end of the for loop. for (i = 0; i <N; i++) ; // Example B for (i = 0; i<N; i++) for(j=0; j<N; j++) for(k=0; k<N; k++) printf(“A”); // Example C (source: Dr. Bob Weems) sum = 0; for (i = 0; i<N; i++) for(j=1; j<N; j = j+j) sum = sum + a[i]/a[j];

Estimate runtime Problem: The total number of instructions in a program (or a piece of a program) is and the computer it runs on executes 10 9 instructions per second. How long will it take to run this program? Give the answer in seconds. If it is very large, transform it in larger units (hours, days, years). Summary: – Total instructions: – Speed: 10 9 instructions per second Answer: – Time = (total instructions)/speed = (10 12 instructions) / (10 9 instr/sec) = 10 3 seconds ~ 15 minutes Note that this computation is similar to computing the time it takes to travel a certain distance ( e.g. 120miles) given the speed (e.g. 60 miles/hour). 3

Estimate runtime A slightly different way to formulate the same problem: – total number of instructions in a program (or a piece of a program) is and – the computer it runs on executes one instruction in one nanosecond (10 -9 seconds) – How long will it take to run this program? Give the answer in seconds. If it is very large, transform it in larger units (hours, days, years) Summary: – total instructions – seconds per instruction Answer: – Time = (total instructions) * (seconds per instruction) = (10 12 instructions)* (10 -9 sec/instr) = 10 3 seconds ~ 15 minutes 4

5

Big-Oh Notation A function g(N) is said to be O(f(N)) if there exist constants c 0 and N 0 such that: g(N) ≤ c 0 f(N) for all N ≥ N 0. Typically, g(N) is the running time of an algorithm, in your favorite units, implementation, and machine. This can be a rather complicated function. In algorithmic analysis, we try to find a f(N) that is simple, and such that g(N) = O(f(N)). 6

Big-Oh Notation Upper bound: A function g(N) is said to be O(f(N)) if there exist constants c 0 and N 0 such that: g(N) ≤ c 0 f(N) for all N ≥ N 0. Lower bound: A function g(N) is said to be Ω(f(N)) if there exist constants c 0 and N 0 such that: c 0 f(N) ≤ g(N) for all N ≥ N 0. Tight bound: A function g(N) is said to be Θ(f(N)) if there exist constants c 0, c 1 and N 0 such that: c 0 f(N) ≤ g(N) ≤ c 1 f(N) for all N ≥ N 0. 7

Simplifying Big-Oh Notation Suppose that we are given this running time: g(N) = 35N N + lg(N) How can we express g(N) in Big-Oh notation? 8

Simplifying Big-Oh Notation Suppose that we are given this running time: g(N) = 35N N + lg(N) How can we express g(N) in Big-Oh notation? Typically we say that g(N) = O(N 2 ). The following are also correct, but unnecessarily complicated, and thus less useful, and rarely used. – g(N) = O(N 2 ) + O(N). – g(N) = O(N 2 ) + O(N) + O(lgN) + O(1). – g(N) = O(35N N + lg(N) ). 9

Simplifying Big-Oh Notation Suppose that we are given this running time: g(N) = 35N N + lg(N) We say that g(N) = O(N 2 ). Why is this mathematically correct? – Why can we ignore the non-quadratic terms? Ans 1: Using the Big-Oh definition: we can find an N 0 such that, for all N ≥ N 0 : g(N) ≤ 36N 2. – If you don't believe this, do the calculations for practice. 10

Polynomial functions If g(N) is a polynomial function, then it is Θ of the dominant term. 11

Properties of O, Ω and Θ 12

Using Limits 13

Using Limits 14

Using Limits: An Example 15

Using Limits: An Example 16

Using Limits: An Example 17

Big-Oh Transitivity 18

Big-Oh Hierarchy 19

Big-Oh Transitivity 20

Using Substitutions 21

Using Substitutions 22

Big-Oh Notation: Example Problem 23

Big-Oh Notation: Example Problem 24

Useful logarithm properties 25

Summary 26