Algorithm Efficiency Chapter 10

What Is a Good Solution? A program incurs a real and tangible cost. Computing time Memory required Difficulties encountered by users Consequences of incorrect actions by program A solution is good if … The total cost incurred over all phases of its life … is minimal

What Is a Good Solution? Important elements of the solution Good structure Good documentation Efficiency Be concerned with efficiency when Developing underlying algorithm Choice of objects and design of interaction between those objects

Measuring Efficiency of Algorithms Important because Choice of algorithm has significant impact Examples Responsive word processors Grocery checkout systems Automatic teller machines Video/Gaming machines Life support systems E-Commerce web sites

Measuring Efficiency of Algorithms Analysis of algorithms The area of computer science that provides tools for contrasting efficiency of different algorithms Comparison of algorithms should focus on significant differences in efficiency (order of magnitude differences as input size increases) We consider comparisons of algorithms, not programs How do we measure efficiency Space utilization – amount of memory required Time required to accomplish the task Time efficiency depends on : size of input and for some algorithms, input order speed of machine quality of source code quality of compiler These vary from one platform to another

Measuring Efficiency of Algorithms Difficulties with comparing programs (instead of algorithms) How are the algorithms coded What computer will be used What data should the program use Algorithm analysis should be independent of Specific implementations, computers, and data We can count the number of times instructions are executed This gives us a measure of efficiency of an algorithm So we measure computing time as: f(n) = computing time of an algorithm for input of size n = number of times the instructions are executed

The Execution Time of Algorithms An algorithm’s execution time is related to number of operations it requires. Example: Towers of Hanoi Solution for n disks required 2n – 1 moves If each move requires time m Solution requires (2n – 1)  m time units

Example: Calculating the Mean Task # times executed Initialize the sum to 0 1 Initialize index i to 0 1 While i < n do following n+1 a) Add x[i] to sum n b) Increment i by 1 n Return mean = sum/n 1 Total f(n) = 3n + 4

Computing Time Order of Magnitude As number of inputs increases f(n) = 3n + 4 grows at a rate proportional to n Thus f(n) has the "order of magnitude" n The computing time of an algorithm on input of size n, f(n) said to have order of magnitude g(n), Written as f(n) is O(g(n)) Defined as, f(n) is O(g(n)) iff there exist positive constants, C and N0 such that 0 < f(n) < Cg(n) for all n ≥ N0

Big Oh Notation Another way of saying this: The complexity of the algorithm is O(g(n)). Example: For the Mean-Calculation Algorithm: f(n) is O(n) Note that constants and multiplicative factors are ignored. f(x) ∈ O(g(x)) as there exists c > 0 (e.g., c = 1) and N0 (e.g., N0 = 5) such that f(x) ≤ cg(x) whenever N ≥ N0. Here, x is n and y is Time.

Algorithm Growth Rates Measure an algorithm’s time requirement as function of problem size Most important thing to learn How quickly algorithm’s time requirement grows as a function of problem size Demonstrates contrast in growth rates

Big Oh Notation g(n) is usually simple: n, n2, n3, ... 2n 1, log2n n log2n log2log2n Note graph of common computing times

Big Oh Notation Graphs of common computing times

Algorithm Growth Rates Time requirements as a function of problem size n

Analysis and Big O Notation The graphs of 3  n2 and n2 - 3  n + 10

Analysis and Big O Notation Order of growth of some common functions

Common Computing Time Functions log2log2n log2n n n log2n n2 n3 2n --- 1 2 0.00 4 8 1.00 16 64 1.58 3 24 512 256 2.00 4096 65536 2.32 5 32 160 1024 32768 4294967296 2.58 6 384 262144 1.84467E+19 3.00 2048 16777216 1.15792E+77 3.32 10 10240 1048576 1.07E+09 1.8E+308 4.32 20 20971520 1.1E+12 1.15E+18 6.7E+315652

Analysis and Big O Notation Worst-case analysis Worst case analysis usually considered Easier to calculate, thus more common Average-case analysis More difficult to perform Must determine relative probabilities of encountering problems of given size

Computing in Real Time Suppose each instruction can be done in 1 microsecond For n = 256 instructions how long for various f(n) Function Time log2log2n 3 microseconds Log2n 8 microseconds n .25 milliseconds n log2n 2 milliseconds n2 65 milliseconds n3 17 seconds 2n 3.7+E64 centuries!!

Keeping Your Perspective ADT used makes a difference Array-based getEntry is O(1) Link-based getEntry is O(n) Choosing implementation of ADT Consider how frequently certain operations will occur Seldom used but critical operations must also be efficient

Keeping Your Perspective If problem size is always small Possible to ignore algorithm’s efficiency Weigh trade-offs between Algorithm’s time and memory requirements Compare algorithms for style and efficiency

Efficiency of Searching Algorithms Sequential search Worst case: O(n) Average case: O(n) Best case: O(1) Binary search O(log2n) in worst case Average case: O(log2n) At same time, maintaining array in sorted order requires overhead cost … can be substantial

Sequential Search Algorithm linear search(x : integer; a1,...,an : distinct integers) i = 1 while (i  n and x  ai) i = i + 1 if i  n then location = i else location = 0 {location is the index(subscript) of the term that equals x, or 0 if x is not found} Give f(n) for Sequential Search? Count operations. Give worst case g(n) for Sequential Search? What is the average case for Sequential Search? On average where do we find x in the array, a? Give average case O(g(n)) for Sequential Search?

Seq. Search Average Case Analysis E[X] = S (x * Pr{X=x}), assume uniform probability distribution, here read E[X] as expectation (average) of the discrete variable X. X is a function that maps elements of a sample space to the real numbers. For Sequential Search, the sample space is the finite set of comparisons required to find the key. We assume the key is located in the array, a. Comparisons(x) Pr{X=x} x* Pr{X=x} 1 1/n 1/n 2 1/n 2/n 3 1/n 3/n 4 1/n 4/n 5 1/n 5/n Sum = 1/n(S i) for 6 1/n 6/n 1 ≤ i ≤ n 7 1/n 7/n = (n(n+1))/2n . . . = (n+1)/2 . . . On average, find key at middle n 1/n n/n

Binary Search Algorithm binary search(x : integer; a1,...,an : increasing integers) i = 1 { i is the left endpoint} j = n { j is the right endpoint} while i < j m = (i+j)/2 if x>am then i=m+1 else j = m if x=ai then location = i else location = -1 {location is the index(subscript) of the term that equals x, or -1 if x is not found} Give f(n) for Binary Search? Give worst case g(n) for Binary Search What is the average case for Binary Search? On average where do we find x in the array, a? Give average case O(g(n)) for Binary Search?

Binary Search Average Case Analysis E[X] = S (x * Pr{X=x}), here read E[X] as expectation (average) of the discrete variable X. X is a function that maps elements of a sample space to the real numbers. For Binary Search, the sample space is the finite set of comparisons required to find the key. We assume the key is located in the array, a. Comparisons(x) Pr{X=x} x* Pr{X=x} You fill in the table? On average, where should we find the key in the array?

Examples Assuming a linked list of n nodes, the statements Node *cur = head; while (cur != null) { cout << curr->item << endl; cur = cur->next; } // end while require ______ assignment(s). The code segment = O(___)? Consider an algorithm that contains loops of this form: for (i = 1 through n) for (j = 1 through i) for (k = 1 through 10) Task T If task T requires t time units, the innermost loop on k requires ___ time units. The middle loop on j requires ___ time units. The code segment = O(____)?

Examples Order the following functions from smallest growth rate to largest. n2 n 2n log2 n Use Big-O notation to specify the asymptotic run-time of the following code segments. Assume variables a and b are unsigned ints. while (a != 0) { cout <<a<<" "; a /=2; } = O(___)? If(a > b) cout<<a<<endl; else cout<<b<<endl; = O(___)?

End Chapter 10