Geoff Holmes and Bernhard Pfahringer COMP206-08S General Programming 2

Department of Computer Science2 Lectures - Today - Algorithm Analysis (Chapter 5 of Weiss)

Department of Computer Science3 Efficient Solutions An algorithm is efficient if:  Produces the desired solution  It is fast (low time complexity)  It consumes little memory (low space complexity) Algorithm analysis studies the time and space complexity of algorithms Mainly study time Appropriate data structures can reduce complexity

Department of Computer Science4 Basics Running time of an algorithm depends on the amount of data that needs to be processed We study the running time as a function of the size of the input Analysis of algorithms ignores:  Computing platform  Compiler  Quality of implementation

Department of Computer Science5 Growth Rates What matters for large input sizes is the growth rate  We can ignore constants and lower-order terms in polynomials The Big-Oh notation is used to capture the most dominant term  Used to represent the growth rate  Makes a mathematically sound statement

Department of Computer Science6 Functions and Growth FunctionName cConstant log NLogarithmic NLinearN log N N 2 Quadratic N 3 Cubic 2 N Exponential

Department of Computer Science7 Example Problems Consider  A = { -2, 11, -4, 13, -5, 2 } Problem: Find the maximum contiguous subsequence sum Solution: = 20

Department of Computer Science8 Solution 1 Brute force solution Compute all subsequences and save the one with the greatest sum Use 3 nested loops Simplest and most obvious solution (often the case) Complexity?

Department of Computer Science9 Solution 2 Remove innermost loop via observation that the sum can be done incrementally (rather than starting from scratch each time) This removes a loop so now we have only 2 Complexity?

Department of Computer Science10 Solution 3 Remove another loop via the observation that negative subsequences are never going to yield useful results Getting rid of loops like this is not always possible and most of the time gets harder as each loop is removed as with this problem

Department of Computer Science11 Big-Oh and Being Formal Big-Oh notation states that the running time T(N) is at most O(F(N)) where T(N) is within a constant of F(N) Big-Omega notation is used to express that T(N) grows at least as quickly as F(N) Big-Theta notation states that the growth rates of T(N) and F(N) are the same Little-Oh notation states that the growth rate of T(N) is strictly less than that of F(N)

Department of Computer Science12 Average and Worst-Case Actual runtime also depends on the type of input – not just size Worst-case bounds are based on the assumption of encountering the worst possible input  Eg Finding a person in phone book when you know their number (actual example would be last num in book!) Average-case bounds are based on the average or expected case  Usually harder to establish

Department of Computer Science13 Logarithms Many algorithms have bounds involving logs Two observations  Log (base B) N = K if B power K = N  Log (base B) N = O( log N) So base is irrelevant (we normally have B = 2)

Department of Computer Science14 Importance of Logs Number of bits used to represent N consecutive integers Gives the number of times we can double a number starting from 1 before we reach N (repeated doubling principle) – if I save $1 a year and double it how many years will it take to save $1,000,000? Repeated halving principle  Phonebook?

Department of Computer Science15 Searching Sequential search  O(N) comparisons in the worst and average case Binary search  O(log N) comparisons in the worst and average case (rep halving) Interpolation search  O(N) in worst-case  O(log log N) in average case

Department of Computer Science16 Limitations Big-Oh no good for small input (use simplest algorithm in this case) Constants can matter for some problems