Introduction to Analysing Costs COMP 103 Introduction to Analysing Costs Marcus Frean, Rashina Hoda, and Peter Andreae Thomas Kuehne School of Engineering and Computer Science, Victoria University of Wellington 2016-T2 Lecture 08

RECAP-TODAY RECAP Implementing the fundamental ArrayList methods 2 RECAP Implementing the fundamental ArrayList methods AbstractList  ArrayList does not have to implement everything TODAY introduction to analysing cost

Analysing Costs (in general) 3 How can we determine the costs of a program? Time: Run the program and count the milliseconds/minutes/days. Count number of steps/operations the algorithm will take. Space: Measure the amount of memory the program occupies. Count the number of elementary data items the algorithm stores. Applies to Programs or Algorithms? Both. programs: called “benchmarking” algorithms: called “analysis”

Benchmarking: program cost 4 Measure: actual programs, on real machines, with specific input measure elapsed time System.currentTimeMillis () → time from the system clock in milliseconds measure real memory Problems: what input? ⇒ use large data sets don’t include user input other users/processes? ⇒ minimise average over many runs which computer? ⇒ specify details how to compare cross-platform? ⇒ measure cost at an abstract level

Analysis: Algorithm “complexity” 5 Abstract away from the details of the hardware, the operating system the programming language, the compiler the specific input Measure number of “steps” as a function of the data size best case (easy, but not interesting) worst case (usually easy) average case (harder) The precise number of steps is not required 3.47 n2 - 67n + 53 steps 3n log(n) + 5n - 3 steps Rather, we are interested in the complexity class

Analysis: The Big Picture 6 cost f1(n) f1(n) = 0.7772n2 + 305n + 2 3,000,000 f2(n) = 1.2724n + 100000 2,000,000 1,000,000 f2(n) input size 125 250 500 1000 2000 Only care about with what shape cost grows with input size

Analysis: Notation Big-O Notation 7 Big-O Notation f(n) is O(g(n)), if there are two positive constants, s and n0 with f(n) £ s  | g(n) |, for all n ³ n0 cost s  g(n) 3000 2000 f(n) 1000 input size 125 250 500 n0 1000 2000

Big-O Notation 8 Only care for large input sets and ignore constant factors 3.47 n2 + 102n + 10064 steps  O(n2) 3n log n + 12n steps  O(n log n) Lower-order terms become insignificant for large n Multiplication factors don’t tell us how things “scale up” We care about how cost grows with input size

Big-O classes 9 “Asymptotic cost”, or “big-O” cost describes how cost grows with input size Examples: O(1) constant cost is independent of n : Fixed cost! O(log n) logarithmic cost grows up by 1, every time n doubles : Slow growth! O(n) linear cost grows linearly with n : can often be beaten O(n2) quadratic costs grows like n  n : severely limits problem size

Big-O costs Cost input size 10 N O(1) Constant O(log N) Logarithmic Linear O(N log N) O(N^2) Quadratic O(N^3) Cubic O(N!) Factorial 1 5 0.00 2.32 11.61 25 125 120 10 3.32 33.22 100 1000 3628800 15 3.91 58.60 225 3375 1.31E+12 20 4.32 86.44 400 8000 2.43E+18 O(n3) O(n2) Cost O(n log n) O(n) O(log n) O(1) input size

Manageable Problem Sizes 11 N 1 min 1 h 1 day 1 week 1 year n 6 ´ 107 3.6 ´ 109 8.64 ´ 1010 6.05 ´ 1011 3.15´ 1013 n log n 2.8 ´ 106 1.3 ´ 108 2.75 ´ 109 1.77 ´ 1010 7.97 ´ 1011 n2 7.75 ´ 103 6.0 ´ 104 2.94 ´ 105 7.78 ´ 105 5.62 ´ 106 n3 3.91 ´ 102 1.53 ´ 13 4.42 ´ 103 8.46 ´ 103 3.16 ´ 104 2n 25 31 36 39 44 High asymptotic cost severely limits problem size

What is a “step”? Of importance comparing data moving data 12 Of importance comparing data moving data anything you consider to be “expensive” public E remove (int index){     if (index < 0 || index >= count) throw new ….Exception();     E ans = data[index];     for (int i=index+1; i< count; i++)       data[i-1]=data[i];         count--; data[count] = null;     return ans;   } ← Key Step

ArrayList: get, set, remove 13 Assume an ArrayList contains n items. Cost of get and set: best, worst, average: Cost of Remove: worst case: what is the worst case? how many steps? average case: what is the average case? n

ArrayList: add (at some position) 14 public void add(int index, E item){ if (index<0 || index>count) throw new IndexOutOfBoundsException(); ensureCapacity(); for (int i=count; i > index; i--)       data[i]=data[i-1]; data[index]=item; count++; }   Cost of add(index, value): key steps? n

ArrayList: add at end Cost of add(value): which are the key steps? 15 Cost of add(value): worst case: average case: public void add (E item) {     ensureCapacity();     data[count++] = item; } private void ensureCapacity () {     if (count < data.length) return;     E [ ] newArray = (E[ ]) (new Object[data.length * 2]);     for (int i = 0; i < count; i++)       newArray[i] = data[i];     data = newArray; }  n which are the key steps?

ArrayList: Amortised cost 16 Some single “adds” are expensive, but what is the big picture? “Amortised” cost: total cost of adding n items, divided by n: first 10: cost = 1 each total = 10 11th: cost = 10+1 total = 21 12-20: cost = 1 each total = 30 21st: cost = 20+1 total = 51 22-40: cost = 1 each total = 70 41st: cost = 40+1 total = 111 42-80: cost = 1 each total = 150 : - n total = Amortised cost ( ) = ? per item

ArrayList: Amortised cost 17 Starting with array length = 1 makes it easier to see the pattern first : cost = 1 each total = 1 2nd: cost = 1+1 total = 3 3rd: cost = 2 + 1 total = 6 4th: cost = 1 each total = 7 5th: cost = 4 + 1 total = 12 6-8: cost = 1 each total = 15 9th: cost = 8 + 1 total = 24 10-16: cost = 1 each total = 31 : n total = Amortised cost ( ) = look at pattern to right (1,3,7,15,31) -> 2^x -1 OR calculate sum i=0..k (2^i) = 2^(k+1)-1 how many copy steps are there? n = 2^steps = 2^k => k=log_2 n now insert steps into count formular steps = 2^(log2(n)+1)-1 -> n -> like previous example with a factor of ten finally: divide by n to achieve amortised cost. per item

ArrayList costs: Summary 18 get O(1) set O(1) remove O(n) add (at i) O(n) (worst and average) add (at end) O(1) (average) O(n) (worst) O(1) (amortised average) To think about: What would the amortised cost be if the array size were increased by a fixed amount (say 10) each time?