1. Week 12 - Wednesday

2.  What did we talk about last time?  Hunters and prey  Class variables  Big Oh notation

6.  We want to compare the running time of one program to another  We want a mathematical description with the following characteristics:  Worst case We care mostly about how bad things could be  Asymptotic We focus on the behavior as the input size gets larger and larger

7.  Enter Big Oh notation  Big Oh simplifies a complicated running time function into a simple statement about its worst case growth rate  All constant coefficients are ignored  All low order terms are ignored  3n + 3 is O(n)  Big Oh is a statement that a particular running time is no worse than some function, with appropriate constants

8.  147n 3 + 2n 2 + 5n + 12083 is O(n3)O(n3)  n 1000 + 2 n is O(2 n )  15n 2 + 6n + 7log n + 145 is O(n2)O(n2)  659n + nlog n + 87829 is O(n log n)  Note: In CS, we use log 2 unless stated otherwise

10.  How long does it take to do multiplication by hand? 123 x 456 738 615 492__ 56088  Let’s assume that the length of the numbers is n digits  (n multiplications + n carries) x n digits + (n + 1 digits) x n additions  Running time: O(n 2 )

11.  How do we find the largest element in an array?  Running time: O(n) if n is the length of the array  What if the array is sorted in ascending order?  Running time: O(1) int largest = array[0]; for( int i = 1; i < array.length; i++ ) if( array[i] > largest ) largest = array[i]; System.out.println("Largest: " + largest); int largest = array[0]; for( int i = 1; i < array.length; i++ ) if( array[i] > largest ) largest = array[i]; System.out.println("Largest: " + largest); System.out.println("Largest: " + array[array.length-1]);

12.  Here is some code that sorts an array in ascending order  What is its running time?  Running time: O(n 2 ) for( int i = 0; i < array.length; i++ ) for( int j = 0; j < array.length - 1; j++ ) if( array[j] > array[j + 1] ) { int temp = array[j]; array[j] = array[j + 1]; array[j + 1] = temp; } for( int i = 0; i < array.length; i++ ) for( int j = 0; j < array.length - 1; j++ ) if( array[j] > array[j + 1] ) { int temp = array[j]; array[j] = array[j + 1]; array[j + 1] = temp; }

14.  Here is a table of several different complexity measures, in ascending order, with their functions evaluated at n = 100 DescriptionBig Ohf(100) ConstantO(1)1 LogarithmicO(log n)6.64 LinearO(n)O(n)100 LinearithmicO(n log n)664.39 QuadraticO(n2)O(n2)10000 CubicO(n3)O(n3)1000000 ExponentialO(2 n )1.27 x 10 30 FactorialO(n!)9.33 x 10 157

15.  Computers get faster, but not in unlimited ways  If computers get 10 times faster, here is how much a problem from each class could grow and still be solvable DescriptionBig OhIncrease in Size ConstantO(1)Unlimited LogarithmicO(log n)1000 LinearO(n)O(n)10 LinearithmicO(n log n)10 QuadraticO(n2)O(n2)3-4 CubicO(n3)O(n3)2-3 ExponentialO(2 n )Hardly changes FactorialO(n!)Hardly changes

16.  There is nothing better than constant time  Logarithmic time means that the problem can become much larger and only take a little longer  Linear time means that time grows with the problem  Linearithmic time is just a little worse than linear  Quadratic time means that expanding the problem size significantly could make it impractical  Cubic time is about the reasonable maximum if we expect the problem to grow  Exponential and factorial time mean that we cannot solve anything but the most trivial problem instances

18.  Memory usage can be a problem  If you run out of memory, your program can crash  Memory usage can have serious performance consequences too

19.  Remember, there are multiple levels of memory on a computer  Each next level is on the order of 500 times larger and 500 times slower Cache Actually on the CPU Fast and expensive RAM Primary memory for a desktop computer Pretty fast and relatively expensive Hard Drive Secondary memory for a desktop computer Slow and cheap 1000X SizeSpeed

20.  If you can do a lot of number crunching without leaving cache, that will be very fast  If you have to fetch data from RAM, that will slow things down  If you have to read and write data to the hard drive (unavoidable with large pieces of data like digital movies), you will slow things down a lot

21.  Memory can be easier to estimate than running time  Depending on your input, you will allocate a certain number of objects, arrays, and primitive data types  It is possible to count the storage for each item allocated  Remember that a reference to an object or an array costs an additional 4 bytes on top of the size of the object

22.  Here are the sizes of various types in Java  Note that the N refers to the number of elements in the array or String TypeBytes boolean 1 char 2 int 4 double 8 TypeBytes boolean[] 16 + N char[] 16 + 2N int[] 16 + 4N double[] 16 + 8N TypeBytes reference4 String 40 + 2N object8 + size of members array of objects16 + (4 + size of members)N

24.  Lets say that I give you a list of numbers, and I ask you, “Is 37 on this list?”  As a human, you have no problem answering this question, as long as the list is reasonably short  What if the list is an array, and I want you to write a Java program to find some number?

25.  Easy!  We just look through every element in the array until we find it or run out  If we find it, we return the index, otherwise we return -1 public static int find( int[] array, int number ) { for( int i = 0; i < array.length; i++ ) if( array[i] == number ) return i; return -1; } public static int find( int[] array, int number ) { for( int i = 0; i < array.length; i++ ) if( array[i] == number ) return i; return -1; }

26.  Unfortunately for you, we know about Big Oh notation  Now we have some way to measure how long this algorithm takes  How long, if n is the length of the array?  O(n) time because we have to look through every element in the array, in the worst case

27.  Is there any way to go smaller than O(n)?  What complexity classes even exist that are smaller than O(n)?  O(1)  O(log n)  Well, on average, we only need to check half the numbers, that’s ½ n which is still O(n)  Darn…

29.  We can do better with more information  For example, if the list is sorted, then we can use that information somehow  How?  We can play a High-Low game

30.  Repeatedly divide the search space in half  We’re looking for 37, let’s say 542331 Check the middle (Too high) Check the middle (Too low) Check the middle (Too low) Check the middle (Found it!) 37

31.  How long can it take?  What if you never find what you’re looking for?  Well, then, you’ve narrowed it down to a single spot in the array that doesn’t have what you want  And what’s the maximum amount of time that could have taken?

34.  We can apply this idea to a guessing game  First we tell the computer that we are going to pick a number between 1 and n  We pick, and it tries to narrow down the number  It should only take log n tries  Remember log 2 (1,000,000) is only about 20

35.  This is a classic interview question asked by Microsoft, Amazon, and similar companies  Imagine that you have 9 red balls  One of them is just slightly heavier than the others, but so slightly that you can’t feel it  You have a very accurate two pan balance you can use to compare balls  Find the heaviest ball in the smallest number of weighings

36.  It’s got to be 8 or fewer  We could easily test one ball against every other ball  There must be some cleverer way to divide them up  Something that is related somehow to binary search

37.  We can divide the balls in half each time  If those all balance, it must be the one we left out to begin with

38.  How?  They key is that you can actually cut the number of balls into three parts each time  We weigh 3 against 3, if they balance, then we know the 3 left out have the heavy ball  When it’s down to 3, weigh 1 against 1, again knowing that it’s the one left out that’s heavy if they balance

39.  The cool thing is…  Yes, this is “cool” in the CS sense, not in the real sense  Anyway, the cool thing is that we are trisecting the search space each time  This means that it takes log 3 n weighings to find the heaviest ball  We could do 27 balls in 3 weighings, 81 balls in 4 weighings, etc.

42.  Sorting  Lab 12

43.  Finish Project 4  Due Friday before midnight