 Last lesson  Arrays for implementing collection classes  Performance analysis (review)  Today  Performance analysis  Logarithm

Complexity Theory  Three different symbols will be discussed in class:  O (called the Big Oh)   (upper case Greek letter OMEGA)   (upper case Greek letter THETA)  Weiss, chapter 5

O (Big Oh)  The Big Oh is the upper bound of a function.  In the case of algorithm analysis, we use it to bound the worst- case running time, or the longest running time possible for any input of size n.  We can say that the maximum running time of the algorithm is in the order of Big Oh.  T(n) = O(f(n)) if there are positive constants c and n 0 such that T(n)  c*f(n) for all n  n 0

   is is the lower bound of a function.  We can say that the minimum running time of the algorithm is in the order of   T(n) =  (f(n)) if there are positive constants c and n 0 such that T(n)  c*f(n) for all n  n 0

   is composed of both O and .  T(n) =  (f(n)) if and only if T(n) is O(f(n)) and T(n) is  (f(n)) n T upper bound lower bound

A Few Remarks  We only care what happens for large n (n  n 0 )  O, ,  "hide" the constant c from us.  Find simple and small functions  E.g. (n 2 + 3n + 4) is O(n 3 ), since (n 2 + 3n + 4) 3, but it is also O(n 2 ), since (n 2 + 3n + 4) 10

The Logarithm  For any b, n>0: log b n=k if b k =n  We use log instead of log 2 (as in Weiss)  Be careful, often  lg instead of log 2  log instead of log 10  Properties of logarithms  log b (x*y) = log b x + log b y  log b (x/y) = log b x - log b y  log b x a = a*log b x  log b x= (log a x)/(log a b)  For complexity analysis the base does not matter  For any constant b>1: log b n = O(log n)

Repeated Doubling and Halving  Repeated doubling  We can repeatedly double only logarithmically many times until we reach n  Repeated halving  Starting at n, we can halve only logarithmically many times before we reach 1

Static Searching Problem  Given an integer num and an array arr, return the position of num in arr or an indication that it is not present. If num occurs more than once, return any occurrence. The array is never altered (is static).  Weiss, chapter “7”“7”“3”“3”“12”“42”“56”“75”“77”

4 Linear (Sequential) Search for (int i = 0; i <= last; i++) { if (arr[i] = num) { return i; }  Complexity  E.g. search for “56”  O(n) (linear) “7”“7”“3”“3”“12”“42”“56”“75”“77”

Linear (Sequential) Search for (int i = 0; i <= last; i++) { if (arr[i] = num) { return i; }  Complexity  E.g. search for “98”  O(n) (linear) “7”“7”“3”“3”“12”“42”“56”“75”“77”

Binary Search (1)  Array must be sorted  Routine to return index where element is equal to argument  Look in middle of occupied part of array  If that is the element we are looking for, we found it  Otherwise, decide whether the element must be in front half or back half (do recursive call to look there)  E. g. search for “56”  This is a divide-and-conquer algorithm “7”“7”“3”“3”“12”“42”“56”“75”“77”

Binary Search (1)  Array must be sorted  Routine to return index where element is equal to argument  Look in middle of occupied part of array  If that is the element we are looking for, we found it  Otherwise, decide whether the element must be in front half or back half (do recursive call to look there)  E. g. search for “56”  This is a divide-and-conquer algorithm “7”“7”“3”“3”“12”“42”“56”“75”“77”

Binary Search (1)  Array must be sorted  Routine to return index where element is equal to argument  Look in middle of occupied part of array  If that is the element we are looking for, we found it  Otherwise, decide whether the element must be in front half or back half (do recursive call to look there)  E. g. search for “56”  This is a divide-and-conquer algorithm “7”“7”“3”“3”“12”“42”“56”“75”“77”

Binary Search (1)  Array must be sorted  Routine to return index where element is equal to argument  Look in middle of occupied part of array  If that is the element we are looking for, we found it  Otherwise, decide whether the element must be in front half or back half (do recursive call to look there)  E. g. search for “56”  This is a divide-and-conquer algorithm “7”“7”“3”“3”“12”“42”“56”“75”“77”

Binary Search (1)  Array must be sorted  Routine to return index where element is equal to argument  Look in middle of occupied part of array  If that is the element we are looking for, we found it  Otherwise, decide whether the element must be in front half or back half (do recursive call to look there)  E. g. search for “56”  This is a divide-and-conquer algorithm “7”“7”“3”“3”“12”“42”“56”“75”“77”

Binary Search (2) “7”“7” “3”“3”“12” “42” “56” “75” “77”

Binary Search (3)  It can also be written as a loop (non recursive)  Warning: the idea (binary search) is simple, but the code is tricky to get right  Avoid infinite recursion or infinite loop  Treat boundary cases properly  Avoid looking at elements which are not in the sequence  Each time we deal with half as much of the array as the previous call (or iteration)  So we make log n calls or iterations  Each time we do const work besides the recursion  So the total cost is O(log n) (much faster than simple searching or searching in an unsorted array)

Code public static int binarySearch(Comparable [ ] a, Comparable x) { int low = 0; int high = a.length - 1; int mid; while(low <= high) { mid = (low + high) / 2; if(a[mid].compareTo(x) < 0) low = mid + 1; else if(a[mid].compareTo(x) > 0) high = mid - 1; else return mid; } return NOT_FOUND; // NOT_FOUND = -1 }

Code public static int binarySearch(Comparable [ ] a, Comparable x) { int low = 0; int high = a.length - 1; int mid; while(low <= high) { mid = (low + high) / 2; if(a[mid].compareTo(x) < 0) low = mid + 1; else if(a[mid].compareTo(x) > 0) high = mid - 1; else return mid; } return NOT_FOUND; // NOT_FOUND = -1 } lowhigh find 56

Code public static int binarySearch(Comparable [ ] a, Comparable x) { int low = 0; int high = a.length - 1; int mid; while(low <= high) { mid = (low + high) / 2; if(a[mid].compareTo(x) < 0) low = mid + 1; else if(a[mid].compareTo(x) > 0) high = mid - 1; else return mid; } return NOT_FOUND; // NOT_FOUND = -1 } lowhigh find 56

Code public static int binarySearch(Comparable [ ] a, Comparable x) { int low = 0; int high = a.length - 1; int mid; while(low <= high) { mid = (low + high) / 2; if(a[mid].compareTo(x) < 0) low = mid + 1; else if(a[mid].compareTo(x) > 0) high = mid - 1; else return mid; } return NOT_FOUND; // NOT_FOUND = -1 } lowhighmid find 56

public static int binarySearch(Comparable [ ] a, Comparable x) { int low = 0; int high = a.length - 1; int mid; while(low <= high) { mid = (low + high) / 2; if(a[mid].compareTo(x) < 0) low = mid + 1; else if(a[mid].compareTo(x) > 0) high = mid - 1; else return mid; } return NOT_FOUND; // NOT_FOUND = -1 } Code low highmid find 56

public static int binarySearch(Comparable [ ] a, Comparable x) { int low = 0; int high = a.length - 1; int mid; while(low <= high) { mid = (low + high) / 2; if(a[mid].compareTo(x) < 0) low = mid + 1; else if(a[mid].compareTo(x) > 0) high = mid - 1; else return mid; } return NOT_FOUND; // NOT_FOUND = -1 } Code low highmid find 56

public static int binarySearch(Comparable [ ] a, Comparable x) { int low = 0; int high = a.length - 1; int mid; while(low <= high) { mid = (low + high) / 2; if(a[mid].compareTo(x) < 0) low = mid + 1; else if(a[mid].compareTo(x) > 0) high = mid - 1; else return mid; } return NOT_FOUND; // NOT_FOUND = -1 } Code low highmid find 56

public static int binarySearch(Comparable [ ] a, Comparable x) { int low = 0; int high = a.length - 1; int mid; while(low <= high) { mid = (low + high) / 2; if(a[mid].compareTo(x) < 0) low = mid + 1; else if(a[mid].compareTo(x) > 0) high = mid - 1; else return mid; } return NOT_FOUND; // NOT_FOUND = -1 } Code low high mid find 56

public static int binarySearch(Comparable [ ] a, Comparable x) { int low = 0; int high = a.length - 1; int mid; while(low <= high) { mid = (low + high) / 2; if(a[mid].compareTo(x) < 0) low = mid + 1; else if(a[mid].compareTo(x) > 0) high = mid - 1; else return mid; } return NOT_FOUND; // NOT_FOUND = -1 } Code low high mid find 56

public static int binarySearch(Comparable [ ] a, Comparable x) { int low = 0; int high = a.length - 1; int mid; while(low <= high) { mid = (low + high) / 2; if(a[mid].compareTo(x) < 0) low = mid + 1; else if(a[mid].compareTo(x) > 0) high = mid - 1; else return mid; } return NOT_FOUND; // NOT_FOUND = -1 } Code low high mid find 56

public static int binarySearch(Comparable [ ] a, Comparable x) { int low = 0; int high = a.length - 1; int mid; while(low <= high) { mid = (low + high) / 2; if(a[mid].compareTo(x) < 0) low = mid + 1; else if(a[mid].compareTo(x) > 0) high = mid - 1; else return mid; } return NOT_FOUND; // NOT_FOUND = -1 } Code low high mid find 56

Code public static int binarySearch(Comparable [ ] a, Comparable x) { int low = 0; int high = a.length - 1; int mid; while(low <= high) { mid = (low + high) / 2; if(a[mid].compareTo(x) < 0) low = mid + 1; else if(a[mid].compareTo(x) > 0) high = mid - 1; else return mid; } return NOT_FOUND; // NOT_FOUND = -1 } lowhigh find 98

Code public static int binarySearch(Comparable [ ] a, Comparable x) { int low = 0; int high = a.length - 1; int mid; while(low <= high) { mid = (low + high) / 2; if(a[mid].compareTo(x) < 0) low = mid + 1; else if(a[mid].compareTo(x) > 0) high = mid - 1; else return mid; } return NOT_FOUND; // NOT_FOUND = -1 } lowhigh find 98

Code public static int binarySearch(Comparable [ ] a, Comparable x) { int low = 0; int high = a.length - 1; int mid; while(low <= high) { mid = (low + high) / 2; if(a[mid].compareTo(x) < 0) low = mid + 1; else if(a[mid].compareTo(x) > 0) high = mid - 1; else return mid; } return NOT_FOUND; // NOT_FOUND = -1 } lowhighmid find 98

public static int binarySearch(Comparable [ ] a, Comparable x) { int low = 0; int high = a.length - 1; int mid; while(low <= high) { mid = (low + high) / 2; if(a[mid].compareTo(x) < 0) low = mid + 1; else if(a[mid].compareTo(x) > 0) high = mid - 1; else return mid; } return NOT_FOUND; // NOT_FOUND = -1 } Code low highmid find 98

public static int binarySearch(Comparable [ ] a, Comparable x) { int low = 0; int high = a.length - 1; int mid; while(low <= high) { mid = (low + high) / 2; if(a[mid].compareTo(x) < 0) low = mid + 1; else if(a[mid].compareTo(x) > 0) high = mid - 1; else return mid; } return NOT_FOUND; // NOT_FOUND = -1 } Code low highmid find 98

public static int binarySearch(Comparable [ ] a, Comparable x) { int low = 0; int high = a.length - 1; int mid; while(low <= high) { mid = (low + high) / 2; if(a[mid].compareTo(x) < 0) low = mid + 1; else if(a[mid].compareTo(x) > 0) high = mid - 1; else return mid; } return NOT_FOUND; // NOT_FOUND = -1 } Code low highmid find 98

public static int binarySearch(Comparable [ ] a, Comparable x) { int low = 0; int high = a.length - 1; int mid; while(low <= high) { mid = (low + high) / 2; if(a[mid].compareTo(x) < 0) low = mid + 1; else if(a[mid].compareTo(x) > 0) high = mid - 1; else return mid; } return NOT_FOUND; // NOT_FOUND = -1 } Code low mid find 98 high

public static int binarySearch(Comparable [ ] a, Comparable x) { int low = 0; int high = a.length - 1; int mid; while(low <= high) { mid = (low + high) / 2; if(a[mid].compareTo(x) < 0) low = mid + 1; else if(a[mid].compareTo(x) > 0) high = mid - 1; else return mid; } return NOT_FOUND; // NOT_FOUND = -1 } Code find 98 midhigh low

public static int binarySearch(Comparable [ ] a, Comparable x) { int low = 0; int high = a.length - 1; int mid; while(low <= high) { mid = (low + high) / 2; if(a[mid].compareTo(x) < 0) low = mid + 1; else if(a[mid].compareTo(x) > 0) high = mid - 1; else return mid; } return NOT_FOUND; // NOT_FOUND = -1 } Code find 98 high low mid

public static int binarySearch(Comparable [ ] a, Comparable x) { int low = 0; int high = a.length - 1; int mid; while(low <= high) { mid = (low + high) / 2; if(a[mid].compareTo(x) < 0) low = mid + 1; else if(a[mid].compareTo(x) > 0) high = mid - 1; else return mid; } return NOT_FOUND; // NOT_FOUND = -1 } Code find 98 high low mid

public static int binarySearch(Comparable [ ] a, Comparable x) { int low = 0; int high = a.length - 1; int mid; while(low <= high) { mid = (low + high) / 2; if(a[mid].compareTo(x) < 0) low = mid + 1; else if(a[mid].compareTo(x) > 0) high = mid - 1; else return mid; } return NOT_FOUND; // NOT_FOUND = -1 } Code find 98 high low mid

Simplifying Rules  If f(n) is in O(g(n)) and g(n) is in O(h(n)), then f(n) is in O(h(n)).  If f(n) is in O(k*g(n)) for any constant k > 0, then f(n) is in O(g(n)).  If f 1 (n) is in O(g 1 (n)) and f 2 (n) is in O(g 2 (n)), then f 1 (n)+f 2 (n) is in max{O(g 1 (n)), O(g 2 (n))}.  If f 1 (n) is in O(g 1 (n)) and f 2 (n) is in O(g 2 (n)), then f 1 (n)*f 2 (n) is in O(g 1 (n)*g 2 (n)).

Analysing Control Statements  While loop  Analyse like a for loop  If statement  Take greater complexity of then/else clauses  Switch statement  Take complexity of most expensive case  Subroutine call  Complexity of the subroutine

Limitations of Big Oh  Not for small amount of input  Is designed for theoretical analysis  E.g. memory access is O(1), does not differentiate between main memory and disc  Constants may be to large for practical use  Worst case is sometimes uncommon an can be ignored

Recursion  This is a review of the main ideas  We expect some experience from first year  Read Weiss Sections

Key Ideas  A method includes one or more calls to the same method  The result of the nested calls is used to compute the result of the outer call  There is a base case  The call may be indirectly  Understand the method by assuming the nested calls work correctly  First, let us look at a non-programming example

Recursive Definition  Define a list of number separated by commas  Example: 23, 27, 31, 1  Like this  LIST is number  or LIST isnumber, LIST

Example  Try to see whether “23, 27, 31, 1” is a list  Apply definition LIST: number or LIST: number, LIST 23, 27, 31, 1 23, LIST 27, LIST 31, LIST 1

Erroneous Examples  4. 54, 23  D, 34, 2 LIST: number or LIST: number, LIST

Base Case  Notice that we had one case (LIST: number) that does no recursion  That is called the base case  Otherwise, we would have infinite recursion LIST: number or LIST: number, LIST 31, LIST 1 …

What About Programs?  We can solve a lot of mathematical problems recursively  E.g.  i=0 i = … + n public static int sum (int n) { if (n < 1) return 0; else return(sum(n – 1) + n); } n

Example public static int sum(int n) { if(n < 1) return 0; return(sum(n – 1) + n); } sum(4) sum(3) sum(2) sum(1) sum(0)

Progress  Notice that we must make progress toward base case  Sum(0)  There is steady reduction in size of problem  Otherwise, trouble

Iterative Solution  For this problem a recursive solution does not make sense  Elegant, but slower than the following solution public static int sum(int n) { int result = 0; for (int i = 1; i <= n; i++) result += i; return(n); }  The best solution public static int sum(int n) { return (n/2)*(n+1); }