Summer 2007CISC121 - Prof. McLeod1 CISC121 – Lecture 9 Last time: –Finished Linked Lists –Stacks & Queues.

Slides:



Advertisements
Similar presentations
Introduction to Algorithms Quicksort
Advertisements

MATH 224 – Discrete Mathematics
CSE 373: Data Structures and Algorithms Lecture 5: Math Review/Asymptotic Analysis III 1.
the fourth iteration of this loop is shown here
Fall 2006CENG 7071 Algorithm Analysis. Fall 2006CENG 7072 Algorithmic Performance There are two aspects of algorithmic performance: Time Instructions.
CSC401 – Analysis of Algorithms Lecture Notes 1 Introduction
© 2004 Goodrich, Tamassia 1 Lecture 01 Algorithm Analysis Topics Basic concepts Theoretical Analysis Concept of big-oh Choose lower order algorithms Relatives.
Lecture 25 Selection sort, reviewed Insertion sort, reviewed Merge sort Running time of merge sort, 2 ways to look at it Quicksort Course evaluations.
Scott Grissom, copyright 2004 Chapter 5 Slide 1 Analysis of Algorithms (Ch 5) Chapter 5 focuses on: algorithm analysis searching algorithms sorting algorithms.
Simple Sorting Algorithms
Complexity Analysis (Part I)
Analysis of Algorithms (Chapter 4)
Analysis of Algorithms1 Estimate the running time Estimate the memory space required. Time and space depend on the input size.
Analysis of Algorithms
Cmpt-225 Algorithm Efficiency.
Lecture 3 Aug 31, 2011 Goals: Chapter 2 (algorithm analysis) Examples: Selection sorting rules for algorithm analysis discussion of lab – permutation generation.
Algorithm Analysis CS 201 Fundamental Structures of Computer Science.
Lecture 3 Feb 7, 2011 Goals: Chapter 2 (algorithm analysis) Examples: Selection sorting rules for algorithm analysis Image representation Image processing.
Analysis of Algorithms COMP171 Fall Analysis of Algorithms / Slide 2 Introduction * What is Algorithm? n a clearly specified set of simple instructions.
Cmpt-225 Simulation. Application: Simulation Simulation  A technique for modeling the behavior of both natural and human-made systems  Goal Generate.
CS 106 Introduction to Computer Science I 10 / 16 / 2006 Instructor: Michael Eckmann.
Analysis of Algorithms Dilemma: you have two (or more) methods to solve problem, how to choose the BEST? One approach: implement each algorithm in C, test.
CHAPTER 7: SORTING & SEARCHING Introduction to Computer Science Using Ruby (c) Ophir Frieder at al 2012.
Algorithm Analysis & Complexity We saw that a linear search used n comparisons in the worst case (for an array of size n) and binary search had logn comparisons.
Analysis of Algorithm Lecture 3 Recurrence, control structure and few examples (Part 1) Huma Ayub (Assistant Professor) Department of Software Engineering.
Week 2 CS 361: Advanced Data Structures and Algorithms
Analysis CS 367 – Introduction to Data Structures.
SEARCHING, SORTING, AND ASYMPTOTIC COMPLEXITY Lecture 12 CS2110 – Fall 2009.
1 Recursion Algorithm Analysis Standard Algorithms Chapter 7.
Data Structures and Algorithms Lecture 5 and 6 Instructor: Quratulain Date: 15 th and 18 th September, 2009 Faculty of Computer Science, IBA.
Analysis of Algorithms1 Running Time Pseudo-Code Analysis of Algorithms Asymptotic Notation Asymptotic Analysis Mathematical facts.
Iterative Algorithm Analysis & Asymptotic Notations
Algorithm Analysis An algorithm is a clearly specified set of simple instructions to be followed to solve a problem. Three questions for algorithm analysis.
Analysis of Algorithms
1 COMP3040 Tutorial 1 Analysis of algorithms. 2 Outline Motivation Analysis of algorithms Examples Practice questions.
C++ Programming: From Problem Analysis to Program Design, Second Edition Chapter 19: Searching and Sorting.
Analysis of Algorithms These slides are a modified version of the slides used by Prof. Eltabakh in his offering of CS2223 in D term 2013.
Recursion Recursion Chapter 12. Outline n What is recursion n Recursive algorithms with simple variables n Recursion and the run-time stack n Recursion.
Week 12 - Wednesday.  What did we talk about last time?  Asymptotic notation.
SortingBigOh Sorting and "Big Oh" Adapted for ASFA from a presentation by: Barb Ericson Georgia Tech Aug 2007 ASFA AP Computer Science.
Program Efficiency & Complexity Analysis. Algorithm Review An algorithm is a definite procedure for solving a problem in finite number of steps Algorithm.
Data Structure Introduction.
Java Methods Big-O Analysis of Algorithms Object-Oriented Programming
Introduction to Analysis of Algorithms CS342 S2004.
3 – SIMPLE SORTING ALGORITHMS
Review 1 Selection Sort Selection Sort Algorithm Time Complexity Best case Average case Worst case Examples.
Algorithm Analysis Part of slides are borrowed from UST.
Analysis of algorithms. What are we going to learn? Need to say that some algorithms are “better” than others Criteria for evaluation Structure of programs.
1/6/20161 CS 3343: Analysis of Algorithms Lecture 2: Asymptotic Notations.
Algorithm Analysis (Big O)
Searching and Sorting Searching: Sequential, Binary Sorting: Selection, Insertion, Shell.
Winter 2006CISC121 - Prof. McLeod1 Stuff No stuff today!
Spring 2006CISC101 - Prof. McLeod1 Last Time The File class. Back to methods –Passing parameters by value and by reference. –Review class attributes. –An.
Spring 2006CISC101 - Prof. McLeod1 Announcements Assn 4 is posted. Note that due date is the 12 th (Monday) at 7pm. (Last assignment!) Final Exam on June.
Vishnu Kotrajaras, PhD.1 Data Structures
1 Chapter 2 Algorithm Analysis Reading: Chapter 2.
Chapter 15 Running Time Analysis. Topics Orders of Magnitude and Big-Oh Notation Running Time Analysis of Algorithms –Counting Statements –Evaluating.
1 COMP9024: Data Structures and Algorithms Week Two: Analysis of Algorithms Hui Wu Session 2, 2014
Algorithmic Foundations COMP108 COMP108 Algorithmic Foundations Algorithm efficiency Prudence Wong.
Winter 2016CISC101 - Prof. McLeod1 CISC101 Reminders Assignment 5 is posted. Exercise 8 is very similar to what you will be doing with assignment 5. Exam.
Algorithmic Foundations COMP108 COMP108 Algorithmic Foundations Algorithm efficiency Prudence Wong
Algorithm Analysis 1.
Analysis of Algorithms
Algorithm design and Analysis
Winter 2018 CISC101 11/19/2018 CISC101 Reminders
Winter 2018 CISC101 12/2/2018 CISC101 Reminders
CS 201 Fundamental Structures of Computer Science
Searching, Sorting, and Asymptotic Complexity
Algorithm Analysis How can we demonstrate that one algorithm is superior to another without being misled by any of the following problems: Special cases.
Presentation transcript:

Summer 2007CISC121 - Prof. McLeod1 CISC121 – Lecture 9 Last time: –Finished Linked Lists –Stacks & Queues

Summer 2007CISC121 - Prof. McLeod2 You Will Need To: Work on exercise 4 and assignment 4. Assn 4 is due Saturday at 5pm. If you hand it in on Friday instead, Krista will have it marked earlier.

Summer 2007CISC121 - Prof. McLeod3 Today Analysis of Complexity Sorting –Insertion –Selection –Bubble Searching –Sequential –Binary Start Recursion?

Summer 2007CISC121 - Prof. McLeod4 Comparing Programs Possible bases for comparison: –Speed of execution –Memory consumption –Ease of debugging –Ease of interfacing with other program elements –Backwards compatibility –Quality of external documentation, support –Ease of use –Quality of user interface –Cost…

Summer 2007CISC121 - Prof. McLeod5 Comparing Algorithms, Cont. Nobody will buy a program that is too slow! –Regardless of all its other “features”. Too much work to finish an entire program only to find out it is too slow! It would be nice if we could predict speed of operation, or at least predict bottlenecks as code is developed.

Summer 2007CISC121 - Prof. McLeod6 Comparing Algorithms, Cont. For example, it was fairly obvious that deleting the tail node in a doubly linked list is much faster than deleting the tail node in a singly linked list. It is not always this easy to compare code on the basis of expected execution speed. The total time of execution is sum of many complex processor and memory operations.

Summer 2007CISC121 - Prof. McLeod7 The Analysis of Complexity Is really all about simplifying complexity. Complexity is what really goes on when a program is run, the accumulation of all the nano- second operations that take place. All we really need to do is to be able to say is that algorithm “A” will be faster than algorithm “B” if they are both tested under the same conditions with the same kind of data.

Summer 2007CISC121 - Prof. McLeod8 Analysis of Complexity - Cont. This theoretical technique can be explained by first generalizing the complexity of what goes on when code is run. First, define a “Primitive Operation”:

Summer 2007CISC121 - Prof. McLeod9 Primitive Operations A primitive operation takes a unit of time (a “tick”). The actual length of time will depend on external factors such as the hardware and software environment. Select operations that would all take about the same length of time. For example: –Assigning a value to a variable. –Calling a method. –Performing an arithmetic operation. –Comparing two numbers. –Indexing an array element. –Following an object reference. –Returning from a method.

Summer 2007CISC121 - Prof. McLeod10 Primitive Operations - Cont. Assume: Each of these kinds of operations will take the same amount of time on a given hardware and software platform. Count primitive operations for a simple method. Example - find the highest integer in an array:

Summer 2007CISC121 - Prof. McLeod11 public static int arrayMax(int[] A) { int currentMax = A[0];// 3 operations int n = A.length; // 3 operations for (int i = 0; i < n; i++)// 2+3n if (currentMax < A[i]) // 3n currentMax = A[i]; // 0 to 2n return currentMax; // 1 } // end arrayMax Number of primitive operations: –Minimum = n+3n+1= 9 + 6n –Maximum = 9 + 8n Primitive Operations - Cont.

Summer 2007CISC121 - Prof. McLeod12 Primitive Operations - Cont. Or, Best case is 9 + 6n, Worst case is 9 + 8n. What is the Average case? If the maximum element had exactly the same probability of being in each of the array locations then the average case would be = 9 + 7n. But if the data is not randomly distributed, then getting the average case is very difficult. It is much easier to just assume the worst case analysis, 9 + 8n.

Summer 2007CISC121 - Prof. McLeod13 Analysis of Complexity - Cont. Consider the case when n gets very large (Asymptotic Analysis). Then 8n is much greater than 9, so just using 8n is a good approximation. Or, it is even easier to say that the running time grows proportionally to n. Or the order is to the first power of n, or just first order, or O(n). For the most simple analysis, it makes sense to just ignore the constant values in the equation.

Summer 2007CISC121 - Prof. McLeod14 “Big O” Notation Mathematical definition of Big O Notation: Let f(n) and g(n) be functions that map nonnegative integers to real numbers. We say that f(n) is O(g(n)) if there is a real constant, c, where c > 0 and an integer constant n 0, where n 0  1 such that f(n)  cg(n) for every integer n  n 0.

Summer 2007CISC121 - Prof. McLeod15 “Big O” Notation - Cont.

Summer 2007CISC121 - Prof. McLeod16 “Big O” Notation - Cont. f(n) is the function that describes the actual time of the program as a function of the size of the dataset, n. The idea here is that g(n) is a much simpler function that what f(n) is. Given all the assumptions and approximations made to get any kind of a function, it does not make sense to use anything more than the simplest representation. The more complicated the function, the harder they are to compare.

Summer 2007CISC121 - Prof. McLeod17 “Big O” Notation - Cont. For example, consider the function: f(n) = n n + log 10 n For small values of n the last term, 1000, dominates. When n is around 10, the terms 100n dominate. When n is around 100, the terms n 2 and 100n dominate. When n gets much larger than 100, the n 2 dominates all others. The log 10 n term never gets to play at all! So it would be safe to say that this function is O(n 2 ) for values of n > 100.

Summer 2007CISC121 - Prof. McLeod18 “Big O” Notation - Cont. Similarly a function like: 20n n(log(n)) + 5 is O(n 3 ). (It can be proved that the function is  35n 3 for n  1. So c is 35 and n 0 is 1.)

Summer 2007CISC121 - Prof. McLeod19 “Big O” Notation - Cont. A question: If an algorithm is O(n 2 ), why is it not also O(n 3 ) or generally O(n i ) where i>2? The answer: It is! O(n 2 ) is just a subset of O(n i ). However, O(n 2 ) is probably the closest representation of the algorithm. While other notations satisfy the definition of “big O” they would lie further away from the algorithm’s actual behaviour, and would not do as good a job at describing the algorithm.

Summer 2007CISC121 - Prof. McLeod20 “Big O” Notation - Cont. So, use the the simplest notation that lies closest to the actual behaviour of the algorithm. If you can choose between algorithms based on the simplest notation, great! If you cannot, then you need to increase the complexity of the equations. If this does not work, then give up and go to experiment!

Summer 2007CISC121 - Prof. McLeod21 “Big O” Notation - Cont. Common big O notations: –ConstantO(1) –LogarithmicO(log(n)) –LinearO(n) –N log nO(n(log(n))) –QuadraticO(n 2 ) –CubicO(n 3 ) –PolynomialO(n x ) –ExponentialO(2 n )

Summer 2007CISC121 - Prof. McLeod22 “Big O” Notation - Cont. On the next slide: how these functions vary with n. Assume a rate of 1 instruction per µsec (micro- second):

Summer 2007CISC121 - Prof. McLeod23 “Big O” Notation - Cont. Notationn O(1)1 µsec O(log(n))3 µsec7 µsec10 µsec13 µsec17 µsec20 µsec O(n)10 µsec 100 µsec1 msec10 msec100 msec 1 sec O(nlog(n))33 µsec664 µsec10 msec13.3 msec 1.6 sec20 sec O(n 2 )100 µsec 10 msec1 sec1.7 min16.7 min 11.6 days O(n 3 )1 msec1 sec16.7 min11.6 days 31.7 years years O(2 n )10 msec3e17 years

Summer 2007CISC121 - Prof. McLeod24 “Big O” Notation - Cont. So, algorithms of O(n 3 ) or O(2 n ) are not practical for any more than a few iterations. Quadratic, O(n 2 ) (common for nested “for” loops!) gets pretty ugly for n > 1000 (Bubble sort for example…). To figure out the big O notation for more complicated algorithms, it is necessary to understand some mathematical concepts.

Summer 2007CISC121 - Prof. McLeod25 Some Math… Logarithms and exponents - a quick review: By definition: log b (a) = c, when a = b c

Summer 2007CISC121 - Prof. McLeod26 More Math… Some rules when using logarithms and exponents - for a, b, c positive real numbers: –log b (ac) = log b (a) + log b (c) –log b (a/c) = log b (a) - log b (c) –log b (a c ) = c(log b (a)) –log b (a) = (log c (a))/log c (b) –b log c (a) = a log c (b) –(b a ) c = b ac –b a b c = b a + c –b a /b c = b a - c

Summer 2007CISC121 - Prof. McLeod27 More Math… Summations: If a does not depend on i:

Summer 2007CISC121 - Prof. McLeod28 More Math… Also:

Summer 2007CISC121 - Prof. McLeod29 Still More Math… The Geometric Sum, for any real number, a > 0 and a  1. Also for 0 < a < 1:

Summer 2007CISC121 - Prof. McLeod30 Still More Math… Finally, for a > 0, a  1, and n  2: Lots of other summation formulae exist, (Mathematicians just love to collect them!) but these are the ones commonly found in proofs of big O notations.

Summer 2007CISC121 - Prof. McLeod31 Properties of Big O Notation 1.If d(n) is O(f(n)), then ad(n) is still O(f(n)), for any constant, a > 0. 2.If d(n) is O(f(n)) and e(n) is O(g(n)), then d(n) + e(n) is O(f(n) + g(n)). 3.If d(n) is O(f(n)) and e(n) is O(g(n)), then d(n)e(n) is O(f(n)g(n)). 4.If d(n) is O(f(n)) and f(n) is O(g(n)), then d(n) is O(g(n)).

Summer 2007CISC121 - Prof. McLeod32 Properties of Big O Notation – Cont. 5.If f(n) is a polynomial of degree d (ie. f(n) = a 0 + a 1 n + a 2 n 2 + a 3 n 3 + … + a d n d, then f(n) is O(n d ). 6.log a (n) is O(log b (n)) for any a, b > 0. That is to say log a (n) is O(log 2 (n)) for any a > 1.

Summer 2007CISC121 - Prof. McLeod33 “Big O” Notation - Cont. While correct, it is considered “bad form” to include constants and lower order terms when presenting a Big O notation unless the other terms cannot be simplified and are close in magnitude. For most comparisons between algorithms, the simplest representation of the notation is sufficient. When simplifying a complex function always look for the obviously dominant term (for large n) and toss all the others.

Summer 2007CISC121 - Prof. McLeod34 Application of Big O to Loops For example, a single “ for ” loop: sum = 0; for (i = 0; i < n; i++) sum += a[i]; 2 operations before the loop, 6 operations inside the loop. Total is 2 + 6n. This gives the loop O(n).

Summer 2007CISC121 - Prof. McLeod35 Application of Big O to Loops - Cont. Nested “ for ” loops, where both loop until n: for (i = 0; i < n; i++) { sum[i] = 0; for (j = 0; j < n; j++) sum[i] += a[i, j]; } Outer loop: 1+5n+n(inner loop). Inner loop = 1+7n. Total = 1+6n+7n 2, so algorithm is O(n 2 ).

Summer 2007CISC121 - Prof. McLeod36 Application of Big O to Loops - Cont. Nested “ for ” loops where the inner loop goes less than n times (used in simple sorts): for (i = 0; i < n; i++) { sum[i] = 0; for (j = 0; j < i; j++) sum[i] += a[i, j]; } Outer loop: 1 + 6n + (7i, for i = 1, 2, 3, …, n-1).

Summer 2007CISC121 - Prof. McLeod37 Application of Big O to Loops - Cont. Or: Which is O(n) + O(n 2 ), which is finally just O(n 2 ).

Summer 2007CISC121 - Prof. McLeod38 Nested for loops are usually O(n 2 ), but not always! for (i = 3; i < n; i++) { sum[i] = 0; for (j = i-3; j <= i; j++) sum[i] += a[i, j]; } Inner loop runs 4 times for every value of i. So, this is actually O(n), linear, not quadratic. Application of Big O to Loops - Cont.

Summer 2007CISC121 - Prof. McLeod39 Big O and Other Algorithms For the rest of this course, as we develop algorithms for: –Searching –Sorting –Recursion –Etc. we will analyze the algorithm in terms of its “Big O” notation. We’ll find out what’s good and what’s bad!

Summer 2007CISC121 - Prof. McLeod40 Sorting – Overview Sorting algorithms can be compared on the basis of: –The number of comparisons for a dataset of size n, –The number of data movements (“swaps”) necessary, –How these measures depend on the state of the data (random, mostly in order, reverse order), and –How these measures change with n (“Big O” notation).

Summer 2007CISC121 - Prof. McLeod41 Simple Sorting Algorithms – Insertion Sort Probably the most “instinctive” kind of sort: Find the location for an element and move all others up one, and insert the element. Pseudocode: Loop through array from i=1 to array.length-1, selecting element at position i = temp. –Locate position for temp (position j, where j <= i), and move all elements above j up one location –Put temp at position j.

Summer 2007CISC121 - Prof. McLeod42 Simple Sorting Algorithms – Insertion Sort – Cont. public static void insertionSort (int[] A) { int temp; int i, j; for (i=1; i < A.length; i++) { temp = A[i]; for (j=i; j>0 && temp < A[j-1]; j--) A[j] = A[j-1]; A[j] = temp; } // end for } // end insertionSort

Summer 2007CISC121 - Prof. McLeod43 Complexity of Insertion Sort “ A.length ” is the same as “n”. Outer loop is going to go n times, regardless of the state of the data. How about the inner loop? Best case (data in order) is O(n). Worst and average case is O(n 2 ).

Summer 2007CISC121 - Prof. McLeod44 Simple Sorting Algorithms – Selection Sort This one works by selecting the smallest element and then putting it in its proper location. Pseudocode: Loop through the array from i=0 to one element short of the end of the array. –Select the smallest element in the array range from i plus one to the end of the array. –Swap this value with the value at position i.

Summer 2007CISC121 - Prof. McLeod45 Simple Sorting Algorithms – Selection Sort First, a “ swap ” method that will be used by this and other sorts: public static void swap(int[] A, int pos1, int pos2) { int temp = A[pos1]; A[pos1] = A[pos2]; A[pos2] = temp; } // end swap

Summer 2007CISC121 - Prof. McLeod46 Simple Sorting Algorithms – Selection Sort public static void selectionSort(int[] A) { int i, j, least; for (i = 0; i < A.length-1; i++) { least = i; for (j = i+1; j < A.length; j++) if (A[j] < A[least]) least = j; if (i != least) swap(A, least, i); } // end for } // end selectionSort

Summer 2007CISC121 - Prof. McLeod47 Complexity of Selection Sort Outer loop goes n times, just like Insertion sort. Does the number of executions of the inner loop depend on the state of the data? (See slides 36 and 37) The complexity for comparisons is O(n 2 ), regardless of the state of the data. The complexity of swaps is O(n), since the swap only takes place in the outer loop.

Summer 2007CISC121 - Prof. McLeod48 Simple Sorting Algorithms – Bubble Sort Is best envisioned as a vertical column of numbers as bubbles. The larger bubbles gradually work their way to the top of the column, with the smaller ones pushed down to the bottom. Pseudocode: Loop through array from i=0 to length of array. –Loop down from the last element in the array to i. -Swap adjacent elements if they are in the wrong order.

Summer 2007CISC121 - Prof. McLeod49 Simple Sorting Algorithms – Bubble Sort – Cont. public static void bubbleSort(int[] A) { int i, j; for (i=0; i < A.length; i++) for (j = A.length-1; j > i; j--) if (A[j] < A[j-1]) swap(A, j, j-1); } // end bubbleSort

Summer 2007CISC121 - Prof. McLeod50 Complexity of Bubble Sort Outer loop goes n times, as usual. Both the comparison and swap are in the inner loop (yuk!) Best case (data already in order): comparisons are still O(n 2 ), but swaps will be O(n). Bubble sort can be improved (slightly) by adding a flag to the inner loop. If you loop through the array without swapping, then the data is in order, and the method can be halted.

Summer 2007CISC121 - Prof. McLeod51 Measurements of Execution Speed Refer to the Excel spreadsheet “SortintTimes.xls”: Sorted between 1000 and 10,000 random int values. Used System.currentTimeMillis() to measure execution time. Measured times for completely random array (average case), almost sorted array (best case) and reverse order array (worst case).

Summer 2007CISC121 - Prof. McLeod52 Choosing Between the Simple Sorts (Assuming you have less than about 10,000 data items…) First: what is the state of the data? Second: What is slower: comparisons or swaps? Third: Never use bubble sort, anyways! Since both selection and insertion are O(n 2 ), you will need to look at the coefficient (c), calculated using sample data, to choose between them.

Summer 2007CISC121 - Prof. McLeod53 Sorting Animations For a collection of animation links see: Here are a couple that I liked:

Summer 2007CISC121 - Prof. McLeod54 Sequential Search Sequential search pseudocode: Loop through array starting at the first element until the value of target matches one of the array elements. If a match is not found, return –1.

Summer 2007CISC121 - Prof. McLeod55 Sequential Search, Cont. As code: public int seqSearch(int[] A, int target) { for (int i = 0; i < A.length; i++) if (A[i] == target) return i; return –1; }

Summer 2007CISC121 - Prof. McLeod56 Complexity of Sequential Search Best case - first element is the match: O(1). Worst case - element not found: O(n). Average case - element in middle: O(n).

Summer 2007CISC121 - Prof. McLeod57 Binary Search Binary search pseudocode. Only works on ordered sets: a)Locate midpoint of array to search. b) Determine if target is in lower half or upper half of array. If in lower half, make this half the array to search. If in upper half, make this half the array to search. Loop back to step a), unless the size of the array to search is one, and this element does not match, in which case return –1.

Summer 2007CISC121 - Prof. McLeod58 Binary Search – Cont. public int binSearch (int[] A, int key) { int lo = 0; int hi = A.length - 1; int mid = (lo + hi) / 2; while (lo <= hi) { if (key < A[mid]) hi = mid - 1; else if (A[mid] < key) lo = mid + 1; else return mid; mid = (lo + hi) / 2; } return -1; }

Summer 2007CISC121 - Prof. McLeod59 Complexity of Binary Search For the best case, the element matches right at the middle of the array, and the loop only executes once. For the worst case, key will not be found and the maximum number of loops will occur. Note that the loop will execute until there is only one element left that does not match. Each time through the loop the number of elements left is halved, giving the progression below for the number of elements:

Summer 2007CISC121 - Prof. McLeod60 Complexity of Binary Search, Cont. Number of elements to be searched - progression: The last comparison is for n/2 m, when the number of elements is one (worst case). So, n/2 m = 1, or n = 2 m. Or, m = log 2 (n). So, the algorithm is O(log(n)) for the worst case.

Summer 2007CISC121 - Prof. McLeod61 Binary Search - Cont. Binary search at O(log(n)) is much better than sequential at O(n). Major reason to sort datasets!

Summer 2007CISC121 - Prof. McLeod62 A Caution on the Use of “Big O” Big O notation usually ignores the constant c. The constant’s magnitude depends on external factors (hardware, OS, etc.) For example, algorithm A has the number of operations = 10 8 n, and algorithm B has the form 10n 2. A is then O(n), and B is O(n 2 ). Based on this alone, A would be chosen over B. But, if c is considered, B would be faster for n up to which is a very large dataset! So, in this case, B would be preferred over A, in spite of the Big O notation.

Summer 2007CISC121 - Prof. McLeod63 A Caution on the Use of “Big O” – Cont. For “mission critical” algorithm comparison, nothing beats experiments where actual times are measured. When measuring times, keep everything else constant, just change the algorithm (same hardware, OS, type of data, etc.).

Summer 2007CISC121 - Prof. McLeod64 Recursion An advanced programming technique that can be used with any language that supports procedure or function calls (or “methods!”). Advanced sorting algorithms, like Quicksort, use recursion. We will also look at how to analyze the complexity of recursive algorithms.

Summer 2007CISC121 - Prof. McLeod65 Recursion, Cont. Most simply defined as when a method “calls itself” as part of its normal operation. For example: public int factorial (int n) { if (n == 0) return 1; else return n * factorial(n – 1); } // end factorial method

Summer 2007CISC121 - Prof. McLeod66 Recursion, Cont. Start by considering recursive definitions: A recursive definition has two parts: –The “anchor”, “ground case” or “stopping case” supplies all the basic elements out of which all other elements of the set are constructed. This part is not recursive. –The second part consists of the rules that determine how all other elements are constructed. It will define a new element starting from an element that has already been constructed (the “recursive” part).

Summer 2007CISC121 - Prof. McLeod67 Recursive Definitions For example, the factorial function, n!, is evaluated as: n! = n(n-1)(n-2)(n-3)…(2)(1) So 15! = 15(14)(13)(12)(11)… = Or, in terms of a mathematical definition:

Summer 2007CISC121 - Prof. McLeod68 Recursive Definitions - Cont. Or, in terms of a recursive definition: The upper part is the anchor, and the lower part is the definition of the rules that are used to get all other values. It is also called the “inductive step”. The lower part is recursive.

Summer 2007CISC121 - Prof. McLeod69 Coding a Recursive Definition In code: public int factorial (int n) { if (n == 0) return 1; else return n * factorial(n – 1); } // end factorial method

Summer 2007CISC121 - Prof. McLeod70 Coding a Recursive Definition – Cont. “But, wait!” you say! Can’t this just be done in a loop? Yes, most recursive codes can be carried out using loop(s) instead. However, the “looping” version is seldom as compact as the recursive one. Recursion is used in a program when it produces code that is shorter and more easily understood (and debugged!) than code using a loop. Recursive versions are usually slower than looping versions.

Summer 2007CISC121 - Prof. McLeod71 Coding a Recursive Definition – Cont. A non-recursive factorial method: public int factorial (int n) { int f = 1; if (n > 0) for (int i = 2; i <= n; i++) f *= i; return f; } // end factorial method

Summer 2007CISC121 - Prof. McLeod72 Coding a Recursive Definition – Cont. (Note that we are ignoring the very real possibility of numeric overflow in these factorial methods. It would be better to use a long, or even an “ BigInt ” variable.)

Summer 2007CISC121 - Prof. McLeod73 Coding a Recursive Definition – Cont. Back to the recursive factorial method: public int factorial (int n) { if (n == 0) return 1; else return n * factorial(n – 1); } // end factorial method This is a slightly more “elegant” piece of code, and it is easy to write and debug. What is not so easy to understand is how it works!

Summer 2007CISC121 - Prof. McLeod74 Back to Stacks - Activation Frames Stacks are an integral part of computer architecture. For example, Java byte code is run by the “Java Virtual Machine”, or “JVM”. The JVM is stack – based. Each thread (or process…) in the JVM has its own private run-time stack in memory (our programs are “single threaded”). Each run-time stack contains “activation frames” – one for each method that has been activated. Only one activation frame (or method) can be active at a time for each thread. An activation frame is created, or “pushed”, every time a method is invoked. When the method is completed, the frame is popped off the stack.

Summer 2007CISC121 - Prof. McLeod75 Recursion and the Run-Time Stack Consider another simple recursive definition (ignore n<0): public static double power (double x, int n) { if (n == 0) return 1.0; else return x * power(x, n-1); } // end power

Summer 2007CISC121 - Prof. McLeod76 Recursion and the Run-Time Stack – Cont. In a main method: public static void main(String[] args) { double z = power(5.6, 2); } // end main Look at what happens in debug mode in Eclipse…

Summer 2007CISC121 - Prof. McLeod77 An activation frame for this first call to power is pushed onto the run-time stack, and execution is transferred to this frame (execution does not continue in main). The power method runs until it hits the second call to power on line: power(5.6, 1) // second call Recursion and the Run-Time Stack – Cont. main power 5.6, 2

Summer 2007CISC121 - Prof. McLeod78 Execution in the first call frame stops and a frame for the second call is pushed onto the stack. Execution in the second call frame continues until it hits the third call to power : power(5.6, 0) // third call Recursion and the Run-Time Stack – Cont. main power 5.6, 1 power

Summer 2007CISC121 - Prof. McLeod79 Execution in the second call frame stops and a frame for the third call is pushed onto the stack. Now, power executes until it reaches the line: This causes the value 1.0 to be placed at the “Return value” part of the third call’s activation frame. Recursion and the Run-Time Stack – Cont. main power 5.6, 0 power return 1.0;

Summer 2007CISC121 - Prof. McLeod80 This third call to power completes, and the third call’s frame is popped off the stack. The second call’s frame accepts the value of 1.0, and execution resumes when 5.6 * 1.0 is calculated. Recursion and the Run-Time Stack – Cont. main power 1.0

Summer 2007CISC121 - Prof. McLeod81 This allows the second call’s frame to complete execution. The resulting value (5.6) is placed in the second call’s Return value part of the frame. Recursion and the Run-Time Stack – Cont. main power 5.6

Summer 2007CISC121 - Prof. McLeod82 Recursion and the Run-Time Stack – Cont. The second call’s frame is popped off the stack, and the value “5.6” is given to the first call’s frame and execution resumes at line when 5.6 * 5.6 is calculated. The first call’s execution completes and the first call’s frame is popped off the stack providing the value “31.36” to the next frame on the stack. main power 31.36

Summer 2007CISC121 - Prof. McLeod83 Recursion and the Run-Time Stack – Cont. Execution resumes in the main frame, where the value “31.36” is put into the variable z. See the factorial method, as well (when does the factorial calculation actually take place?). main y=31.36

Summer 2007CISC121 - Prof. McLeod84 Recursion and the Run-Time Stack – Summary Execution stops at any method call and execution is passed to that method. Information on the calling method is preserved in the run- time stack, and a new frame is created for the newly called method. As recursive calls continue, the frames continue to pile up, each one waiting to complete their execution. Finally, the last recursive call is made and the “anchor condition” or the “stopping case” is encountered. The method completes without making another recursive call and may provide a Return value, for example.

Summer 2007CISC121 - Prof. McLeod85 Recursion and the Run-Time Stack – Summary, Cont. The frame for the last call, the “stopping case” is popped off the stack, with the Return value being passed down into the next frame. The frame below accepts the Return value, completes, gets popped off, and so on, until the only frame remaining is the frame that contains the original call to the recursive method. This frame accepts the value from the method, and continues its execution.

Summer 2007CISC121 - Prof. McLeod86 Another Example Remember that the activation frames can also hold code after the recursive call. See TestRecursion.java