UMass Lowell Computer Science Analysis of Algorithms Prof. Karen Daniels Fall, 2001 Midterm Review Fri. Oct 26

MIDTERM EXAM Logistics, Coverage, Format Handout for part of test

Course Grading ä Homework 40% ä Exam 1 15% (closed book) ä Midterm 20% (open book) ä Final Exam 25% (open book) Results are scaled if necessary.

Text/Chapter/Topic Coverage ä Discrete Math Review & Basic Algorithm Analysis Techniques : Chapters 1-6 ä Summations, Recurrences, Sets, Trees, Graph, Counting, Probability, Growth of Functions, Divide- and-Conquer ä Sorting: Chapters 7-9 ä Heapsort, Quicksort, LinearTime-Sorting

Format: Similar to Exam 1 ä Mixture of questions of the following types: - True/False - Short Answer - Analyze Pseudo-Code - Solve a Problem by Designing an Algorithm ä Select an appropriate paradigm/ design pattern ä Use appropriate data structures ä Write pseudo-code ä Justify correctness ä Analyze asymptotic complexity 70% 30%

MIDTERM HANDOUT Layered Directed Network Graph Complete Binary Tree with Maximum Heap Property 55% of the Midterm Exam is based on the representations defined here. (This material is already on our Web site. )

Discrete Math Review Chapters 1-6 Growth of Functions, Summations, Recurrences, Sets, Counting, Probability

Topics ä Discrete Math Review : Chapters 1-6 ä Solving Summations & Recurrences ä Sets, Basic Tree & Graph concepts ä Counting: Permutations/Combinations ä Probability: Basics, including Expectation of a Random Variable ä Proof Techniques: Induction ä Basic Algorithm Analysis Techniques: Chapters 1-6 ä Asymptotic Growth of Functions ä Types of Input: Best/Average/Worst ä Bounds on Algorithm vs. Bounds on Problem ä Algorithmic Paradigms/Design Patterns: Divide-and-Conquer ä Analyze pseudocode running time to form summations &/or recurrences

What are we measuring? ä Some Analysis Criteria: ä Scope ä The problem itself? ä A particular algorithm that solves the problem? ä “Dimension” ä Time Complexity? Space Complexity? ä Type of Bound ä Upper? Lower? Both? ä Type of Input ä Best-Case? Average-Case? Worst-Case? ä Type of Implementation ä Choice of Data Structure

Function Order of Growth O( ) upper bound  ( ) lower bound  ( ) upper & lower bound n 1 n lg(n) n lg 2 (n) 2n2n2n2n n5n5n5n5 lg(n) lg(n)lglg(n) n2n2n2n2 know how to use asymptotic complexity notation to describe time or space complexity know how to order functions asymptotically (behavior as n becomes large)

Types of Algorithmic Input Best-Case Input: of all possible algorithm inputs of size n, it generates the “best” result for Time Complexity: “best” is smallest running time for Time Complexity: “best” is smallest running time Best-Case Input Produces Best-Case Running Time Best-Case Input Produces Best-Case Running Time provides a lower bound on the algorithm’s asymptotic running time provides a lower bound on the algorithm’s asymptotic running time (subject to any implementation assumptions) (subject to any implementation assumptions) for Space Complexity: “best” is smallest storage for Space Complexity: “best” is smallest storage Average-Case Input Worst-Case Input these are defined similarly Best-Case Time <= Average-Case Time <= Worst-Case Time

Bounding Algorithmic Time (using cases) n 1 n lg(n) n lg 2 (n) 2n2n2n2n n5n5n5n5 lg(n) lg(n)lglg(n) n2n2n2n2 T(n) =  (1) T(n) =  (2 n ) very loose bounds are not very useful! Worst-Case time of T(n) =  (2 n ) tells us that worst-case inputs cause the algorithm to take at most exponential time (i.e. exponential time is sufficient). But, can the algorithm every really take exponential time? (i.e. is exponential time necessary?) If, for arbitrary n, we find a worst-case input that forces the algorithm to use exponential time, then this tightens the lower bound on the worst-case running time. If we can force the lower and upper bounds on the worst-case time to match, then we can say that, for the worst-case running time, T(n) =  (2 n ) (i.e. we’ve found the minimum upper bound, so the bound is tight.) Using “case” we can discuss lower and/or upper bounds on: best-case running time or average-case running time or worst-case running time

Know the Difference! n 1 2n2n2n2n n5n5n5n5 worst-case bounds on problem on problem An inefficient algorithm for the problem might exist that takes this much time, but would not help us. No algorithm for the problem exists that can solve it for worst-case inputs in less than linear time. Strong Bound: This worst-case lower bound on the problem holds for every algorithm that solves the problem and abides by our problem’s assumptions. Weak Bound: This worst-case upper bound on the problem comes from just considering one algorithm. Other, less efficient algorithms that solve this problem might exist, but we don’t care about them! Both the upper and lower bounds are probably loose (i.e. probably can be tightened later on).

Master Theorem Master Theorem : Let with a > 1 and b > 1. Then : Case 1: If f(n) = O ( n (log b a) -  ) for some  > o then T ( n ) =  ( n log b a ) Case 2: If f (n) =  (n log b a ) then T ( n ) =  (n log b a * log n ) Case 3: If f ( n ) =  (n log b (a +  ) for some  > o and if a f( n/b) N 0 then T ( n ) =  ( f ( n ) ) Use ratio test to distinguish between cases: f(n)/ f(n)/ n log b a Look for “polynomially larger” dominance.

Sorting Chapters 7-9 Heapsort, Quicksort, LinearTime-Sorting

Topics ä Sorting: Chapters 7-9 ä Sorting Algorithms: ä [Insertion & MergeSort from Chapters 1-6)], Heapsort, Quicksort, LinearTime-Sorting ä Comparison-Based Sorting and its lower bound ä Breaking the lower bound using special assumptions ä Tradeoffs: Selecting an appropriate sort for a given situation ä Time vs. Space Requirements ä Comparison-Based vs. Non-Comparison-Based

Comparison-Based Sorting In algebraic decision tree model, comparison-based sorting of n items requires  (n lg n) time. HeapSort To breaking the lower bound and obtain linear time, forego direct value comparisons and/or make stronger assumptions about input. InsertionSort MergeSort QuickSort  (n lg n)  (n 2 ) BestCaseAverageCaseWorstCase Time: Algorithm:  (n lg n)   (n lg n)  (n lg n)  (n lg n)  (n lg n)  (n lg n)  (n 2 )