CSE332: Data Abstractions Lecture 21: Amortized Analysis Dan Grossman Spring 2010.

Slides:



Advertisements
Similar presentations
ArrayLists David Kauchak cs201 Spring Extendable array Arrays store data in sequential locations in memory Elements are accessed via their index.
Advertisements

CSE332: Data Abstractions Lecture 14: Beyond Comparison Sorting Dan Grossman Spring 2010.
Today’s Agenda  Stacks  Queues  Priority Queues CS2336: Computer Science II.
Quicksort CS 3358 Data Structures. Sorting II/ Slide 2 Introduction Fastest known sorting algorithm in practice * Average case: O(N log N) * Worst case:
UMass Lowell Computer Science Graduate Analysis of Algorithms Prof. Karen Daniels Spring, 2010 Lecture 3 Tuesday, 2/9/10 Amortized Analysis.
CSE332: Data Abstractions Lecture 4: Priority Queues Dan Grossman Spring 2010.
CSE332: Data Abstractions Lecture 9: B Trees Dan Grossman Spring 2010.
CPSC 411, Fall 2008: Set 6 1 CPSC 411 Design and Analysis of Algorithms Set 6: Amortized Analysis Prof. Jennifer Welch Fall 2008.
CSE 326: Data Structures Splay Trees Ben Lerner Summer 2007.
UMass Lowell Computer Science Graduate Analysis of Algorithms Prof. Karen Daniels Spring, 2005 Lecture 3 Tuesday, 2/8/05 Amortized Analysis.
Tirgul 9 Amortized analysis Graph representation.
UMass Lowell Computer Science Graduate Analysis of Algorithms Prof. Karen Daniels Spring, 2009 Lecture 3 Tuesday, 2/10/09 Amortized Analysis.
CSE 326: Data Structures Binary Search Trees Ben Lerner Summer 2007.
Amortized Analysis (chap. 17) Not just consider one operation, but a sequence of operations on a given data structure. Average cost over a sequence of.
CS333 / Cutler Amortized Analysis 1 Amortized Analysis The average cost of a sequence of n operations on a given Data Structure. Aggregate Analysis Accounting.
Andreas Klappenecker [based on the slides of Prof. Welch]
DAST 2005 Week 4 – Some Helpful Material Randomized Quick Sort & Lower bound & General remarks…
UMass Lowell Computer Science Analysis of Algorithms Prof. Karen Daniels Fall, 2001 Lecture 2 (Part 2) Tuesday, 9/11/01 Amortized Analysis.
EXPANDING STACKS AND QUEUES CS16: Introduction to Data Structures & Algorithms 1 Tuesday, February 10, 2015.
CSE332: Data Abstractions Lecture 26: Amortized Analysis Tyler Robison Summer
CSE 373 Data Structures Lecture 15
CSE373: Data Structures & Algorithms Lecture 7: AVL Trees Nicki Dell Spring 2014 CSE373: Data Structures & Algorithms1.
CS 473Lecture 121 CS473-Algorithms I Lecture 12 Amortized Analysis.
CSE373: Data Structures and Algorithms Lecture 2: Proof by Induction & Algorithm Analysis Lauren Milne Summer 2015.
1 Time Analysis Analyzing an algorithm = estimating the resources it requires. Time How long will it take to execute? Impossible to find exact value Depends.
Introduction to Algorithms 6.046J/18.401J/SMA5503 Lecture 14 Prof. Charles E. Leiserson.
Nirmalya Roy School of Electrical Engineering and Computer Science Washington State University Cpt S 223 – Advanced Data Structures Course Review Midterm.
Amortized Analysis The problem domains vary widely, so this approach is not tied to any single data structure The goal is to guarantee the average performance.
Amortized Analysis Typically, most data structures provide absolute guarantees on the worst case time for performing a single operation. We will study.
CSE332: Data Abstractions Lecture 14: Beyond Comparison Sorting Dan Grossman Spring 2012.
CS 61B Data Structures and Programming Methodology July 28, 2008 David Sun.
CSE373: Data Structures & Algorithms Lecture 11: Implementing Union-Find Aaron Bauer Winter 2014.
CSE373: Data Structures & Algorithms Lecture 6: Priority Queues Dan Grossman Fall 2013.
1 Chapter 17: Amortized Analysis I. 2 About this lecture Given a data structure, amortized analysis studies in a sequence of operations, the average time.
Stacks And Queues Chapter 18.
CSE373: Data Structures & Algorithms Lecture 12: Amortized Analysis and Memory Locality Lauren Milne Summer 2015.
CSE373: Data Structures & Algorithms Lecture 12: Amortized Analysis and Memory Locality Nicki Dell Spring 2014.
Advanced Algorithm Design and Analysis (Lecture 12) SW5 fall 2004 Simonas Šaltenis E1-215b
Oct 26, 2001CSE 373, Autumn A Forest of Trees Binary search trees: simple. –good on average: O(log n) –bad in the worst case: O(n) AVL trees: more.
CSE373: Data Structures & Algorithms Lecture 11: Implementing Union-Find Nicki Dell Spring 2014.
CSE373: Data Structures & Algorithms Lecture 10: Implementing Union-Find Dan Grossman Fall 2013.
Amortized Analysis Some of the slides are from Prof. Leong Hon Wai’s resources at National University of Singapore.
CSCE 411H Design and Analysis of Algorithms Set 6: Amortized Analysis Prof. Evdokia Nikolova* Spring 2013 CSCE 411H, Spring 2013: Set 6 1 * Slides adapted.
CSE373: Data Structures & Algorithms Lecture 6: Priority Queues Kevin Quinn Fall 2015.
1/6/20161 CS 3343: Analysis of Algorithms Lecture 2: Asymptotic Notations.
Question of the Day  How can you change the position of 1 toothpick and leave the giraffe in exactly the same form, but possibly mirror-imaged or oriented.
FALL 2005CENG 213 Data Structures1 Priority Queues (Heaps) Reference: Chapter 7.
CSE373: Data Structures & Algorithms Lecture 7: AVL Trees Linda Shapiro Winter 2015.
Introduction to Algorithm Analysis Concepts Fundamental Data Structures and Algorithms Peter Lee January 21, 2003.
CSE332: Data Abstractions Lecture 25: Amortized Analysis Sandra B. Fan Winter
CSE373: Data Structures & Algorithms Lecture 10: Amortized Analysis and Disjoint Sets Aaron Bauer Winter 2014.
CSE373: Data Structures & Algorithms Lecture 8: Amortized Analysis Dan Grossman Fall 2013.
Amortized Analysis In amortized analysis, the time required to perform a sequence of operations is averaged over all the operations performed Aggregate.
CSE373: Data Structures & Algorithms Lecture 8: AVL Trees and Priority Queues Catie Baker Spring 2015.
Amortized Analysis and Heaps Intro David Kauchak cs302 Spring 2013.
CSE373: Data Structures & Algorithms Lecture 5: Dictionary ADTs; Binary Trees Lauren Milne Summer 2015.
CSE332: Data Abstractions Lecture 4: Priority Queues Dan Grossman Spring 2012.
1 Chapter 7 Quicksort. 2 About this lecture Introduce Quicksort Running time of Quicksort – Worst-Case – Average-Case.
CSE373: Data Structures & Algorithms Lecture 8: AVL Trees and Priority Queues Linda Shapiro Spring 2016.
CSE373: Data Structures & Algorithms Lecture Supplement: Amortized Analysis Hunter Zahn Summer 2016 CSE373: Data Structures & Algorithms1.
1 Priority Queues (Heaps). 2 Priority Queues Many applications require that we process records with keys in order, but not necessarily in full sorted.
CSC317 Selection problem q p r Randomized‐Select(A,p,r,i)
CSE373: Data Structures & Algorithms
Amortized Analysis The problem domains vary widely, so this approach is not tied to any single data structure The goal is to guarantee the average performance.
Instructor: Lilian de Greef Quarter: Summer 2017
CSE373: Data Structures & Algorithms Lecture 7: AVL Trees
Cse 373 April 26th – Exam Review.
Instructor: Lilian de Greef Quarter: Summer 2017
CSE373: Data Structures & Algorithms Lecture 11: Implementing Union-Find Linda Shapiro Spring 2016.
CSE373: Data Structures & Algorithms Lecture 5: AVL Trees
Presentation transcript:

CSE332: Data Abstractions Lecture 21: Amortized Analysis Dan Grossman Spring 2010

Amortized Recall our plain-old stack implemented as an array that doubles its size if it runs out of room –How can we claim push is O( 1 ) time if resizing is O( n ) time? –We can’t, but we can claim it’s an O( 1 ) amortized operation What does amortized mean? When are amortized bounds good enough? How can we prove an amortized bound? Will just do two simple examples –The text has more complicated examples and proof techniques –The idea of how amortized describes average cost is essential Spring 20102CSE332: Data Abstractions

Amortized Complexity If a sequence of M operations takes O( M f(n) ) time, we say the amortized runtime is O( f(n) ) The worst case time per operation can be larger than f(n) –For example, maybe f(n)=1, but the worst-case is n But the worst-case for any sequence of M operations is O( M f(n) ) Amortized guarantee ensures the average time per operation for any sequence is O( f(n) ) Amortized bound: worst-case guarantee over sequences of operations –Example: If any n operations take O( n ), then amortized O( 1 ) –Example: If any n operations take O( n 3 ), then amortized O( n 2 ) Spring 20103CSE332: Data Abstractions

Example #1: Resizing stack From lecture 1: A stack implemented with an array where we double the size of the array if it becomes full Claim: Any sequence of push / pop / isEmpty is amortized O( 1 ) Need to show any sequence of M operations takes time O( M ) –Recall the non-resizing work is O( M ) (i.e., M* O( 1 )) –The resizing work is proportional to the total number of element copies we do for the resizing –So it suffices to show that: After M operations, we have done < 2M total element copies (So number of copies per operation is bounded by a constant) Spring 20104CSE332: Data Abstractions

Amount of copying After M operations, we have done < 2M total element copies Let n be the size of the array after M operations –Then we’ve done a total of: n/2 + n/4 + n/8 + … INITIAL_SIZE < n element copies –Since we must have done at least enough push operations to cause resizing up to size n : M  n/2 –So 2M  n > number of element copies Spring 20105CSE332: Data Abstractions

Other approaches If array grows by a constant amount (say 1000), operations are not amortized O( 1 ) –After O( M ) operations, you may have done  ( M 2 ) copies If array shrinks when 1/2 empty, operations are not amortized O( 1 ) –Terrible case: pop once and shrink, push once and grow, pop once and shrink, … If array shrinks when 3/4 empty, it is amortized O( 1 ) –Proof is more complicated, but basic idea remains: by the time an expensive operation occurs, many cheap ones occurred Spring 20106CSE332: Data Abstractions

Example #2: Queue with two stacks A clever and simple queue implementation using only stacks Spring 20107CSE332: Data Abstractions class Queue { Stack in = new Stack (); Stack out = new Stack (); void enqueue(E x){ in.push(x); } E dequeue(){ if(out.isEmpty()) { while(!in.isEmpty()) { out.push(in.pop()); } return out.pop(); } CBACBA in out enqueue: A, B, C

Example #2: Queue with two stacks A clever and simple queue implementation using only stacks Spring 20108CSE332: Data Abstractions class Queue { Stack in = new Stack (); Stack out = new Stack (); void enqueue(E x){ in.push(x); } E dequeue(){ if(out.isEmpty()) { while(!in.isEmpty()) { out.push(in.pop()); } return out.pop(); } in out dequeue BCBC A

Example #2: Queue with two stacks A clever and simple queue implementation using only stacks Spring 20109CSE332: Data Abstractions class Queue { Stack in = new Stack (); Stack out = new Stack (); void enqueue(E x){ in.push(x); } E dequeue(){ if(out.isEmpty()) { while(!in.isEmpty()) { out.push(in.pop()); } return out.pop(); } in out enqueue D, E BCBC A EDED

Example #2: Queue with two stacks A clever and simple queue implementation using only stacks Spring CSE332: Data Abstractions class Queue { Stack in = new Stack (); Stack out = new Stack (); void enqueue(E x){ in.push(x); } E dequeue(){ if(out.isEmpty()) { while(!in.isEmpty()) { out.push(in.pop()); } return out.pop(); } in out dequeue twice C B A EDED

Example #2: Queue with two stacks A clever and simple queue implementation using only stacks Spring CSE332: Data Abstractions class Queue { Stack in = new Stack (); Stack out = new Stack (); void enqueue(E x){ in.push(x); } E dequeue(){ if(out.isEmpty()) { while(!in.isEmpty()) { out.push(in.pop()); } return out.pop(); } in out dequeue again D C B A E

Correctness and usefulness If x is enqueued before y, then x will be popped from in later than y and therefore popped from out sooner than y –So it’s a queue Example: –Wouldn’t it be nice to have a queue of t-shirts to wear instead of a stack (like in your dresser)? –So have two stacks in: stack of t-shirts go after you wash them out: stack of t-shirts to wear if out is empty, reverse in into out Spring CSE332: Data Abstractions

Analysis dequeue is not O( 1 ) worst-case because out might be empty and in may have lots of items But if the stack operations are (amortized) O( 1 ), then any sequence of queue operations is amortized O( 1 ) –The total amount of work done per element is 1 push onto in, 1 pop off of in, 1 push onto out, 1 pop off of out –When you reverse n elements, there were n earlier O( 1 ) enqueue operations to average with Spring CSE332: Data Abstractions

Amortized useful? When the average per operation is all we care about (i.e., sum over all operations), amortized is perfectly fine –This is the usual situation If we need every operation to finish quickly (e.g., in a concurrent setting), amortized bounds are too weak While amortized analysis is about averages, we are averaging cost-per-operation on worst-case input –Contrast: Average-case analysis is about averages across possible inputs. Example: if all initial permutations of an array are equally likely, then quicksort is O( n log n ) on average even though on some inputs it is O( n 2 )) Spring CSE332: Data Abstractions

Not always so simple Proofs for amortized bounds can be much more complicated Example: Splay trees are dictionaries with amortized O( log n ) operations –No extra height field like AVL trees –See Chapter 4.5 For more complicated examples, the proofs need much more sophisticated invariants and “potential functions” to describe how earlier cheap operations build up “energy” or “money” to “pay for” later expensive operations –See Chapter 11 But complicated proofs have nothing to do with the code! Spring CSE332: Data Abstractions