Introduction to Algorithms 6.046J/18.401J/SMA5503 Lecture 14 Prof. Charles E. Leiserson.

Slides:



Advertisements
Similar presentations
October 31, 2005Copyright © by Erik D. Demaine and Charles E. LeisersonL13.1 Introduction to Algorithms LECTURE 11 Amortized Analysis Dynamic tables.
Advertisements

Amortized Analysis Some of the slides are from Prof. Leong Hon Wais resources at National University of Singapore Prof. Muhammad Saeed.
CS 473Lecture X1 CS473-Algorithms I Lecture X Dynamic Tables.
Introduction to Algorithms 6.046J Lecture 1 Prof. Shafi Goldwasser Prof. Erik Demaine.
ArrayLists David Kauchak cs201 Spring Extendable array Arrays store data in sequential locations in memory Elements are accessed via their index.
David Luebke 1 5/4/2015 CS 332: Algorithms Dynamic Programming Greedy Algorithms.
1 Chapter 17: Amortized Analysis III. 2 Dynamic Table Sometimes, we may not know in advance the #objects to be stored in a table We may allocate space.
CSE332: Data Abstractions Lecture 21: Amortized Analysis Dan Grossman Spring 2010.
UMass Lowell Computer Science Graduate Analysis of Algorithms Prof. Karen Daniels Spring, 2010 Lecture 3 Tuesday, 2/9/10 Amortized Analysis.
CPSC 411, Fall 2008: Set 6 1 CPSC 411 Design and Analysis of Algorithms Set 6: Amortized Analysis Prof. Jennifer Welch Fall 2008.
Introduction to Analysis of Algorithms
UMass Lowell Computer Science Graduate Analysis of Algorithms Prof. Karen Daniels Spring, 2005 Lecture 3 Tuesday, 2/8/05 Amortized Analysis.
Tirgul 9 Amortized analysis Graph representation.
UMass Lowell Computer Science Graduate Analysis of Algorithms Prof. Karen Daniels Spring, 2009 Lecture 3 Tuesday, 2/10/09 Amortized Analysis.
Amortized Anaysis of Algorihms, A.Yazici, Spring 2007CEng Amortized Analysis of Algorithms Adnan YAZICI Dept. of Computer Engineering Middle East.
Theory I Algorithm Design and Analysis (8 – Dynamic tables) Prof. Th. Ottmann.
Amortized Analysis (chap. 17) Not just consider one operation, but a sequence of operations on a given data structure. Average cost over a sequence of.
Tirgul 8 Universal Hashing Remarks on Programming Exercise 1 Solution to question 2 in theoretical homework 2.
CS305/503, Spring 2009 Amortized Analysis Michael Barnathan.
This material in not in your text (except as exercises) Sequence Comparisons –Problems in molecular biology involve finding the minimum number of edit.
CS333 / Cutler Amortized Analysis 1 Amortized Analysis The average cost of a sequence of n operations on a given Data Structure. Aggregate Analysis Accounting.
Point Location Computational Geometry, WS 2007/08 Lecture 5 Prof. Dr. Thomas Ottmann Algorithmen & Datenstrukturen, Institut für Informatik Fakultät für.
Andreas Klappenecker [based on the slides of Prof. Welch]
17.Amortized analysis Hsu, Lih-Hsing. Computer Theory Lab. Chapter 17P.2 The time required to perform a sequence of data structure operations in average.
UMass Lowell Computer Science Analysis of Algorithms Prof. Karen Daniels Fall, 2001 Lecture 2 (Part 2) Tuesday, 9/11/01 Amortized Analysis.
Amortized Analysis Not just consider one operation, but a sequence of operations on a given data structure. Average cost over a sequence of operations.
CS503: Fourteenth Lecture, Fall 2008 Amortized Analysis, Sets Michael Barnathan.
CSE332: Data Abstractions Lecture 26: Amortized Analysis Tyler Robison Summer
Ajinkya Nene, Lynbrook CS Club. 04/21/2014. Definition Allows you to figure the worst-case bound for the performance of an algorithm (most useful for.
CS 473Lecture 121 CS473-Algorithms I Lecture 12 Amortized Analysis.
1 Summary of lectures 1.Introduction to Algorithm Analysis and Design (Chapter 1-3). Lecture SlidesLecture Slides 2.Recurrence and Master Theorem (Chapter.
Introduction to Algorithms 6.046J/18.401J/SMA5503 Lecture 1 Prof. Charles E. Leiserson.
Amortized Analysis The problem domains vary widely, so this approach is not tied to any single data structure The goal is to guarantee the average performance.
Amortized Analysis Typically, most data structures provide absolute guarantees on the worst case time for performing a single operation. We will study.
TECH Computer Science Dynamic Sets and Searching Analysis Technique  Amortized Analysis // average cost of each operation in the worst case Dynamic Sets.
Design & Analysis of Algorithms COMP 482 / ELEC 420 John Greiner.
Advanced Algorithm Design and Analysis (Lecture 12) SW5 fall 2004 Simonas Šaltenis E1-215b
© 2001 by Charles E. Leiserson Introduction to AlgorithmsDay 12 L8.1 Introduction to Algorithms 6.046J/18.401J/SMA5503 Lecture 8 Prof. Charles E. Leiserson.
Amortized Analysis Some of the slides are from Prof. Leong Hon Wai’s resources at National University of Singapore.
CSCE 411H Design and Analysis of Algorithms Set 6: Amortized Analysis Prof. Evdokia Nikolova* Spring 2013 CSCE 411H, Spring 2013: Set 6 1 * Slides adapted.
CMSC 341 Amortized Analysis.
David Luebke 1 12/12/2015 CS 332: Algorithms Amortized Analysis.
Introduction to Algorithms 6.046J/18.401J LECTURE7 Hashing I Direct-access tables Resolving collisions by chaining Choosing hash functions Open addressing.
Lecture 2 Algorithm Analysis Arne Kutzner Hanyang University / Seoul Korea.
Introduction to Algorithms 6.046J/18.401J/SMA5503 Lecture 1 Prof. Charles E. Leiserson.
Amortized Analysis. Problem What is the time complexity of n insert operations into an dynamic array, which doubles its size each time it is completely.
CSE373: Data Structures & Algorithms Lecture 8: Amortized Analysis Dan Grossman Fall 2013.
Introduction to Algorithms Amortized Analysis My T. UF.
David Luebke 1 2/26/2016 CS 332: Algorithms Dynamic Programming.
Big O David Kauchak cs302 Spring Administrative Assignment 1: how’d it go? Assignment 2: out soon… Lab code.
Amortized Analysis In amortized analysis, the time required to perform a sequence of operations is averaged over all the operations performed Aggregate.
1 Chapter 17: Amortized Analysis II. 2 About this lecture Previous lecture shows Aggregate Method This lecture shows two more methods: (2)Accounting Method.
Introduction to Algorithms: Amortized Analysis. Introduction to Algorithms Amortized Analysis Dynamic tables Aggregate method Accounting method Potential.
Amortized Analysis.
Algorithm Design Techniques, Greedy Method – Knapsack Problem, Job Sequencing, Divide and Conquer Method – Quick Sort, Finding Maximum and Minimum, Dynamic.
Andreas Klappenecker [partially based on the slides of Prof. Welch]
Complexity analysis.
Amortized Analysis (chap. 17)
CSCE 411 Design and Analysis of Algorithms
Presentation by Marty Krogel
Amortized Analysis The problem domains vary widely, so this approach is not tied to any single data structure The goal is to guarantee the average performance.
Chapter 17 Amortized Analysis Lee, Hsiu-Hui
Lecture 16 Amortized Analysis
Introduction to Algorithms 6.046J/18.401J
Haim Kaplan, Uri Zwick March 2018
Introduction to Algorithms
CSCE 411 Design and Analysis of Algorithms
CS 332: Algorithms Amortized Analysis Continued
Lecture 21 Amortized Analysis
CSCE 411 Design and Analysis of Algorithms
Presentation transcript:

Introduction to Algorithms 6.046J/18.401J/SMA5503 Lecture 14 Prof. Charles E. Leiserson

(C)2001 by Charles E. Leiserson Introduction to Algorithms -Day 26 L14 2 How large should a hash table be? Goal: Make the table as small as possible, but large enough so that it won’t overflow (or otherwise become inefficient). Problem: What if we don’t know the proper size in advance? Solution: Dynamic tables. IDEA: Whenever the table overflows, “grow” it by allocating (via malloc or new) a new, larger table. Move all items from the old table into the new one, and free the storage for the old table.

(C)2001 by Charles E. Leiserson Introduction to Algorithms -Day 26 L14 3 Example of a dynamic table 1. INSERT 2. INSERT

(C)2001 by Charles E. Leiserson Introduction to Algorithms -Day 26 L14 4 Example of a dynamic table 1. INSERT 2. INSERT

(C)2001 by Charles E. Leiserson Introduction to Algorithms -Day 26 L14 5 Example of a dynamic table 1. INSERT 2. INSERT

(C)2001 by Charles E. Leiserson Introduction to Algorithms -Day 26 L14 6 Example of a dynamic table 1. INSERT 2. INSERT 3. INSERT

(C)2001 by Charles E. Leiserson Introduction to Algorithms -Day 26 L14 7 Example of a dynamic table 1. INSERT 2. INSERT 3. INSERT

(C)2001 by Charles E. Leiserson Introduction to Algorithms -Day 26 L14 8 Example of a dynamic table 1. INSERT 2. INSERT 3. INSERT

(C)2001 by Charles E. Leiserson Introduction to Algorithms -Day 26 L14 9 Example of a dynamic table 1. INSERT 2. INSERT 3. INSERT 4. INSERT

(C)2001 by Charles E. Leiserson Introduction to Algorithms -Day 26 L14 10 Example of a dynamic table 1. INSERT 2. INSERT 3. INSERT 4. INSERT 5. INSERT

(C)2001 by Charles E. Leiserson Introduction to Algorithms -Day 26 L14 11 Example of a dynamic table 1. INSERT 2. INSERT 3. INSERT 4. INSERT 5. INSERT

(C)2001 by Charles E. Leiserson Introduction to Algorithms -Day 26 L14 12 Example of a dynamic table 1. INSERT 2. INSERT 3. INSERT 4. INSERT

(C)2001 by Charles E. Leiserson Introduction to Algorithms -Day 26 L14 13 Example of a dynamic table 1. INSERT 2. INSERT 3. INSERT 4. INSERT 5. INSERT 6. INSERT 7. INSERT 5. INSERT

(C)2001 by Charles E. Leiserson Introduction to Algorithms -Day 26 L14 14 Worst-case analysis Consider a sequence of n insertions. The worst-case time to execute one insertion is Θ(n). Therefore, the worst-case time for n insertions is n · Θ(n) = Θ(n 2 ). WRONG! In fact, the worst-case cost for n insertions is only Θ(n) 《 Θ (n 2 ). Let’s see why.

(C)2001 by Charles E. Leiserson Introduction to Algorithms -Day 26 L14 15 Tighter analysis Let c i = the cost of the i th insertion =if i – 1 is an exact power of 2, 1 otherwise.

(C)2001 by Charles E. Leiserson Introduction to Algorithms -Day 26 L14 16 Tighter analysis Let c i = the cost of the i th insertion =i if i – 1 is an exact power of 2, 1 otherwise.

(C)2001 by Charles E. Leiserson Introduction to Algorithms -Day 26 L14 17 Cost of n insertions Thus, the average cost of each dynamic-table operation is Θ(n)/n = Θ(1). Tighter analysis (continued)

(C)2001 by Charles E. Leiserson Introduction to Algorithms -Day 26 L14 18 Amortized analysis An amortized analysis is any strategy for analyzing a sequence of operations to show that the average cost per operation is small, even though a single operation within the sequence might be expensive. Even though we’re taking averages, however, probability is not involved! An amortized analysis guarantees the average performance of each operation in the worst case.

(C)2001 by Charles E. Leiserson Introduction to Algorithms -Day 26 L14 19 Types of amortized analyses Three common amortization arguments: the aggregate method, the accounting method, the potential method. We’ve just seen an aggregate analysis. The aggregate method, though simple, lacks the precision of the other two methods. In particular, the accounting and potential methods allow a specific amortized cost to be allocated to each operation.

(C)2001 by Charles E. Leiserson Introduction to Algorithms -Day 26 L14 20 Accounting method Charge i th operation a fictitious amortized cost, where $1 pays for 1 unit of work (i.e., time). This fee is consumed to perform the operation. Any amount not immediately consumed is stored in the bank for use by subsequent operations. The bank balance must not go negative! We must ensure that for all n. Thus, the total amortized costs provide an upper bound on the total true costs.

(C)2001 by Charles E. Leiserson Introduction to Algorithms -Day 26 L14 21 Accounting analysis of dynamic tables Charge an amortized cost of = $3 for the i th insertion. $1 pays for the immediate insertion. $2 is stored for later table doubling. Example: When recent item, and $1 pays to move an old item.

(C)2001 by Charles E. Leiserson Introduction to Algorithms -Day 26 L14 22 Accounting analysis of dynamic tables Charge an amortized cost of = $3 for the i th insertion. $1 pays for the immediate insertion. $2 is stored for later table doubling. When the table doubles, $1 pays to move a recent item, and $1 pays to move an old item. Example:

(C)2001 by Charles E. Leiserson Introduction to Algorithms -Day 26 L14 23 Accounting analysis of dynamic tables Charge an amortized cost of = $3 for the i th insertion. $1 pays for the immediate insertion. $2 is stored for later table doubling. When the table doubles, $1 pays to move a recent item, and $1 pays to move an old item. Example:

(C)2001 by Charles E. Leiserson Introduction to Algorithms -Day 26 L14 24 Accounting analysis (continued) Key invariant: Bank balance never drops below 0. Thus, the sum of the amortized costs provides an upper bound on the sum of the true costs. *Okay, so I lied. The first operation costs only $2, not $3.

(C)2001 by Charles E. Leiserson Introduction to Algorithms -Day 26 L14 25 Potential method IDEA: View the bank account as the potential energy (à la physics) of the dynamic set. Framework: Start with an initial data structure D 0. Operation i transforms D i –1 to D i. The cost of operation i is c i. R Define a potential function Φ : {D i } → R, such that Φ(D 0 ) = 0 and Φ(D i ) ≧ 0 for all i. The amortized cost with respect to Φ is defined to be = c i + Φ(D i ) – Φ(D i –1).

(C)2001 by Charles E. Leiserson Introduction to Algorithms -Day 26 L14 26 Understanding potentials

(C)2001 by Charles E. Leiserson Introduction to Algorithms -Day 26 L14 27 The amortized costs bound the true costs The total amortized cost of n operations is Summing both sides.

(C)2001 by Charles E. Leiserson Introduction to Algorithms -Day 26 L14 28 The amortized costs bound the true costs The total amortized cost of n operations is The series telescopes.

(C)2001 by Charles E. Leiserson Introduction to Algorithms -Day 26 L14 29 The amortized costs bound the true costs The total amortized cost of n operations is

(C)2001 by Charles E. Leiserson Introduction to Algorithms -Day 26 L14 30 Potential analysis of table doubling Define the potential of the table after the ith insertion by(Assume that ) Note: Φ(D 0 ) = 0, Φ(D i ). 0 for all i. Example:

(C)2001 by Charles E. Leiserson Introduction to Algorithms -Day 26 L14 31 Calculation of amortized costs The amortized cost of the i th insertion is

(C)2001 by Charles E. Leiserson Introduction to Algorithms -Day 26 L14 32 Calculation (Case 1) Case 1: i – 1 is an exact power of 2.

(C)2001 by Charles E. Leiserson Introduction to Algorithms -Day 26 L14 33 Calculation (Case 2) Case 2: i – 1 is not an exact power of 2. Therefore, n insertions cost Θ(n) in the worst case. Exercise: Fix the bug in this analysis to show that the amortized cost of the first insertion is only 2.

(C)2001 by Charles E. Leiserson Introduction to Algorithms -Day 26 L14 34 Conclusions Amortized costs can provide a clean abstraction of data-structure performance. Any of the analysis methods can be used when an amortized analysis is called for, but each method has some situations where it is arguably the simplest. Different schemes may work for assigning amortized costs in the accounting method, or potentials in the potential method, sometimes yielding radically different bounds.