Amortized Analysis

How large should a hash table be Goal: Make the table as small as possible, but large enough so that it won’t overflow (or otherwise become inefficient). Problem: What if we don’t know the proper size in advance? Solution: Dynamic tables. IDEA: Whenever the table overflows, “grow” it by allocating (via malloc or new) a new, larger table. Move all items from the old table into the new one, and free the storage for the old table.

Example of a dynamic table

Example of a dynamic table

Example of a dynamic table

Example of a dynamic table

Example of a dynamic table

Example of a dynamic table

Example of a dynamic table

Example of a dynamic table

Example of a dynamic table

Example of a dynamic table

Example of a dynamic table

Example of a dynamic table

Worst-case analysis Consider a sequence of n insertions. The worst-case time to execute one insertion is O(n). Therefore, the worst-case time for n insertions is n*O(n)=O(n^2) It is not wrong, but not tight! In fact, the worst-case cost for n insertions is only O(n)<<O(n^2) Let’s see why.

Tighter analysis Let ci=the cost of the ith insertion. If i-1 is an exact power of 2, ci=i Otherwise, ci=1

Tighter analysis Let ci=the cost of the ith insertion. If i-1 is an exact power of 2, ci=i Otherwise, ci=1

Tighter analysis Let ci=the cost of the ith insertion. If i-1 is an exact power of 2, ci=i Otherwise, ci=1 Cost of n insertions is O(n)! Thus the average cost of each dynamic-table operation is O(1).

Amortized analysis An amortized analysis is any strategy for analyzing a sequence of operations to show that the average cost per operation is small, even though a single operation within the sequence might be expensive. Differs from average-case analysis: 1) Probability is not involved; 2) Guarantees the average performance of each operation in the worst case

Types of amortized analyses Three common amortization arguments: • the aggregate method, • the accounting method, • the potential method. We’ve just seen an aggregate analysis.

Aggregate analysis In aggregate analysis, we show that for all n, a sequence of n operations takes worst time T(n) in total. In the worst case, the average cost, or amortized cost, per operation is therefore T(n)/n. This cost applies to each operation, even when there are several types of operations in the sequence. The other two methods may assign different amortized costs to different types of operation.

Stack operations PUSH(S,x) POP(S) MULTIPOP(S,k) 1. while not STACK-EMPTY(S) and k!=0 2. POP(S) 3. kk-1 4. endwhile What is the running time of a sequence of n PUSH, POP, and MULTIPOP operations on an initially empty stack? Using aggregate analysis, any sequence of n operations takes a total of O(n) time. The average cost of an operation is O(n)/n=O(1) In aggregate analysis, we assign the amortized cost of each operation to be the average cost.

Incrementing a binary counter INCREMENT(A) 1. i0 2. while i<length[A] and A[i]=1 3. A[i]=0 4. ii+1 5. endwhile 6. if i<length[A] 7. then A[i]1 The worst-case time for a sequence of n INCREMENT operations on an initially zero counter is O(n). The average cost of each operation, and therefore the amortized cost per operation, is O(n)/n=O(1)

The accounting method Assign differing charges (amortized cost) to different operations, with some operations charged more or less than they actually cost. When an operation’s amortized cost exceeds its actual cost, the difference is assigned to specific objects in the data structure as credit. Credit can be used later on to help pay for operations whose amortized cost is less than their actual cost. Attention: Total credit stored in the data structure should never becomes negative!

Stack operation The actual costs of the operations: PUSH 1 POP 1 MULTIPOP min(k,s) Assign the amortized costs: PUSH 2 POP 0 MULTIPOP 0 For any sequence of n PUSH, POP, and MULTIPOP operations, the total amortized cost is an upper bound on the total actual cost. Since the total cost is O(n), so is the total actual cost.

Incrementing a binary counter Assume flipping a bit costs a dollar: Charge an amortized cost of 2 dollars to set a bit to 1 Credit on “1”: 1 dollar

Incrementing a binary counter When a bit is set, we use 1 dollar to pay for the actual setting of the bit, and place the other dollar on the bit as credit to be used later when we flip the bit back to 0. Every “1” has a dollar of credit on it, thus we need not charge anything to rest a bit to 0 Credit never becomes negative! In each operation, at least one bit is set: cost 2 dollars Total cost for n operations: O(n)

Potential method Represents the prepaid work as “potential energy” or just “potential”, that can be released to pay for future operations. The potential is associated with the data structure as a whole rather than with specific objects within the data structure Framework: 1 Start with an initial data structure D0 2 Operation i transform Di-1 to Di 3 The cost of operation i is ci 4 Define a potential function Φ: {Di}→R such that Φ(D0)=0 and Φ(Di)≥0 for all i 5 The amortized cost ĉi with respect to Φ is defined to be ĉi=ci+Φ(Di)-Φ(Di-1)

Understanding potentials The amortized cost ĉi with respect to Φ is defined to be ĉi=ci+Φ(Di)-Φ(Di-1). The potential difference ∆Φi=Φ(Di)-Φ(Di-1). If ∆Φi>0, then ĉi>ci. Operation i stores work in the data structure for later use If ∆Φi<0, then ĉi<ci. The data structure delivers up stored work to help pay for operation i. The total amortized cost of n operations is

Stack operation Define the potential function Φ on a stack to be the number of objects in the stack. Φ(Di)≥0=Φ(D0) If the ith operation is PUSH, then Φ(Di)−Φ(Di−1) =(s+1)−s=1. Therefore, the amortized cost of this PUSH is ĉi=ci+Φ(Di)−Φ(Di−1)=1+1=2. The amortized cost of POP is zero. Also is MULTIPOP. So, the total amortized cost is O(n).

Incrementing a binary counter Define the potential of the counter after ith INCREMENT operation to be bi, the number of 1’s in the counter. Suppose that the ith INCREMENT resets ti bits. The actual cost of the operation is at most ti +1. 1) If bi = 0, then the ith INCREMENT resets all k bits, and so bi−1 = ti = k. 2) If bi > 0, then bi = bi−1 − ti +1. In either case, bi ≤ bi−1 − ti +1, therefore Φ(Di) − Φ(Di−1) ≤ (bi−1 − ti +1) − bi−1= 1− ti The amortized cost is therefore ĉi= ci+Φ(Di) − Φ(Di−1) ≤ (ti +1)+(1 − ti) = 2 The worst-case cost of n INCREMENT operations is O(n)

Incrementing a binary counter Suppose there are b0 1’s initially and there are bn 1’s after n INCREMENT operations:

Exercises Analysis Dynamic tables by the following methods: Accounting method Potential method

Accounting method for dynamic table Charge ith operation a fictitious amortized cost ĉi, where $1 pays for 1 unit of work . This fee is consumed to perform the operation. Any amount not immediately consumed is stored in the bank for use by subsequent operations. The bank balance must not go negative! We must ensure that ∑ci<=∑ĉi for all n. Thus, the total amortized costs provide an upper bound on the total true costs.

Accounting method for dynamic table Charge an amortized cost of ĉi = $3 for the ith insertion. • $1 pays for the immediate insertion. • $2 is stored for later table doubling. When the table doubles, $1 pays to move a recent item, and $1 pays to move an old item.

Accounting method for dynamic table Charge an amortized cost of ĉi = $3 for the ith insertion. • $1 pays for the immediate insertion. • $2 is stored for later table doubling. When the table doubles, $1 pays to move a recent item, and $1 pays to move an old item.

Accounting method for dynamic table Charge an amortized cost of ĉi = $3 for the ith insertion. • $1 pays for the immediate insertion. • $2 is stored for later table doubling. When the table doubles, $1 pays to move a recent item, and $1 pays to move an old item.

Accounting method for dynamic table Key invariant: Bank balance never drops below 0. Thus, the sum of the amortized costs provides an upper bound on the sum of the true costs. The first operation costs only $2, not $3.

Potential method for dynamic table Define the potential of the table after the ith insertion by Φ(Di) = 2i – 2logi. (Assume that 2log0 = 0.) Note: • Φ(D0 ) = 0, • Φ(Di) ≥ 0 for all i. Therefore, n insertions cost Θ(n) in the worst case.

Potential method for dynamic table Notice that immediately before an expansion, the potential has built up to the number of items in the table, and therefore it can pay for moving all the items to the new table. Afterwards, the potential drops to 0, but it is immediately increased by 2 when the item that caused the expansion is inserted.

Dynamic table with expansion and contraction To contract the table when the load factor of the table becomes too small. A natural strategy is to double the table size when an item is inserted into a full table and halve the size when a deletion would cause the table to become less than half full. This strategy guarantees that the load factor never drops below ½, but the amortized cost of an operation is (n) Reason: After an expansion, we do not perform enough deletions to pay for a contraction. After a contraction, we do not perform enough insertions to pay for an expansion. A new strategy: double the table size when an item is inserted into a full table and halve the size when a deletion would cause the table to become less than 1/4 full. The actual time for any sequence of n operations on a dynamic table is O(n)

Dynamic table with expansion and contraction Notice that immediately before an expansion, the potential has built up to the number of items in the table, and therefore it can pay for moving all the items to the new table. Likewise, immediately before a contraction, the potential has built up to the number of items in the table.

Conclusions Amortized costs can provide a clean abstraction of data-structure performance. Any of the analysis methods can be used when an amortized analysis is called for, but each method has some situations where it is arguably the simplest. Different schemes may work for assigning amortized costs in the accounting method, or potentials in the potential method, sometimes yielding radically different bounds.

Making Binary Search Dynamic 设n的二进制表示为<nk-1, nk-2, ... , n0>，其中k=log(n+1)。有k个相互之间无关的非降序数组A0, A1, ..., Ak-1，数组Ai的规模为2i。当ni＝1时，数组Ai为满的，当ni＝0时，Ai为空，因此这k个数组中元素总数为n。 对此数据结构，如何进行元素的查找？最坏情况下的时间复杂性是多少？如何进行元素的插入？此时最坏情况下的时间复杂性如何？平摊分析的结果呢？