Chapter 17 Amortized Analysis Lee, Hsiu-Hui Ack: This presentation is based on the lecture slides from Hsu, Lih-Hsing, as well as various materials from the web.

Amortized Analysis The time required to perform a sequence of data structure operations is averaged over all the operations performed. An amortized analysis guarantees the average performance of each operation in the worst case. 20071228 Hsiu-Hui Lee

Three common techniques aggregate analysis accounting method potential method 20071228 Hsiu-Hui Lee

17.1 Aggregate Analysis For all n, a sequence of n operations takes worst time T(n) in total. The amortize cost of each operation is . 20071228 Hsiu-Hui Lee

EX1-Stack operation PUSH(S, x) / POP(S) / MULTIPOP(S, k) 1 while not STACK-EMPTY(S) and k  0 2 do POP(S) 3 k  k – 1 20071228 Hsiu-Hui Lee

Action of MULTIPOP on a stack S  top  10 47  top  23 17 6 39 10 47  MULTIPOP(S,4) MULTIPOP(S,7) 20071228 Hsiu-Hui Lee

PUSH(S, x) / POP(S) MULTIPOP(S, k) Each runs in O(1) time. Actual running time for a sequence of n PUSH, POP operations is (n). MULTIPOP(S, k) pops the top k objects of stack Total cost : min(s, k) s : stack size 20071228 Hsiu-Hui Lee

We can get a better bound. (see the next) Analyze a sequence of n PUSH, POP, and MULTIPOP operation on an initially empty stack. 問每一個 operation 在 worst-case 之 amortized cost? PUSH O(1) POP O(1) MULTIPOP O(n) (the stack size is at most n) Total cost of n operations: O(n2) We can get a better bound. (see the next) Total cost of any seq of n operations: O(n) 20071228 Hsiu-Hui Lee

Each object can be popped at most once for each time it is pushed. Number of times the POP can be called on a nonempty stack, including calls within MULTIPOP, is at most the number of PUSH operatons, which is at most n. Total cost of any seq of n operations: O(n) better bound ! The amortized cost of an operation is . 20071228 Hsiu-Hui Lee

EX2-Incrementing a binary counter A[i] 每 2i 次改變一次 20071228 Hsiu-Hui Lee

2 while i < length[A] and A[i] = 1 3 do A[i]  0 4 i  i + 1 INCREMENT(A) 1 i  0 2 while i < length[A] and A[i] = 1 3 do A[i]  0 4 i  i + 1 5 if i < length[A] 6 then A[i]  1 length[A]=k Cost : (k) 20071228 Hsiu-Hui Lee

Analysis: A seq of n increment operations : O(nk) (k is the word length) Amortize Analysis: The total number of flips in the sequence is 20071228 Hsiu-Hui Lee

17.2 Accounting method We assign differing charges to different operations, with some operations charged more or less than the actual cost. The amount we charge an operation is called its amortized cost. Some are charged more than actual cost. Some are charged less. (在aggregate method所有operation有相同的amortized cost) 20071228 Hsiu-Hui Lee

If the amortized cost > actual cost, the difference is treated as credit Credit can be used later on to help pay for operations whose amortized cost is less than their actual cost. 20071228 Hsiu-Hui Lee

If we want analysis with amortized costs to show that in the worst cast the average cost per operation is small, the total amortized cost of a sequence of operations must be an upper bound on the total actual cost of the sequence. Moreover, as in aggregate analysis, this relationship must hold for all sequences of operations. 20071228 Hsiu-Hui Lee

for all sequence of n operations. If we denote the actual cost of the ith operation by ci and the amortized cost of the ith operation by , we require (i.e. total amortized cost  total actual cost) for all sequence of n operations. The total credit stored in the data structure is the difference between the total actual cost, or The total credit must be nonnegative at all times. 20071228 Hsiu-Hui Lee

EX1-Stack operation n 個運作之後, total amortized cost = O(n) PUSH 1 2 POP actual cost n 個運作之後, total amortized cost = O(n) amortized cost PUSH 1 2 POP MULTIPOP min{k,s} 20071228 Hsiu-Hui Lee

EX2-Incrementing a binary counter actual cost amortized cost Each time, there is exactly one 0 that is changed into 1. The number of 1’s in the counter is never negative! Amortized cost is at most 2 = O(1). 執行 n 次 Increment 其 total amortized cost = O(n) 01 1 2 10 20071228 Hsiu-Hui Lee

17.3 Potential method Like the accounting method, but think of the credit as potential stored with the entire data structure. Accounting method stores credit with specific objects. Potential method stores potential in the data structure as a whole. Can release potential to pay for future operations. Most flexible of the amortized analysis methods. 20071228 Hsiu-Hui Lee

20071228 Hsiu-Hui Lee

20071228 Hsiu-Hui Lee

If then the potential increases. If for all i, then the amortized cost is always an upper bound on actual cost . If then the potential increases. 20071228 Hsiu-Hui Lee

EX1-Stack operation the number of objects in the stack of the ith operation. 20071228 Hsiu-Hui Lee

PUSH 20071228 Hsiu-Hui Lee

MULTIPOP 20071228 Hsiu-Hui Lee

POP The amortized cost of each of these three operations is O(1). the total amortized cost of a sequence of n operations is O(n) 20071228 Hsiu-Hui Lee

EX2-Incrementing a binary counter the number of 1’s in the counter after the ith operations The ith INCREMENT operation resets bits. The Counter starts at zero Amortized cost = O(1) Total amortized cost of n operations is O(n) 20071228 Hsiu-Hui Lee

Even if the counter does not start at zero: 20071228 Hsiu-Hui Lee

17.4 Dynamic tables A nice use of amortized analysis. Scenario Have a table—maybe a hash table. Don’t know in advance how many objects will be stored in it. When it fills, must reallocate with a larger size, copying all objects into the new larger table. When it gets sufficiently small, might want to reallocate with a smaller size. Details of table organization not important. 20071228 Hsiu-Hui Lee

Load factor α = num/size, Goals 1. O(1) amortized time per operation. 2. Unused space always ≤ constant fraction of allocated space. Load factor α = num/size, num = # items stored size = allocated size. If size = 0, then num = 0. Call α = 1. Never allow α > 1. Keepα > a constant fraction ⇒ goal (2). 20071228 Hsiu-Hui Lee

Table expansion Consider only TABLE-INSERT TABLE-DELETE (discuss later) Guarantees that load-factor (load factor) Each time we actually insert an item into the table, it’s an elementary insertion. 20071228 Hsiu-Hui Lee

2 then allocate table[T] with 1 slot 3 size[T]  1 TABLE_INSERT(T, x) 1 if size[T] = 0 2 then allocate table[T] with 1 slot 3 size[T]  1 4 if num[T] = size[T] 5 then allocate new-table with 2  size[T] slots 6 insert all items in table[T] in new-table 7 free table[T] 8 table[T]  new-table 9 size[T]  2  size[T] 10 insert x into table[T] 11 num[T]  num[T] + 1 20071228 Hsiu-Hui Lee

Analysis Running time: Charge 1 per elementary insertion. Count only elementary insertions, since all other costs together are constant per call. ci = actual cost of ith operation If not full, ci = 1. If full, have i - 1 items in the table at the start of the ith operation. Have to copy all i - 1 existing items, then insert ith item⇒ci = i . n operations ⇒ci = O(n)⇒ O(n2) time for n operations. (?) 20071228 Hsiu-Hui Lee

Aggregate method: amortized cost = 3 20071228 Hsiu-Hui Lee

Accounting method: Each item pays for 3 elementary insertions; 1. inserting itself in the current table, 2. moving itself when the table is expanded, and 3. moving another item that has already been moved once when the table is expanded. Charge $3 per insertion of x. $1 pays for x’s insertion. $1 pays for x to be moved in the future. $1 pays for some other item to be moved. 20071228 Hsiu-Hui Lee

Potential method: (not expansion) 20071228 Hsiu-Hui Lee

Example: 20071228 Hsiu-Hui Lee

(expansion) 20071228 Hsiu-Hui Lee

Example: Amortized cost = 3 20071228 Hsiu-Hui Lee

Table expansion and contraction To implement a TABLE-DELETE operation, it is desirable to contract the table when the load factor of the table becomes too small, so that the waste space is not exorbitant. 20071228 Hsiu-Hui Lee

Goal: The load factor of the dynamic table is bounded below by a constant. The amortized cost of a table operation is bounded above by a constant. Set load factor 20071228 Hsiu-Hui Lee

The first n/2 operations are inserted: The second n/2 operations, we perform I, D, D, I, I, D, D, … Total cost of these n operations is . Hence the amortized cost is . Set load factor (as TABLE_DELETE) (after the contraction, the load factor become ) 20071228 Hsiu-Hui Lee

20071228 Hsiu-Hui Lee

Initial 20071228 Hsiu-Hui Lee

TABLE-INSERT if , same as before. if 20071228 Hsiu-Hui Lee

Example: 20071228 Hsiu-Hui Lee

If 20071228 Hsiu-Hui Lee

Example: Amortized cost of Table insert is O(1). 20071228 Hsiu-Hui Lee

TABLE-DELETE If does not cause a contraction (i.e., 20071228 Hsiu-Hui Lee

Example: 20071228 Hsiu-Hui Lee

causes a contraction 20071228 Hsiu-Hui Lee

20071228 Hsiu-Hui Lee

Example: 20071228 Hsiu-Hui Lee

if (Exercise 18.4.3) Amortized cost O(1). 20071228 Hsiu-Hui Lee