1. Chapter 17 Amortized Analysis Lee, Hsiu-Hui
Ack: This presentation is based on the lecture slides from Hsu, Lih-Hsing, as well as various materials from the web.

2. Amortized Analysis
The time required to perform a sequence of data structure operations is averaged over all the operations performed.
An amortized analysis guarantees
the average performance of each operation
in the worst case.
Hsiu-Hui Lee

3. Three common techniques
aggregate analysis
accounting method
potential method
Hsiu-Hui Lee

4. 17.1 Aggregate Analysis
For all n, a sequence of n operations takes worst time T(n) in total. The amortize cost of each operation is
Hsiu-Hui Lee

5. EX1-Stack operation PUSH(S, x) / POP(S) / MULTIPOP(S, k)
1 while not STACK-EMPTY(S) and k  0
2 do POP(S)
k  k – 1
Hsiu-Hui Lee

6. Action of MULTIPOP on a stack S
top  10 47 
top  23 17 6 39 10 47 
MULTIPOP(S,4)
MULTIPOP(S,7)
Hsiu-Hui Lee

7. PUSH(S, x) / POP(S) MULTIPOP(S, k) Each runs in O(1) time.
Actual running time for a sequence of n PUSH, POP operations is (n).
MULTIPOP(S, k)
pops the top k objects of stack
Total cost : min(s, k) s : stack size
Hsiu-Hui Lee

8. We can get a better bound. (see the next)
Analyze a sequence of n PUSH, POP, and MULTIPOP operation on an initially empty stack.
問每一個 operation 在 worst-case 之 amortized cost?
PUSH O(1)
POP O(1)
MULTIPOP O(n) (the stack size is at most n)
Total cost of n operations: O(n2)
We can get a better bound. (see the next)
Total cost of any seq of n operations: O(n)
Hsiu-Hui Lee

9. Each object can be popped at most once for each time it is pushed.
Number of times the POP can be called on a nonempty stack, including calls within MULTIPOP, is at most the number of PUSH operatons, which is at most n.
Total cost of any seq of n operations: O(n)
better bound !
The amortized cost of an operation is
Hsiu-Hui Lee

10. EX2-Incrementing a binary counter
A[i] 每 2i 次改變一次
Hsiu-Hui Lee

11. 2 while i < length[A] and A[i] = 1 3 do A[i]  0 4 i  i + 1
INCREMENT(A)
1 i  0
2 while i < length[A] and A[i] = 1
3 do A[i]  0
4 i  i + 1
5 if i < length[A]
6 then A[i]  1
length[A]=k
Cost : (k)
Hsiu-Hui Lee

12. Analysis:
A seq of n increment operations : O(nk) (k is the word length)
Amortize Analysis:
The total number of flips in the sequence is
Hsiu-Hui Lee

13. 17.2 Accounting method
We assign differing charges to different operations, with some operations charged more or less than the actual cost. The amount we charge an operation is called its amortized cost.
Some are charged more than actual cost.
Some are charged less.
(在aggregate method所有operation有相同的amortized cost)
Hsiu-Hui Lee

14. If the amortized cost > actual cost, the difference is treated as credit
Credit can be used later on to help pay for operations whose amortized cost is less than their actual cost.
Hsiu-Hui Lee

15. If we want analysis with amortized costs to show that in the worst cast the average cost per operation is small, the total amortized cost of a sequence of operations must be an upper bound on the total actual cost of the sequence.
Moreover, as in aggregate analysis, this relationship must hold for all sequences of operations.
Hsiu-Hui Lee

16. for all sequence of n operations.
If we denote the actual cost of the ith operation by ci and the amortized cost of the ith operation by , we require
(i.e. total amortized cost  total actual cost)
for all sequence of n operations.
The total credit stored in the data structure is the difference between the total actual cost, or
The total credit must be nonnegative at all times.
Hsiu-Hui Lee

17. EX1-Stack operation n 個運作之後, total amortized cost = O(n) PUSH 1 2 POP
actual cost
n 個運作之後, total amortized cost = O(n)
amortized cost
PUSH 1 2
POP
MULTIPOP
min{k,s}
Hsiu-Hui Lee

18. EX2-Incrementing a binary counter
actual cost
amortized cost
Each time, there is exactly one 0 that is changed into 1.
The number of 1’s in the counter is never negative!
Amortized cost is at most 2 = O(1).
執行 n 次 Increment 其 total amortized cost = O(n)
01 1 2
10
Hsiu-Hui Lee

19. 17.3 Potential method
Like the accounting method, but think of the credit as potential stored with the entire data structure.
Accounting method stores credit with specific objects.
Potential method stores potential in the data structure as a whole.
Can release potential to pay for future operations.
Most flexible of the amortized analysis methods.
Hsiu-Hui Lee

20.
Hsiu-Hui Lee

21.
Hsiu-Hui Lee

22. If then the potential increases.
If for all i,
then the amortized cost is always an upper bound on actual cost .
If then the potential increases.
Hsiu-Hui Lee

23. EX1-Stack operation
the number of objects in the stack of the ith operation.
Hsiu-Hui Lee

24. PUSH
Hsiu-Hui Lee

25. MULTIPOP
Hsiu-Hui Lee

26. POP The amortized cost of each of these three operations is O(1).
the total amortized cost of a sequence of n operations is O(n)
Hsiu-Hui Lee

27. EX2-Incrementing a binary counter
the number of 1’s in the counter after the ith operations
The ith INCREMENT operation resets bits.
The Counter starts at zero
Amortized cost = O(1)
Total amortized cost of n operations is O(n)
Hsiu-Hui Lee

28. Even if the counter does not start at zero:
Hsiu-Hui Lee

29. 17.4 Dynamic tables A nice use of amortized analysis. Scenario
Have a table—maybe a hash table.
Don’t know in advance how many objects will be stored in it.
When it fills, must reallocate with a larger size, copying all objects into the new larger table.
When it gets sufficiently small, might want to reallocate with a smaller size.
Details of table organization not important.
Hsiu-Hui Lee

30. Load factor α = num/size,
Goals
1. O(1) amortized time per operation.
2. Unused space always ≤ constant fraction of allocated space.
Load factor α = num/size,
num = # items stored
size = allocated size.
If size = 0, then num = 0. Call α = 1.
Never allow α > 1.
Keepα > a constant fraction ⇒ goal (2).
Hsiu-Hui Lee

31. Table expansion Consider only TABLE-INSERT
TABLE-DELETE (discuss later)
Guarantees that load-factor
(load factor)
Each time we actually insert an item into the table, it’s an elementary insertion.
Hsiu-Hui Lee

32. 2 then allocate table[T] with 1 slot 3 size[T]  1
TABLE_INSERT(T, x)
1 if size[T] = 0
2 then allocate table[T] with 1 slot
3 size[T]  1
4 if num[T] = size[T]
5 then allocate new-table with 2  size[T] slots
6 insert all items in table[T] in new-table
7 free table[T]
8 table[T]  new-table
9 size[T]  2  size[T]
10 insert x into table[T]
11 num[T]  num[T] + 1
Hsiu-Hui Lee

33. Analysis
Running time: Charge 1 per elementary insertion. Count only elementary insertions, since all other costs together are constant per call.
ci = actual cost of ith operation
If not full, ci = 1.
If full, have i - 1 items in the table at the start of the ith operation. Have to copy all i - 1 existing items, then insert ith item⇒ci = i .
n operations ⇒ci = O(n)⇒ O(n2) time for n operations. (?)
Hsiu-Hui Lee

34. Aggregate method:
amortized cost = 3
Hsiu-Hui Lee

35. Accounting method: Each item pays for 3 elementary insertions;
1. inserting itself in the current table,
2. moving itself when the table is expanded, and
3. moving another item that has already been moved once when the table is expanded.
Charge $3 per insertion of x.
$1 pays for x’s insertion.
$1 pays for x to be moved in the future.
$1 pays for some other item to be moved.
Hsiu-Hui Lee

36. Potential method:
(not expansion)
Hsiu-Hui Lee

37. Example:
Hsiu-Hui Lee

38. (expansion)
Hsiu-Hui Lee

39. Example:
Amortized cost = 3
Hsiu-Hui Lee

40. Table expansion and contraction
To implement a TABLE-DELETE operation, it is desirable to contract the table when the load factor of the table becomes too small, so that the waste space is not exorbitant.
Hsiu-Hui Lee

41. Goal:
The load factor of the dynamic table is bounded below by a constant.
The amortized cost of a table operation is bounded above by a constant.
Set load factor
Hsiu-Hui Lee

42. The first n/2 operations are inserted:
The second n/2 operations, we perform
I, D, D, I, I, D, D, …
Total cost of these n operations is Hence the amortized cost is
Set load factor (as TABLE_DELETE)
(after the contraction, the load factor become )
Hsiu-Hui Lee

43.
Hsiu-Hui Lee

44. Initial
Hsiu-Hui Lee

45. TABLE-INSERT
if , same as before. if
Hsiu-Hui Lee

46. Example:
Hsiu-Hui Lee

47. If
Hsiu-Hui Lee

48. Example:
Amortized cost of Table insert is O(1).
Hsiu-Hui Lee

49. TABLE-DELETE If does not cause a contraction (i.e., 20071228
Hsiu-Hui Lee

50. Example:
Hsiu-Hui Lee

51. causes a contraction
Hsiu-Hui Lee

52.
Hsiu-Hui Lee

53. Example:
Hsiu-Hui Lee

54. if (Exercise )
Amortized cost O(1).
Hsiu-Hui Lee