1. 1 Abstract Data Types Stack, Queue Amortized analysis Cormen: Ch 10, 17 (11, 18)

2. 2 ADT is an interface It defines –the type of the data stored –operations, what each operation does (not how) –parameters of each operation

3. 3 ADT Application Implementation of the Data structure חוזה בין מתכנת האפליקציה ומיישם מבנה הנתונים ממשק

4. 4 ADT : how to work with Advantage: –Application programming can be done INDEPENDENTLY of implementation –If care about efficiency (complexity): BE CAREFUL!! Using the ADT the “wrong way” might become quite inefficient!!!

5. 5 Example: Stacks Push(x,S) : Insert element x into S Pop(S) : Delete the last (time) element inserted into S Empty?(S): Return yes if S is empty Top(S): Return the last element inserted into S Size(S) Make-stack()

6. 6 The Stack Data Abstraction push

7. 7 The Stack Data Abstraction push pop push Last in, First out.

8. 8 The Stack: Applications?? 1.REAL-LIFE: 1.Rifle 2.Some document handling? Last in, First out. 1.Computers/ Communications: 1.In some applications – LIFO preferred on FIFO 2.Program control + algorithm == a KEY

9. 9 Evaluate an expression in postfix or Reverse Polish Notation InfixPostfix (2+ 3) * 52 3 + 5 * ( (5 * (7 / 3) ) – (2 * 7) )5 7 3 / * 2 7 *- A stack application Application Implementation חוזה בין מתכנת האפליקציה ומיישם מבנה הנתונים ממשק

10. 10 2 3 + 5 * A stack application 2 3 Application Implementation חוזה בין מתכנת האפליקציה ומיישם מבנה הנתונים ממשק

11. 11 2 3 + 5 * A stack application 5 5

12. 12 2 3 + 5 * A stack application 25

13. 13 Pseudo = כאילו Combination of: 1.Programming (like) commands 2.“free style English” Allows to conveniently express and algorithm. See Cormen (one page) for his language constructs. No need for FORMAL syntax. (can deviate Cormen) A Word on Pseudo-code

14. 14 S ← make-stack() while ( not eof ) do B ← read the next data; if B is an operand then push(B,S) else X ← pop(S) Y ← pop(S) Z ← Apply the operation B on X and Y push(Z,S) return(top(S)) Pseudo-code

15. 15 Implementation We will be interested in algorithms to implement the ADT.. And their efficiency.. Application Implementation חוזה בין מתכנת האפליקציה ומיישם מבנה הנתונים ממשק

16. 16 Using an array 1213 A A[0] A[1] t The stack is represented by the array A and variable t A[2] A[N-1] 12 1 3

17. 17 Using an array 1213 A A[0] A[1] t make-stack(): Allocates the array A, which is of some fixed size N, sets t ← -1 The stack is represented by the array A and variable t A[2] A[N-1]

18. 18 Operations 1213 A t size(S): return (t+1) empty?(S): return (t < 0) top(S): if empty?(S) then error else return A[t] A[N-1] A[0] A[1] A[2]

19. 19 Pop 1213 A t pop(S): if empty?(S) then error else e ← A[t] t ← t – 1 return (e) A[N-1] A[0] A[1] A[2] pop(S)

20. 20 Pop 1213 A t A[N-1] A[0] A[1] A[2] pop(S): if empty?(S) then error else e ← A[t] t ← t – 1 return (e) pop(S)

21. 21 Push 1213 A t push(x,S): if size(S) = N then error else t ← t+1 A[t] ← x A[N-1] A[0] A[1] A[2] push(5,S)

22. 22 Push 1215 A t push(x,S): if size(S) = N then error else t ← t+1 A[t] ← x A[N-1] A[0] A[1] A[2] push(5,S)

23. 23 Implementation with lists 12 1 5 top size=3 x.element x.next x

24. 24 Implementation with lists 12 1 5 top make-stack(): top ← null size ← 0 size=3

25. 25 Operations 12 1 5 top size(S): return (size) empty?(S): return (top = null) top(S): if empty?(S) then error else return top.element size=3

26. 26 Pop 12 1 5 top pop(S): if empty?(S) then error else e ← top.element top ← top.next size ← size-1 return (e) pop(S) size=3

27. 27 Pop 12 1 5 top pop(S): if empty?(S) then error else e ← top.element top ← top.next size ← size-1 return (e) pop(S) size=2

28. 28 Garbage collection 1 5 top pop(S): if empty?(S) then error else e ← top.element top ← top.next size ← size-1 return (e) pop(S) size=2

29. 29 Push 1 5 top size=2 push(x,S): n = new node n.element ← x n.next ← top top ← n size ← size + 1 push(5,S)

30. 30 Push 1 5 top size=2 push(x,S): n = new node n.element ← x n.next ← top top ← n size ← size + 1 push(5,S) 5

31. 31 Push 1 5 top size=2 push(x,S): n = new node n.element ← x n.next ← top top ← n size ← size + 1 push(5,S) 5

32. 32 Push 1 5 top size=3 push(x,S): n = new node n.element ← x n.next ← top top ← n size ← size + 1 push(5,S) 5

33. 33 Analysis Bound the running time of an operation on the worst-case As a function of the “size”, n, of the data structure Example: T(n) < 4n+7 Too detailed, most cases we are just interested in the order of growth

34. 34 Why order of growth Precise cost can change from computer to computer From Programmer to programmer Anyhow MOORE  may go down by factor of 1.5 -2 next year

35. 35 Big-O - קיים c ו- כך ש: דוגמא:

36. 36 Big-O cg(n) f(n) n0n0

37. 37 4n O(n 2 ) 4n 2 O(n 2 ) 2 n O(n 10 ) 10 O(1) 100n 3 +10n O(n 3 ) log 2 (n) O(log 10 (n)) More examples

38. 38 The running time of our stack and queue operations Each operation takes O(1) time

39. 39 Stacks via extendable arrays Array implementation difficulties: Pick large N: wasted space  Pick small N: stack is limited  Solution: –Pick moderate N, and: –When the array is full we will double its size (N) (DOUBLING)

40. 40 Push push(x,S): /* N is size of array*/ if size(S) = N then {allocate a new array of size 2N copy the old array to the new one; N ← 2N } t ← t+1 A[t] ← x 121 A t A[N-1] A[0] A[1]

41. 41 Push push(x,S): if size(S) = N then {allocate a new array of size 2N copy the old array to the new one; N ← 2N } t ← t+1 A[t] ← x 12133 A 4573281 A[N-1] A[0] A[1] t push(5,S)

42. 42 Push push(x,S): if size(S) = N then {allocate a new array of size 2N copy the old array to the new one; N ← 2N } t ← t+1 A[t] ← x 12133 A 4573281 A[0] A[1] t push(5,S)

43. 43 Push push(x,S): if size(S) = N then {allocate a new array of size 2N copy the old array to the new one; N ← 2N } t ← t+1 A[t] ← x 12133 A 4573281 A[0] A[1] 121334573281 t push(5,S)

44. 44 Push push(x,S): if size(S) = N then {allocate a new array of size 2N copy the old array to the new one; N ← 2N } t ← t+1 A[t] ← x t A[2N-1] 12133 A 4573281 A[0] A[1] push(5,S)

45. 45 Push push(x,S): /* N is array size */ if size(S) = N then {allocate a new array of size 2N copy the old array to the new one; N ← 2N } t ← t+1 A[t] ← x 5 A[2N-1] 12133 A 4573281 A[0] A[1] t push(5,S)

46. 46 Analysis An operation may take O(n) worst case time ! But that cannot happen often..

47. 47 Amortized Analysis How long it takes to do m operations in the worst case ? Well, O(nm) Yes, but can it really take that long ?

48. 48

49. 49 x

50. 50 xx

51. 51 xx xxx

52. 52 xx xxxx

53. 53 xx xxxx xxxx x

54. 54 xx xxxx xxxx xx

55. 55 xx xxxx xxxx xxx

56. 56 xx xxxx xxxx xxxx

57. 57 xx xxxx xxxx xxxx xxxx xxxx x

58. 58 xx xxxx xxxx xxxx xxxx xxxx xx

59. 59 xx xxxx xxxx xxxx xxxx xxxx xxx

60. 60 xxxx xxxx xxxx xxxx x After N-1 pushes that cost 1 we will have a push that costs 2N+1 Let N be the size of the array we just copied (half the size of the new array) Total time 3N for N pushes, 3 per push N xxxx

61. 61 xx xxxx xxxx xxxx xxxx xxxx xxx 3+(2∙4+1)= 3∙4 7+(2∙8+1)= 3∙8 Total cost O(m) !! Start from the second row: z=2 1+(2∙2+1)=3∙2 z=4 z=8

62. 62 Theorem: A sequence of m operations on a stack takes O(m) time We proved:

63. 63 Amortized Analysis (The bank’s view) You come to do an operation with a bunch of tokens (amortized cost of the operation)

64. 64 Operation

65. 65 Operation You have to pay for each unit of work by a token (from your POCKET)

66. 66 Operation

67. 67 Operation If you have more tokens than work you put the remaining tokens in the bank

68. 68 Operation

69. 69 Operation If we have more work than tokens we pay with token from the bank

70. 70 Operation If we have more work than tokens we pay with tokens from the bank

71. 71 Amortized Analysis (The bank’s view) Suppose we prove that the bank is never in a deficit  The total work is no larger than the total number of tokens 3*m in our example if m is the number of operations

72. 72 For a proof: Define exactly how each operation pays for its work, and how many tokens it puts in the bank Prove that the balance in the bank is always non-negative Count the total # of tokens – that’s your bound

73. 73 “Easy” push Each push comes with 3 tokens (pocket) If it’s an easy push then we pay with one token for the single unit of work, and put two in the bank. A[N-1] 12133 A 4573281 A[0] A[1] t

74. 74 5 A[N-1] 12133 A 4573281 A[0] A[1] t “Easy” push Imagine that the tokens in the bank are placed on the items of the stack We put one token on the item just inserted and another token on an arbitrary item without a token

75. 75 5 6 A[N-1] 12133 A 4573281 A[0] A[1] t

76. 76 5 667110467257 A[N-1] 12133 A 4573281 A[0] A[1] t

77. 77 “hard” push Tokens from the bank “pay” for copying the array into the larger array 5 667110467257 A[N-1] 12133 A 4573281 A[0] A[1] t

78. 78 Then you pay a token and put two in the bank as an “easy” push 5 667110467257121334573281 A t 5 667110467257121334573281 4 t “hard” push

79. 79 Need to prove The balance is never negative: When we get to an expensive push there are enough tokens to pay for copying By induction: prove that after the i-th push following a copying there are 2i tokens in the bank N/2 easy pushes before the hard push  N tokens are at bank at the hard push. 5 667110467257121334573281 4 t Starting with empty array

80. 80 How many tokens we spent ? Each operation spent 3 tokens

81. 81 Summary So the total # of tokens is 3m THM: A sequence of m operations takes O(m) time !! (we finished the proof) Each operation takes O(1) amortized time

82. 82 Theorem: A sequence of m operations on a stack takes O(m) time proof (3). We formalize the bank as a potential function

83. 83 Amortized(op) = actual(op) +  Define Let  be a potential function (can represent the amount in the bank) Out of pocket = on Operation + on  (bank)

84. 84 Amortized(op 1 ) = actual(op 1 ) +  1 -  0 Amortized(op 2 ) = actual(op 2 ) +  2 -  1 Amortized(op m ) = actual(op m ) +  m -  (m-1) ………… +  i Amortized(op i ) =  i actual(op i ) +  m -  0  i Amortized(op i )   i actual(op i ) if  m -  0  0 We need to find  that gives a nontrivial statement Out of pocket = on Operation + on  (bank)

85. 85 Example: Our extendable arrays Define: the potential of the stack 5 667110467257121334573281 4 t

86. 86 Amortized(push) = actual(push) +  = 1 + 2(t’+1) – N - (2(t+1) – N) = 1+2(t’-t) = 3 Amortized(push) = N + 1 +  = N + 1 + (2 - N) = 3 Can we gain (make money)? No! #pop ≤ # push  amortz>= 2 With copying (hard): Without copying (easy): [ N + 1 + (  after -  before )] Amortized(pop) = 1 +  = 1 + 2(t-1) – N - (2t – N) = -1

87. 87 Amortized(op 1 ) = actual(op 1 ) +  1 -  0 Amortized(op 2 ) = actual(op 2 ) +  2 -  1 Amortized(op n ) = actual(op n ) +  n -  (n-1) ………… +  i Amortized(op i ) =  i actual(op i ) +  n -  0  i Amortized(op i )   i actual(op i ) if  n -  0  0 3m ≥

88. 88 Summary So the total # of tokens is 3m  THM: A sequence of m operations starting from an empty stack takes O(m) time !! Each operations take O(1) amortized time

89. 89 Queue Inject(x,Q) : Insert last element x into Q Pop(Q) : Delete the first element in Q Empty?(Q): Return yes if Q is empty Front(Q): Return the first element in Q Size(Q) Make-queue()

90. 90 The Queue Data Abstraction inject

91. 91 The Queue Data Abstraction inject pop First in, First out (FIFO).

92. 92 Using an array 12142 A 5 A[0] A[1] t pop(Q) A[2] A[N-1]

93. 93 Using an array 1425 A A[0] A[1] t pop(Q) A[2] A[N-1]

94. 94 Using an array 1425 A A[0] A[1] t A[2] A[N-1] This would be inefficient if we insist that elements span a prefix of the array USE a MOVING ARRAY

95. 95 Using an array 12142 A 5 A[0] A[1] r A[2] A[N-1] f A A[0] A[1] r A[2] f Empty queue f=r

96. 96 Using an array 12142 A 5 A[0] A[1] r pop(Q) A[2] A[N-1] f

97. 97 Using an array 142 A 5 A[0] A[1] A[2] A[N-1] f r pop(Q) inject(5,Q)

98. 98 Using an array 142 A 55 A[0] A[1] A[2] A[N-1] f r pop(Q) inject(5,Q)

99. 99 Using an array 142 A 555 A[0] A[1] A[2] A[N-1] f r pop(Q) inject(5,Q) pop(Q)

100. 100 Using an array 2 A 555 A[0] A[1] A[2] A[N-1] f r pop(Q) inject(5,Q) pop(Q) pop(Q), inject(5,Q), pop(Q), inject(5,Q),……….

101. 101 Using an array A 5555 A[0] A[1] r A[2] A[N-1] f pop(Q) inject(5,Q) pop(Q) pop(Q), inject(5,Q), pop(Q), inject(5,Q),………. Fall off the edge? 

102. 102 Make the array “circular” 5 5 A 55 A[0] A[1] r A[2] A[N-1] f Pop(Q), inject(5,Q), pop(Q), inject(5,Q),……….

103. 103 Operations empty?(Q): return (f = r) top(Q): if empty?(Q) then error else return A[f] 142 A 5 A[0] A[1] A[2] A[N-1] f r

104. 104 Operations size(Q): if (r >= f) then return (r-f) else return N-(f-r) 142 A 5 A[0] A[1] A[2] A[N-1] f r

105. 105 Operations size(Q): if (r >= f) then return (r-f) else return N-(f-r) 5 5 A 55 A[0] A[1] r A[2] A[N-1] f

106. 106 Pop pop(Q) 142 A 5 A[0] A[1] A[2] f r pop(Q): if empty?(Q) then error else e ← A[f] f ← (f + 1) mod N return (e)

107. 107 Pop pop(Q): if empty?(Q) then error else e ← A[f] f ← (f + 1) mod N return (e) pop(Q) 142 A 5 A[0] A[1] A[2] f r

108. 108 Inject inject(x,Q): if size(Q) = N-1 then error else A[r] ← x r ← (r+1) mod N inject(5,Q) 42 A 5 A[0] A[1] A[2] f r Why? f=r  empty

109. 109 Inject inject(x,Q): if size(Q) = N-1 then error else A[r] ← x r ← (r+1) mod N inject(5,Q) 42 A 55 A[0] A[1] A[2] f r Or could do doubling

110. 110 Implementation with lists 12 1 5 head size=3 tail inject(4,Q)

111. 111 Implementation with lists 12 1 5 head size=3 tail inject(4,Q) 4

112. 112 Implementation with lists 12 1 5 head size=3 tail inject(4,Q) 4 Complete the details by yourself

113. 113 Continue in DS2…. Slide 26

114. 114 Double ended queue (deque) Push(x,D) : Insert x as the first in D Pop(D) : Delete the first element of D Inject(x,D): Insert x as the last in D Eject(D): Delete the last element of D Size(D) Empty?(D) Make-deque()

115. 115 Implementation with doubly linked lists 5 head size=2 tail 13 x.next x.element x.prev x

116. 116 Empty list head size=0 tail We use two sentinels here to make the code simpler

117. 117 Push push(x,D): n = new node n.element ← x n.next ← head.next (head.next).prev ← n head.next ← n n.prev ← head size ← size + 1 5 head size=1 tail

118. 118 push(x,D): n = new node n.element ← x n.next ← head.next (head.next).prev ← n head.next ← n n.prev ← head size ← size + 1 5 head size=1 tail push(4,D) 4

119. 119 push(x,D): n = new node n.element ← x n.next ← head.next (head.next).prev ← n head.next ← n n.prev ← head size ← size + 1 5 head size=1 tail push(4,D) 4

120. 120 push(x,D): n = new node n.element ← x n.next ← head.next (head.next).prev ← n head.next ← n n.prev ← head size ← size + 1 5 head size=1 tail push(4,D) 4

121. 121 push(x,D): n = new node n.element ← x n.next ← head.next (head.next).prev ← n head.next ← n n.prev ← head size ← size + 1 5 head size=2 tail push(4,D) 4

122. 122 Implementations of the other operations are similar Try by yourself