1. Premaster Course Algorithms 1 Chapter 3: Elementary Data Structures
Christian Scheideler
SS 2019
Chapter 3

2. Overview Basic data structures Search structures (successor searching)
Dictionaries (exact searching)
Chapter 3

3. Stacks and Queues
Stack: implements the LIFO (last-in-first-out) rule, i.e., the youngest element will be removed first Queue: implements the FIFO (first-in-first-out) rule, i.e., the oldest element will be removed first
Chapter 3

4. Stack Stack operations: Push(S,x): insert element x into stack S
Pop(S): remove youngest element from stack
Empty(S): checks if stack is empty
Simplest implementation:
Array S[1..N] (#elements in S never more than N)
S.top: number of elements currently in S[1..N]
Chapter 3

5. Stack S.top=4 Example: Implementation: Push(S,x):
if S.top=N then error „overflow“
else
S.top:=S.top+1
S[S.top]:=x
Pop(S):
if Empty(S) then error „underflow“
S.top:=S.top-1
return S[S.top+1] 15 6 2 9
S.top=4
Empty(S):
if S.top=0 then
return true
else
return false
Chapter 3

6. Stack Starting point: After Push(S,17), Push(S,3): After Pop(S):15 6 2 9
S.top=4 15 6 2 9 17 3
S.top=6 15 6 2 9 17 3
S.top=5
Chapter 3

7. Queue Queue operations: Enqueue(Q,x): insert element x into queue Q
Dequeue(Q): remove oldest element from queue
Empty(Q): checks if queue is empty
Simplest implementation:
Array Q[1..N] (#elements in Q at most N)
Q.head: starting point in Q[1..N]
Q.tail: end point (next free position) in Q[1..N]
Initially, Q.head=Q.tail=1
Chapter 3

8. Queue 15 6 9 8 4 Q.head=7 Q.tail=12 Example: Implementation: Empty(Q):
if Q.head=Q.tail then
return true
else
return false
Enqueue(Q,x):
if Q.head=Q.tail-1 or (Q.head=1 and Q.tail=N) then error „overflow“
Q[Q.tail]:=x
if Q.tail=Q.length then
Q.tail:=1
else Q.tail:=Q.tail+1 15 6 9 8 4
Q.head=7
Q.tail=12
Dequeue(Q):
if Empty(Q) then error „underflow“
else
x:=Q[Q.head]
if Q.head=Q.length then
Q.head:=1
else Q.head:=Q.head+1
return x
Chapter 3

9. Queue Example: After Dequeue: Q.tail=3 Q.head=7 Q.tail=3 Q.head=8 3 515 6 9 8 4 17
Q.tail=3
Q.head=7 15 6 9 8 4 17 5 3
Q.tail=3
Q.head=8
Chapter 3

10. Linked Lists List operations:
List-Insert(L,x): insert element x into list
List-Delete(L,x): remove element x from list
List-Search(L,k): search for key k in list
Implementation as doubly linked list:
L.head: refers to first element of list L
x: element of list, which contains x.prev, X.key, and x.next
x.prev: refers to predecessor of x (NIL: no predecessor)
x.next: refers to successor of x (NIL: no successor)
x.key: stores key of x
L.head 9 / 16 1 4
key
prev
next
prev=NIL
Chapter 3

11. Linked Lists Abstract view: linked list
L.head 9 / 16 4 1 /
Internal representation: linear addressable memory
Contents:
1523
1523
2846
NIL
1280
L.head 16 9 /
Addresses:
867
1280
1523
Chapter 3

12. Linked Lists List-Search(L,k): x:=L.head while xNIL and x.keyk do
x:=x.next
return x
Example: List-Search(L,1)
L.head 9 / 16 1 4
key
prev
next
prev=NIL
Chapter 3

13. Linked Lists List-Insert(L,x): x.next:=L.head if L.headNIL then
L.head.prev:=x
L.head:=x
x.prev:=NIL
x: pointer that points to address of element that needs to be inserted x 5
nil 7 4
nil
L.head
Chapter 3

14. Linked Lists List-Insert(L,x): x.next:=L.head if L.headNIL then
L.head.prev:=x
L.head:=x
x.prev:=NIL x 5
nil 7 4
nil
L.head
Chapter 3

15. Linked Lists List-Insert(L,x): x.next:=L.head if L.headNIL then
L.head.prev:=x
L.head:=x
x.prev:=NIL x 5 7 4
nil
L.head
Chapter 3

16. Linked Lists List-Insert(L,x): x.next:=L.head if L.headNIL then
L.head.prev:=x
L.head:=x
x.prev:=NIL x 5 7 4
nil
L.head
Chapter 3

17. Linked Lists List-Insert(L,x): x.next:=L.head if L.headNIL then
L.head.prev:=x
L.head:=x
x.prev:=NIL x
nil 5 7 4
nil
L.head
Chapter 3

18. Linked Lists List-Delete(L,x): if x.prevNIL then x.prev.next:=x.next
else L.head:=x.next
if x.nextNIL then
x.next.prev:=x.prev x
nil 5 7 4
nil
L.head
Chapter 3

19. Linked Lists List-Delete(L,x): if x.prevNIL then x.prev.next:=x.next
else L.head:=x.next
if x.nextNIL then
x.next.prev:=x.prev
nil 5 4
nil x
L.head 7
Chapter 3

20. Linked Lists List-Delete(L,x): if x.prevNIL then x.prev.next:=x.next
else L.head:=x.next
if x.nextNIL then
x.next.prev:=x.prev x
nil 5 4
nil
L.head 7
Chapter 3

21. Overview Basic data structures Search structures (successor searching)
Dictionaries (exact searching)
Chapter 3

22. Search Structure 8 4 11 20 18 3
Chapter 3

23. Search Structure
insert(15) 8 4 11 20 18 15 3
Chapter 3

24. Search Structure
delete(20) 8 4 11 20 18 15 3
Chapter 3

25. Search Structure search(7) gives 8 (closest successor) 8 4 11 18 15 3
Chapter 3

26. Search Structure S: set of elements
Every element e identified by key(e).
Operations:
S.insert(e: Element): S:=S∪{e}
S.delete(k: Key): S:=S\{e}, where e is the element with key(e)=k
S.search(k: Key): outputs eS with minimal key(e) so that key(e)≥k
Chapter 3

27. Static Search Structure
1. Store elements in sorted array.
search: via binary search (in O(log n) time)
search(12) 1 3 5 10 14 19 28 31 58 60 82 85 89 94 98
Chapter 3

28. Binary Search Input: number x and sorted array A[1],…,A[n]
Algorithm BinarySearch:
l:=1; r:=n
while l < r do
m:=(r+l) div 2
if A[m] = x then return m
if A[m] < x then l:=m+1
else r:=m
return l
Chapter 3

29. Dynamic Search Structure
insert und delete Operations:
Sorted array difficult to update!
Worst case: (n) time 15 1 3 5 10 14 19 28 31 58 60 82 85
Chapter 3

30. Search Structure … 2. Sorted List (with an ∞-Element)
Problem: insert, delete and search take (n) time in the worst case
Observation: If search could be implemented efficiently, then also all other operations 1 3 19 ∞ …
Chapter 3

31. Search Structure
Idee: add navigation structure that allows search to run efficiently
navigation structure 1 3 19 ∞ …
Chapter 3

32. Binary Search Tree search(12) 10 3 19 1 5 14 28 1 3 5 10 14 19 28 ∞
Chapter 3

33. Binary Search Tree 10 3 19 1 5 14 28
Usual setup: without list (full elements and not just keys stored in tree). Updates easier to understand if list is added.
Chapter 3

34. Binary Search Tree Search tree invariant:k
For all keys k´ in T1 and k´´ in T2: k´  k < k´´ T1 T2
Chapter 3

35. Binary Search Tree Formally: for every tree node v let
key(v) be the key stored at v
d(v) the number of children of v
Search tree invariant: (as above)
Degree invariant: All tree nodes have exactly two children (as long as the number of elements in the list is >1)
Key invariant: For every element e in the list there is exactly one tree node v with key(v)=key(e).
Chapter 3

36. Binary Search Tree Search tree invariant: (as before)
Degree invariant: All tree nodes have exactly two children (as long as the number of elements is >1)
Key invariant: For every element e in the list there is exactly one tree node v with key(v)=key(e).
From the search tree and key invariants it follows that for every left subtree T of a node v, the rightmost list element e under T satisfies key(v)=key(e). v T … 1 e
Chapter 3

37. search(x) Operation k
Search strategy:
Start at the root, v, of the search tree
while v is a tree node:
if x  key(v) then let v be the left child of v, otherwise let v be the right child of v
Output (list node) v
For all keys k´ in T1 and k´´ in T2: k´  k < k´´ T1 T2
Chapter 3

38. search(x) Operation k
Correctness of search strategy:
For every left subtree T of a node v, the rightmost list element e under T satisfies key(v)=key(e).
So whenever search(x) enters T, there is an element e in the list below T with key(e)≥x.
For all keys k´ in T1 and k´´ in T2: k´  k < k´´ T1 T2 v T … 1 e
Chapter 3

39. Search(9) 10 3 19 1 5 14 28 1 3 5 10 14 19 28 ∞
Chapter 3

40. Insert and Delete Operations
Strategy:
insert(e): First, execute search(key(e)) to obtain a list element e´. If key(e)=key(e´), replace e´ by e, otherwise insert e between e´ and its predecessor in the list and add a new search tree leaf for e and e´ with key key(e) whose parent is the old parent of e´.
delete(k): First, execute search(k) to obtain a list element e. If key(e)=k, then delete e from the list and the parent v of e from the search tree, and set in the tree node w with key(w)=k: key(w):=key(v). The new parent of the remaining child of v is set to the parent of v. (Thus, a delete can be seen as a reversal of an insert operation.)
Chapter 3

41. Insert(5) 10 1 14 28 1 10 14 28 ∞
Chapter 3

42. Insert(5) 10 1 14 5 28 1 5 10 14 28 ∞
Chapter 3

43. Insert(12) 10 1 14 5 28 1 5 10 14 28 ∞
Chapter 3

44. Insert(12) 10 1 14 5 12 28 1 5 10 12 14 28 ∞
Chapter 3

45. Delete(1) 10 1 14 5 12 28 1 5 10 12 14 28 ∞
Chapter 3

46. Delete(1) 10 14 5 12 28 5 10 12 14 28 ∞
Chapter 3

47. Delete(14) 10 14 5 12 28 5 10 12 14 28 ∞
Chapter 3

48. Delete(14) 10 12 5 28 5 10 12 28 ∞
Chapter 3

49. Binary Search Tree Problem: binary tree can degenerate!
Example: numbers are inserted in sorted order
Search requires (n) time in the worst case 1 3 10 14 19 5 28 ∞
Chapter 3

50. Binary Search Tree Lowest possible depth for n elements: log n.
Goal: maintain a depth of O(log n). Such a search tree is called balanced. 10 3 19 1 5 14 28
Chapter 3

51. Search Trees Problem: binary tree can degenerate!
Trees that stay balanced:
(a,b)-tree (presented here)
red-black tree
AVL-tree …
Gewichtsbalancierter Baum: gut für Bereichsanfragen.
Chapter 3

52. (a,b)-Trees Problem: how to maintain a balanced search tree Idea:
All nodes v (except for the root) have degree d(v) with a≤d(v)≤b, where a≥2 and b≥2a-1 (otherwise this cannot be enforced)
All leaves have the same depth (which implies that the tree stays balanced)
 (a,b)-trees, which are a general form of B-trees (see Chapter 18).
Chapter 3

53. (a,b)-Trees Formally: for a tree node v let
d(v) be the number of children of v
t(v) be the depth of v (root has depth 0)
Form Invariant: For all leaves v,w: t(v)=t(w)
Degree Invariant: For all inner nodes v except for root: d(v)[a,b], for root r: d(r)[2,b] (as long as #elements >1)
Chapter 3

54. (a,b)-Trees
Lemma 3.10: An (a,b)-tree with n elements has depth at most 1+⌊loga (n+1)/2⌋
Proof:
The root has degree ≥2 and every other inner node has degree ≥a.
At depth t there are at least 2at-1 nodes
n+1≥2at-1 ⇔ t≤1+⌊loga (n+1)/2⌋
Chapter 3

55. (a,b)-Trees (a,b)-Tree-Rule: Then search operation easy to implement.
s1, s2,…,sd-1
For all keys k in Ti and k´ in Ti+1: k  si < k´ T1 T2 Td
Chapter 3

56. Search(9)
10 19 14 28 1 3 5 10 14 19 28 ∞
Chapter 3

57. Insert(e) Operation Strategy:
First search(key(e)) until some e´ found in the list. If key(e´)>key(e), insert e in front of e´, otherwise replace e´ by e.
. . . e´
. . . ∞
Chapter 3

58. Insert(e) Operation Strategy:
First search(key(e)) until some e´ found in the list. If key(e´)>key(e), insert e in front of e´, otherwise replace e´ by e.
. . . e e´
. . . ∞
Chapter 3

59. Insert(e) Operation
Add key(e) and pointer to e in tree node v above e´. If we still have d(v)[a,b] after-wards, then we are done. v v
… x z …
… x y z …
. . . x y z
. . .
. . . x y z
. . . e e’ e e’
Chapter 3

60. Insert(e) Operation
If d(v)>b, then cut v into two nodes. (Example: a=2, b=4) w
… b z …
… b u z …
split v
x u u‘ y x
u‘ y e e’ e e’ x u u‘ y z x u u‘ y z
Chapter 3

61. Insert(e) Operation
If after splitting v, d(w)>b, then cut w into two nodes (and so on, until all nodes have degree ≤b or we reached the root)
… r s …
… r b s …
split w
a b u z a
u z
Chapter 3

62. Insert(e) Operation
If for the root v of T, d(v)>b, then cut v into two nodes and create a new root node. b
split v
a b c d a
c d
Chapter 3

63. Insert(8)
a=2, b=4
10 19 14 28 1 3 5 10 14 19 28 ∞
Chapter 3

64. Insert(8)
a=2, b=4
10 19 14 28 1 3 5 8 10 14 19 28 ∞
Chapter 3

65. Insert(8)
a=2, b=4 1
5 8 14 28 1 3 5 8 10 14 19 28 ∞
Chapter 3

66. Insert(6)
a=2, b=4 1
5 8 14 28 1 3 5 8 10 14 19 28 ∞
Chapter 3

67. Insert(6)
a=2, b=4
. . . 1 14 1 3 5 6 8 10 14 19
Chapter 3

68. Insert(7)
a=2, b=4
. . . 1 14 1 3 5 6 8 10 14 19
Chapter 3

69. Insert(7)
a=2, b=4
. . . 1 14 1 3 5 6 7 8 10 14 19
Chapter 3

70. Insert(7)
a=2, b=4
. . . 1 5
7 8 14 1 3 5 6 7 8 10 14 19
Chapter 3

71. Insert(7)
a=2, b=4 6 3
10 19
. . . 1 5
7 8 14 1 3 5 6 7 8 10 14 19
Chapter 3

72. Insert Operation
Form Invariant: For all leaves v,w: t(v)=t(w) Satisfied by Insert!
Degree Invariant: For all inner nodes v except for the root: d(v)[a,b], for root r: d(r)[2,b]
1) Insert splits nodes of degree b+1 into nodes of degree ⌊(b+1)/2⌋ and ⌈(b+1)/2⌉. If b≥2a-1, then both values are at least a.
2) If root has reached degree b+1, then a new root of degree 2 is created.
Chapter 3

73. Delete(k) Operation Strategy:
First search(k) until some element e is reached in the list. If key(e)=k, remove e from the list, otherwise we are done.
. . . e´ e
e´´
. . . ∞
Chapter 3

74. Delete(k) Operation Strategy:
First search(k) until some element e is reached in the list. If key(e)=k, remove e from the list, otherwise we are done.
. . . e´
e´´
. . . ∞
Chapter 3

75. Delete(k) Operation
Remove pointer to e and key k from the leaf node v above e. (e rightmost child: perform key exchange like in binary tree!) If afterwards we still have d(v)≥a, we are done. v v
… x k y …
… x y …
. . . x k y
. . .
. . . x y
. . . e
Chapter 3

76. Delete(k) Operation
Remove pointer to e and key k from the leaf node v above e. (e rightmost child: perform key exchange like in binary tree!) If afterwards we still have d(v)≥a, we are done.
… y …
… k … v v
… x y
… x
. . . x y k
. . .
. . . x y
. . . e
Chapter 3

77. Delete(k) Operation
If d(v)<a and the preceding or succeeding sibling of v has degree >a, steal an edge from that sibling. (Example: a=2, b=4)
u k t
u y t v
y r s k
r s x y r s t x y r s t
Chapter 3

78. Delete(k) Operation
If d(v)<a and the preceeding and succeeding siblings of v have degree a, merge v with one of these. (Example: a=3, b=5)
u y t
u t v
merge x
r s
x y r s x y r s t x y r s t
Chapter 3

79. Delete(k) Operation
Peform changes upwards until all inner nodes (except for the root) have degree ≥a. If root has degree <2: remove root.
x y z
x y z
Chapter 3

80. Delete(10)
a=2, b=4
10 19 14 28 1 3 5 10 14 19 28 ∞
Chapter 3

81. Delete(10)
a=2, b=4
5 19
1 3 14 28 1 3 5 14 19 28 ∞
Chapter 3

82. Delete(14)
a=2, b=4
5 19
1 3 14 28 1 3 5 14 19 28 ∞
Chapter 3

83. Delete(14)
a=2, b=4
5 19
1 3 28 1 3 5 19 28 ∞
Chapter 3

84. Delete(14)
a=2, b=4
3 19 1 5 28 1 3 5 19 28 ∞
Chapter 3

85. Delete(3)
a=2, b=4
3 19 1 5 28 1 3 5 19 28 ∞
Chapter 3

86. Delete(3)
a=2, b=4
3 19 1 5 28 1 5 19 28 ∞
Chapter 3

87. Delete(3)
a=2, b=4
1 19 5 28 1 5 19 28 ∞
Chapter 3

88. Delete(3)
a=2, b=4 19
1 5 28 1 5 19 28 ∞
Chapter 3

89. Delete(1)
a=2, b=4 19
1 5 28 1 5 19 28 ∞
Chapter 3

90. Delete(1)
a=2, b=4 19 5 28 5 19 28 ∞
Chapter 3

91. Delete(19)
a=2, b=4 19 5 28 5 19 28 ∞
Chapter 3

92. Delete(19)
a=2, b=4 5 28 5 28 ∞
Chapter 3

93. Delete(19)
a=2, b=4
5 28 5 28 ∞
Chapter 3

94. Delete(19)
a=2, b=4
5 28 5 28 ∞
Chapter 3

95. Delete Operation
Form Invariant: For all leaves v,w: t(v)=t(w) Satisfied by Delete!
Degree Invariant: For all inner nodes v except for the root: d(v)[a,b], for root r: d(r)[2,b]
1) Delete merges node of degree a-1 with node of degree a. Since b2a-1, the resulting node has degree at most b.
2) Delete moves edge from a node of degree >a to a node of degree a-1. Also OK.
3) Root deleted: children have been merged, degree of the remaining child is ≥a (and also ≤b), so also OK.
Chapter 3

96. Application: External (a,b)-Tree
(a,b)-trees well suited for large amounts of data
Internal memory (RAM)
size M
block size B  choose a,b=Q(B)
External memory (harddisk)
Chapter 3

97. Overview Basic data structures Search structures (successor searching)
Dictionaries (exact searching)
Chapter 3

98. Dictionary 8 4 11 20 18 3
Chapter 3

99. Dictionary
insert(15) 8 4 11 20 18 15 3
Chapter 3

100. Dictionary
delete(20) 8 4 11 20 18 15 3
Chapter 3

101. Dictionary
lookup(8) outputs 8 8 4 11 18 15 3
Chapter 3

102. Dictionary
lookup(7) outputs ⊥ 8 4 11 18 15 3
Chapter 3

103. Dictionary Data Structure
S: set of elements in data structure
Every element e identified by key(e).
Operations:
S.insert(e: Element): S:=S∪{e}
S.delete(k: Key): S:=S\{e}, where e is the element with key(e)=k
S.lookup(k: Key): If there is an e∈S with key(e)=k, output e, otherwise output ⊥
Chapter 3

104. Hashing time- and space-efficient hash function h:U → T hash table T 13 5 10 14 19
time- and space-efficient hash function h:U → T 14 5 3 19 1 10
hash table T
Chapter 3

105. Hashing Task of the hash function: Good spreading of elements across T1 3 5 10 14 19
Task of the hash function: Good spreading of elements across T 14 5 3 19 1 10
hash table T
Chapter 3

106. Hashing 1 3 5 10 14 19
No collisions of elements in T: insert, delete, lookup in O(1) time 14 5 3 19 1 10
hash table T
Chapter 3

107. Hashing (no collisions)
Procedure insert(e: Element) T[h(key(e))] := e
Procedure delete(k: Key) if key(T[h(k)])=k then T[h(k)] := ⊥
Function lookup(k: Key): Element∪{⊥} return T[h(k)]
Chapter 3

108. Problem: handling collisions
Hashing
Problem: handling collisions
Solutions:
Closed addressing: append elements in hash table entry to a list
Open addressing: linear, quadratic probing
Two hash tables: Cuckoo hashing
Chapter 3

109. Open Addressing Linear probing: Store new element x in the first free1 3 5 10 14 19
Store new element x in the first free
slot starting with T[i], where i=h(x.key) 14 5 3 19 1 10
Chapter 3

110. Cuckoo Hashing hash function h1 hash function h2 hash table T15 14
Goal: store element in one of its two positions
hash function h1
hash function h2 1 14 5
hash table T1
hash table T2
Chapter 3

111. Cuckoo Hashing T1, T2: Array [0..m-1] of Element
Procedure lookup(k: Key): Element∪{⊥} if key(T1[h1(k)])=k then return T1[h1(k)] if key(T2[h2(k)])=k then return T2[h2(k)] return ⊥
Procedure delete(k: Key) if key(T1[h1(k)])=k then T1[h1(k)]:=⊥ if key(T2[h2(k)])=k then T2[h2(k)]:=⊥
Chapter 3

112. Cuckoo Hashing
Procedure insert(e: Element) // key(e) already in hash table? if key(T1[h1(key(e))])=key(e) then T1[h1(key(e))] := e; return if key(T2[h2(key(e))])=key(e) then T2[h2(key(e))] := e; return // no, then insert it while true do e ↔ T1[h1(key(e))] // switch e with element in T1 if e=⊥ then return e ↔ T2[h2(key(e))] // switch e with element in T if e=⊥ then return
Better: at most d log n often, where the constant d is „sufficiently large“.
If more than d log n rounds are needed, then rehash all elements with
new h1, h2
Chapter 3

113. Cuckoo Hashing 1 5 10 10 14 1 5 14
Chapter 3

114. Cuckoo Hashing 1 5 10 14 5 1 14
Infinite loop!
Chapter 3

115. Cuckoo Hashing Runtime: lookup, delete: O(1) (worst case!)
insert: O(number of replacements + possibly the time for a complete rehash)
Runtime of insert operation: 2 cases
after t rounds, insert enters an infinite loop, which requires rehash
insert terminates after t rounds
Nevertheless, expected runtime of insert: O(1) if m2n, where m is the hash table size.
Chapter 3

116. Next Lecture
Chapter 4 (dynamic programming and Greedy algorithms) skipped this time.
We continue with
Chapter 5 (Basic graph algorithms).
Chapter 3