Download presentation
Presentation is loading. Please wait.
Published byIda Lehtonen Modified over 5 years ago
1
Premaster Course Algorithms 1 Chapter 3: Elementary Data Structures
Christian Scheideler SS 2019 Chapter 3
2
Overview Basic data structures Search structures (successor searching)
Dictionaries (exact searching) Chapter 3
3
Stacks and Queues Stack: implements the LIFO (last-in-first-out) rule, i.e., the youngest element will be removed first Queue: implements the FIFO (first-in-first-out) rule, i.e., the oldest element will be removed first Chapter 3
4
Stack Stack operations: Push(S,x): insert element x into stack S
Pop(S): remove youngest element from stack Empty(S): checks if stack is empty Simplest implementation: Array S[1..N] (#elements in S never more than N) S.top: number of elements currently in S[1..N] Chapter 3
5
Stack S.top=4 Example: Implementation: Push(S,x):
if S.top=N then error „overflow“ else S.top:=S.top+1 S[S.top]:=x Pop(S): if Empty(S) then error „underflow“ S.top:=S.top-1 return S[S.top+1] 15 6 2 9 S.top=4 Empty(S): if S.top=0 then return true else return false Chapter 3
6
Stack Starting point: After Push(S,17), Push(S,3): After Pop(S):
15 6 2 9 S.top=4 15 6 2 9 17 3 S.top=6 15 6 2 9 17 3 S.top=5 Chapter 3
7
Queue Queue operations: Enqueue(Q,x): insert element x into queue Q
Dequeue(Q): remove oldest element from queue Empty(Q): checks if queue is empty Simplest implementation: Array Q[1..N] (#elements in Q at most N) Q.head: starting point in Q[1..N] Q.tail: end point (next free position) in Q[1..N] Initially, Q.head=Q.tail=1 Chapter 3
8
Queue 15 6 9 8 4 Q.head=7 Q.tail=12 Example: Implementation: Empty(Q):
if Q.head=Q.tail then return true else return false Enqueue(Q,x): if Q.head=Q.tail-1 or (Q.head=1 and Q.tail=N) then error „overflow“ Q[Q.tail]:=x if Q.tail=Q.length then Q.tail:=1 else Q.tail:=Q.tail+1 15 6 9 8 4 Q.head=7 Q.tail=12 Dequeue(Q): if Empty(Q) then error „underflow“ else x:=Q[Q.head] if Q.head=Q.length then Q.head:=1 else Q.head:=Q.head+1 return x Chapter 3
9
Queue Example: After Dequeue: Q.tail=3 Q.head=7 Q.tail=3 Q.head=8 3 5
15 6 9 8 4 17 Q.tail=3 Q.head=7 15 6 9 8 4 17 5 3 Q.tail=3 Q.head=8 Chapter 3
10
Linked Lists List operations:
List-Insert(L,x): insert element x into list List-Delete(L,x): remove element x from list List-Search(L,k): search for key k in list Implementation as doubly linked list: L.head: refers to first element of list L x: element of list, which contains x.prev, X.key, and x.next x.prev: refers to predecessor of x (NIL: no predecessor) x.next: refers to successor of x (NIL: no successor) x.key: stores key of x L.head 9 / 16 1 4 key prev next prev=NIL Chapter 3
11
Linked Lists Abstract view: linked list
L.head 9 / 16 4 1 / Internal representation: linear addressable memory Contents: 1523 1523 2846 NIL 1280 L.head 16 9 / Addresses: 867 1280 1523 Chapter 3
12
Linked Lists List-Search(L,k): x:=L.head while xNIL and x.keyk do
x:=x.next return x Example: List-Search(L,1) L.head 9 / 16 1 4 key prev next prev=NIL Chapter 3
13
Linked Lists List-Insert(L,x): x.next:=L.head if L.headNIL then
L.head.prev:=x L.head:=x x.prev:=NIL x: pointer that points to address of element that needs to be inserted x 5 nil 7 4 nil L.head Chapter 3
14
Linked Lists List-Insert(L,x): x.next:=L.head if L.headNIL then
L.head.prev:=x L.head:=x x.prev:=NIL x 5 nil 7 4 nil L.head Chapter 3
15
Linked Lists List-Insert(L,x): x.next:=L.head if L.headNIL then
L.head.prev:=x L.head:=x x.prev:=NIL x 5 7 4 nil L.head Chapter 3
16
Linked Lists List-Insert(L,x): x.next:=L.head if L.headNIL then
L.head.prev:=x L.head:=x x.prev:=NIL x 5 7 4 nil L.head Chapter 3
17
Linked Lists List-Insert(L,x): x.next:=L.head if L.headNIL then
L.head.prev:=x L.head:=x x.prev:=NIL x nil 5 7 4 nil L.head Chapter 3
18
Linked Lists List-Delete(L,x): if x.prevNIL then x.prev.next:=x.next
else L.head:=x.next if x.nextNIL then x.next.prev:=x.prev x nil 5 7 4 nil L.head Chapter 3
19
Linked Lists List-Delete(L,x): if x.prevNIL then x.prev.next:=x.next
else L.head:=x.next if x.nextNIL then x.next.prev:=x.prev nil 5 4 nil x L.head 7 Chapter 3
20
Linked Lists List-Delete(L,x): if x.prevNIL then x.prev.next:=x.next
else L.head:=x.next if x.nextNIL then x.next.prev:=x.prev x nil 5 4 nil L.head 7 Chapter 3
21
Overview Basic data structures Search structures (successor searching)
Dictionaries (exact searching) Chapter 3
22
Search Structure 8 4 11 20 18 3 Chapter 3
23
Search Structure insert(15) 8 4 11 20 18 15 3 Chapter 3
24
Search Structure delete(20) 8 4 11 20 18 15 3 Chapter 3
25
Search Structure search(7) gives 8 (closest successor) 8 4 11 18 15 3
Chapter 3
26
Search Structure S: set of elements
Every element e identified by key(e). Operations: S.insert(e: Element): S:=S∪{e} S.delete(k: Key): S:=S\{e}, where e is the element with key(e)=k S.search(k: Key): outputs eS with minimal key(e) so that key(e)≥k Chapter 3
27
Static Search Structure
1. Store elements in sorted array. search: via binary search (in O(log n) time) search(12) 1 3 5 10 14 19 28 31 58 60 82 85 89 94 98 Chapter 3
28
Binary Search Input: number x and sorted array A[1],…,A[n]
Algorithm BinarySearch: l:=1; r:=n while l < r do m:=(r+l) div 2 if A[m] = x then return m if A[m] < x then l:=m+1 else r:=m return l Chapter 3
29
Dynamic Search Structure
insert und delete Operations: Sorted array difficult to update! Worst case: (n) time 15 1 3 5 10 14 19 28 31 58 60 82 85 Chapter 3
30
Search Structure … 2. Sorted List (with an ∞-Element)
Problem: insert, delete and search take (n) time in the worst case Observation: If search could be implemented efficiently, then also all other operations 1 3 19 ∞ … Chapter 3
31
Search Structure Idee: add navigation structure that allows search to run efficiently navigation structure 1 3 19 ∞ … Chapter 3
32
Binary Search Tree search(12) 10 3 19 1 5 14 28 1 3 5 10 14 19 28 ∞
Chapter 3
33
Binary Search Tree 10 3 19 1 5 14 28 Usual setup: without list (full elements and not just keys stored in tree). Updates easier to understand if list is added. Chapter 3
34
Binary Search Tree Search tree invariant:
k For all keys k´ in T1 and k´´ in T2: k´ k < k´´ T1 T2 Chapter 3
35
Binary Search Tree Formally: for every tree node v let
key(v) be the key stored at v d(v) the number of children of v Search tree invariant: (as above) Degree invariant: All tree nodes have exactly two children (as long as the number of elements in the list is >1) Key invariant: For every element e in the list there is exactly one tree node v with key(v)=key(e). Chapter 3
36
Binary Search Tree Search tree invariant: (as before)
Degree invariant: All tree nodes have exactly two children (as long as the number of elements is >1) Key invariant: For every element e in the list there is exactly one tree node v with key(v)=key(e). From the search tree and key invariants it follows that for every left subtree T of a node v, the rightmost list element e under T satisfies key(v)=key(e). v T … 1 e Chapter 3
37
search(x) Operation k Search strategy: Start at the root, v, of the search tree while v is a tree node: if x key(v) then let v be the left child of v, otherwise let v be the right child of v Output (list node) v For all keys k´ in T1 and k´´ in T2: k´ k < k´´ T1 T2 Chapter 3
38
search(x) Operation k Correctness of search strategy: For every left subtree T of a node v, the rightmost list element e under T satisfies key(v)=key(e). So whenever search(x) enters T, there is an element e in the list below T with key(e)≥x. For all keys k´ in T1 and k´´ in T2: k´ k < k´´ T1 T2 v T … 1 e Chapter 3
39
Search(9) 10 3 19 1 5 14 28 1 3 5 10 14 19 28 ∞ Chapter 3
40
Insert and Delete Operations
Strategy: insert(e): First, execute search(key(e)) to obtain a list element e´. If key(e)=key(e´), replace e´ by e, otherwise insert e between e´ and its predecessor in the list and add a new search tree leaf for e and e´ with key key(e) whose parent is the old parent of e´. delete(k): First, execute search(k) to obtain a list element e. If key(e)=k, then delete e from the list and the parent v of e from the search tree, and set in the tree node w with key(w)=k: key(w):=key(v). The new parent of the remaining child of v is set to the parent of v. (Thus, a delete can be seen as a reversal of an insert operation.) Chapter 3
41
Insert(5) 10 1 14 28 1 10 14 28 ∞ Chapter 3
42
Insert(5) 10 1 14 5 28 1 5 10 14 28 ∞ Chapter 3
43
Insert(12) 10 1 14 5 28 1 5 10 14 28 ∞ Chapter 3
44
Insert(12) 10 1 14 5 12 28 1 5 10 12 14 28 ∞ Chapter 3
45
Delete(1) 10 1 14 5 12 28 1 5 10 12 14 28 ∞ Chapter 3
46
Delete(1) 10 14 5 12 28 5 10 12 14 28 ∞ Chapter 3
47
Delete(14) 10 14 5 12 28 5 10 12 14 28 ∞ Chapter 3
48
Delete(14) 10 12 5 28 5 10 12 28 ∞ Chapter 3
49
Binary Search Tree Problem: binary tree can degenerate!
Example: numbers are inserted in sorted order Search requires (n) time in the worst case 1 3 10 14 19 5 28 ∞ Chapter 3
50
Binary Search Tree Lowest possible depth for n elements: log n.
Goal: maintain a depth of O(log n). Such a search tree is called balanced. 10 3 19 1 5 14 28 Chapter 3
51
Search Trees Problem: binary tree can degenerate!
Trees that stay balanced: (a,b)-tree (presented here) red-black tree AVL-tree … Gewichtsbalancierter Baum: gut für Bereichsanfragen. Chapter 3
52
(a,b)-Trees Problem: how to maintain a balanced search tree Idea:
All nodes v (except for the root) have degree d(v) with a≤d(v)≤b, where a≥2 and b≥2a-1 (otherwise this cannot be enforced) All leaves have the same depth (which implies that the tree stays balanced) (a,b)-trees, which are a general form of B-trees (see Chapter 18). Chapter 3
53
(a,b)-Trees Formally: for a tree node v let
d(v) be the number of children of v t(v) be the depth of v (root has depth 0) Form Invariant: For all leaves v,w: t(v)=t(w) Degree Invariant: For all inner nodes v except for root: d(v)[a,b], for root r: d(r)[2,b] (as long as #elements >1) Chapter 3
54
(a,b)-Trees Lemma 3.10: An (a,b)-tree with n elements has depth at most 1+⌊loga (n+1)/2⌋ Proof: The root has degree ≥2 and every other inner node has degree ≥a. At depth t there are at least 2at-1 nodes n+1≥2at-1 ⇔ t≤1+⌊loga (n+1)/2⌋ Chapter 3
55
(a,b)-Trees (a,b)-Tree-Rule: Then search operation easy to implement.
s1, s2,…,sd-1 For all keys k in Ti and k´ in Ti+1: k si < k´ T1 T2 Td Chapter 3
56
Search(9) 10 19 14 28 1 3 5 10 14 19 28 ∞ Chapter 3
57
Insert(e) Operation Strategy:
First search(key(e)) until some e´ found in the list. If key(e´)>key(e), insert e in front of e´, otherwise replace e´ by e. . . . e´ . . . ∞ Chapter 3
58
Insert(e) Operation Strategy:
First search(key(e)) until some e´ found in the list. If key(e´)>key(e), insert e in front of e´, otherwise replace e´ by e. . . . e e´ . . . ∞ Chapter 3
59
Insert(e) Operation Add key(e) and pointer to e in tree node v above e´. If we still have d(v)[a,b] after-wards, then we are done. v v … x z … … x y z … . . . x y z . . . . . . x y z . . . e e’ e e’ Chapter 3
60
Insert(e) Operation If d(v)>b, then cut v into two nodes. (Example: a=2, b=4) w … b z … … b u z … split v x u u‘ y x u‘ y e e’ e e’ x u u‘ y z x u u‘ y z Chapter 3
61
Insert(e) Operation If after splitting v, d(w)>b, then cut w into two nodes (and so on, until all nodes have degree ≤b or we reached the root) … r s … … r b s … split w a b u z a u z Chapter 3
62
Insert(e) Operation If for the root v of T, d(v)>b, then cut v into two nodes and create a new root node. b split v a b c d a c d Chapter 3
63
Insert(8) a=2, b=4 10 19 14 28 1 3 5 10 14 19 28 ∞ Chapter 3
64
Insert(8) a=2, b=4 10 19 14 28 1 3 5 8 10 14 19 28 ∞ Chapter 3
65
Insert(8) a=2, b=4 1 5 8 14 28 1 3 5 8 10 14 19 28 ∞ Chapter 3
66
Insert(6) a=2, b=4 1 5 8 14 28 1 3 5 8 10 14 19 28 ∞ Chapter 3
67
Insert(6) a=2, b=4 . . . 1 14 1 3 5 6 8 10 14 19 Chapter 3
68
Insert(7) a=2, b=4 . . . 1 14 1 3 5 6 8 10 14 19 Chapter 3
69
Insert(7) a=2, b=4 . . . 1 14 1 3 5 6 7 8 10 14 19 Chapter 3
70
Insert(7) a=2, b=4 . . . 1 5 7 8 14 1 3 5 6 7 8 10 14 19 Chapter 3
71
Insert(7) a=2, b=4 6 3 10 19 . . . 1 5 7 8 14 1 3 5 6 7 8 10 14 19 Chapter 3
72
Insert Operation Form Invariant: For all leaves v,w: t(v)=t(w) Satisfied by Insert! Degree Invariant: For all inner nodes v except for the root: d(v)[a,b], for root r: d(r)[2,b] 1) Insert splits nodes of degree b+1 into nodes of degree ⌊(b+1)/2⌋ and ⌈(b+1)/2⌉. If b≥2a-1, then both values are at least a. 2) If root has reached degree b+1, then a new root of degree 2 is created. Chapter 3
73
Delete(k) Operation Strategy:
First search(k) until some element e is reached in the list. If key(e)=k, remove e from the list, otherwise we are done. . . . e´ e e´´ . . . ∞ Chapter 3
74
Delete(k) Operation Strategy:
First search(k) until some element e is reached in the list. If key(e)=k, remove e from the list, otherwise we are done. . . . e´ e´´ . . . ∞ Chapter 3
75
Delete(k) Operation Remove pointer to e and key k from the leaf node v above e. (e rightmost child: perform key exchange like in binary tree!) If afterwards we still have d(v)≥a, we are done. v v … x k y … … x y … . . . x k y . . . . . . x y . . . e Chapter 3
76
Delete(k) Operation Remove pointer to e and key k from the leaf node v above e. (e rightmost child: perform key exchange like in binary tree!) If afterwards we still have d(v)≥a, we are done. … y … … k … v v … x y … x . . . x y k . . . . . . x y . . . e Chapter 3
77
Delete(k) Operation If d(v)<a and the preceding or succeeding sibling of v has degree >a, steal an edge from that sibling. (Example: a=2, b=4) u k t u y t v y r s k r s x y r s t x y r s t Chapter 3
78
Delete(k) Operation If d(v)<a and the preceeding and succeeding siblings of v have degree a, merge v with one of these. (Example: a=3, b=5) u y t u t v merge x r s x y r s x y r s t x y r s t Chapter 3
79
Delete(k) Operation Peform changes upwards until all inner nodes (except for the root) have degree ≥a. If root has degree <2: remove root. x y z x y z Chapter 3
80
Delete(10) a=2, b=4 10 19 14 28 1 3 5 10 14 19 28 ∞ Chapter 3
81
Delete(10) a=2, b=4 5 19 1 3 14 28 1 3 5 14 19 28 ∞ Chapter 3
82
Delete(14) a=2, b=4 5 19 1 3 14 28 1 3 5 14 19 28 ∞ Chapter 3
83
Delete(14) a=2, b=4 5 19 1 3 28 1 3 5 19 28 ∞ Chapter 3
84
Delete(14) a=2, b=4 3 19 1 5 28 1 3 5 19 28 ∞ Chapter 3
85
Delete(3) a=2, b=4 3 19 1 5 28 1 3 5 19 28 ∞ Chapter 3
86
Delete(3) a=2, b=4 3 19 1 5 28 1 5 19 28 ∞ Chapter 3
87
Delete(3) a=2, b=4 1 19 5 28 1 5 19 28 ∞ Chapter 3
88
Delete(3) a=2, b=4 19 1 5 28 1 5 19 28 ∞ Chapter 3
89
Delete(1) a=2, b=4 19 1 5 28 1 5 19 28 ∞ Chapter 3
90
Delete(1) a=2, b=4 19 5 28 5 19 28 ∞ Chapter 3
91
Delete(19) a=2, b=4 19 5 28 5 19 28 ∞ Chapter 3
92
Delete(19) a=2, b=4 5 28 5 28 ∞ Chapter 3
93
Delete(19) a=2, b=4 5 28 5 28 ∞ Chapter 3
94
Delete(19) a=2, b=4 5 28 5 28 ∞ Chapter 3
95
Delete Operation Form Invariant: For all leaves v,w: t(v)=t(w) Satisfied by Delete! Degree Invariant: For all inner nodes v except for the root: d(v)[a,b], for root r: d(r)[2,b] 1) Delete merges node of degree a-1 with node of degree a. Since b2a-1, the resulting node has degree at most b. 2) Delete moves edge from a node of degree >a to a node of degree a-1. Also OK. 3) Root deleted: children have been merged, degree of the remaining child is ≥a (and also ≤b), so also OK. Chapter 3
96
Application: External (a,b)-Tree
(a,b)-trees well suited for large amounts of data Internal memory (RAM) size M block size B choose a,b=Q(B) External memory (harddisk) Chapter 3
97
Overview Basic data structures Search structures (successor searching)
Dictionaries (exact searching) Chapter 3
98
Dictionary 8 4 11 20 18 3 Chapter 3
99
Dictionary insert(15) 8 4 11 20 18 15 3 Chapter 3
100
Dictionary delete(20) 8 4 11 20 18 15 3 Chapter 3
101
Dictionary lookup(8) outputs 8 8 4 11 18 15 3 Chapter 3
102
Dictionary lookup(7) outputs ⊥ 8 4 11 18 15 3 Chapter 3
103
Dictionary Data Structure
S: set of elements in data structure Every element e identified by key(e). Operations: S.insert(e: Element): S:=S∪{e} S.delete(k: Key): S:=S\{e}, where e is the element with key(e)=k S.lookup(k: Key): If there is an e∈S with key(e)=k, output e, otherwise output ⊥ Chapter 3
104
Hashing time- and space-efficient hash function h:U → T hash table T 1
3 5 10 14 19 time- and space-efficient hash function h:U → T 14 5 3 19 1 10 hash table T Chapter 3
105
Hashing Task of the hash function: Good spreading of elements across T
1 3 5 10 14 19 Task of the hash function: Good spreading of elements across T 14 5 3 19 1 10 hash table T Chapter 3
106
Hashing 1 3 5 10 14 19 No collisions of elements in T: insert, delete, lookup in O(1) time 14 5 3 19 1 10 hash table T Chapter 3
107
Hashing (no collisions)
Procedure insert(e: Element) T[h(key(e))] := e Procedure delete(k: Key) if key(T[h(k)])=k then T[h(k)] := ⊥ Function lookup(k: Key): Element∪{⊥} return T[h(k)] Chapter 3
108
Problem: handling collisions
Hashing Problem: handling collisions Solutions: Closed addressing: append elements in hash table entry to a list Open addressing: linear, quadratic probing Two hash tables: Cuckoo hashing Chapter 3
109
Open Addressing Linear probing: Store new element x in the first free
1 3 5 10 14 19 Store new element x in the first free slot starting with T[i], where i=h(x.key) 14 5 3 19 1 10 Chapter 3
110
Cuckoo Hashing hash function h1 hash function h2 hash table T1
5 14 Goal: store element in one of its two positions hash function h1 hash function h2 1 14 5 hash table T1 hash table T2 Chapter 3
111
Cuckoo Hashing T1, T2: Array [0..m-1] of Element
Procedure lookup(k: Key): Element∪{⊥} if key(T1[h1(k)])=k then return T1[h1(k)] if key(T2[h2(k)])=k then return T2[h2(k)] return ⊥ Procedure delete(k: Key) if key(T1[h1(k)])=k then T1[h1(k)]:=⊥ if key(T2[h2(k)])=k then T2[h2(k)]:=⊥ Chapter 3
112
Cuckoo Hashing Procedure insert(e: Element) // key(e) already in hash table? if key(T1[h1(key(e))])=key(e) then T1[h1(key(e))] := e; return if key(T2[h2(key(e))])=key(e) then T2[h2(key(e))] := e; return // no, then insert it while true do e ↔ T1[h1(key(e))] // switch e with element in T1 if e=⊥ then return e ↔ T2[h2(key(e))] // switch e with element in T if e=⊥ then return Better: at most d log n often, where the constant d is „sufficiently large“. If more than d log n rounds are needed, then rehash all elements with new h1, h2 Chapter 3
113
Cuckoo Hashing 1 5 10 10 14 1 5 14 Chapter 3
114
Cuckoo Hashing 1 5 10 14 5 1 14 Infinite loop! Chapter 3
115
Cuckoo Hashing Runtime: lookup, delete: O(1) (worst case!)
insert: O(number of replacements + possibly the time for a complete rehash) Runtime of insert operation: 2 cases after t rounds, insert enters an infinite loop, which requires rehash insert terminates after t rounds Nevertheless, expected runtime of insert: O(1) if m2n, where m is the hash table size. Chapter 3
116
Next Lecture Chapter 4 (dynamic programming and Greedy algorithms) skipped this time. We continue with Chapter 5 (Basic graph algorithms). Chapter 3
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.