1. Multicore programming
Expanding hash tables, and synchronization primitives
Week 3 – Wednesday
Trevor Brown

2. Last time Efficiently expanding a hash table
Marking buckets to prevent errors
Approximate counters for detecting table > 50% full

3. Implementation sketch
With expansion
struct hashmap
1 volatile char padding0[64];
2 table * volatile currentTable;
3 volatile char padding1[64];
/* code for operations ... */
Without expansion
struct hashmap
1 volatile char padding0[64];
2 volatile uint64_t * data;
3 const int capacity;
4 volatile char padding1[64];
/* code for operations ... */
struct table
1 volatile char padding0[64];
2 volatile uint64_t * data;
3 volatile uint64_t * old;
4 volatile int chunksClaimed;
5 volatile int size;
6 const int capacity;
7 volatile char padding1[64];

4. Implementation sketch
Check if expansion is in progress, and help if so.
Otherwise, create new struct table and CAS it into currentTable.
int hashmap::insert(int key)
table * t = currentTable;
int h = hash(key);
for (int i=0;i<capacity;++i) {
if (isTooFull(t)) { tryExpand(); return insert(key); }
int index = (h+i) % t->capacity;
int found = t->data[index];
if (found & MARKED_MASK) { helpExpand(); return insert(key); }
else if (found == key) return false;
else if (found == NULL) {
if (CAS(&t->data[index], NULL, key)) return true;
else {
found = t->data[index];
} } }
assert(false);
Check if expansion is in progress, and help if so.
Check if expansion is in progress, and help if so.
Could fail CAS become someone inserted, or because someone marked

5. Making migration more efficient
Normal function to get bucket index from key:
index = hash(key) % capacity
When capacity doubles, indexes of keys are scrambled
% 12 = % 24 = 15 bucket 3  bucket 15
% 12 = % 24 = 1 bucket 2  bucket 1
Scaled index function
index = floor( hash(key) / |hash universe| * capacity)
When capacity doubles, indexes of keys are doubled
bucket 3 -> bucket 6 bucket 2  bucket 4 bucket 15  bucket 30
With predictable indexes, can expand much more efficiently!

6. idea Old table 7 6 3 4 2 One thread can copy without synchronization 7
New table

7. More complex data structures

8. What else is worth understanding?
We’ve seen hash tables…
What about node based data structures?
(That aren’t just a single pointer like stacks, or two pointers like queues)
Singly-linked lists, doubly-linked lists, skip-lists, trees, tries, hash tries, …
New challenges:
Nodes get deleted when threads might be trying to work on them
Operations may require atomic changes to multiple nodes

9. Lock-free singly-Linked lists: Attempting to use CAS
Ordered set implemented with singly-linked list
Delete(15)
Traverse list, then CAS next from to
Insert(17)
Traverse list, create node , then CAS next from to 7 15 20 17 7 20 17
head 7 15 20 17

10. The problem What if the operations are concurrent?
Delete(15): pause just before CAS next from to
Insert(17): traverse list, create node , then CAS next from to
Delete(15): resume and CAS next from to 7 15 20 17 15 20 17 7 15 20
head 7 15 20
Erroneously deleted 17! 17

11. Solution: marking [Harris2001]
Idea: prevent changes to nodes that will be deleted
Before deleting a node, mark its next pointer
How does this fix the Insert(17), Delete(15) example?
Delete(15) marks before using CAS to delete it
Insert(17) cannot modify next because it is marked 15
Okay.
We can do lists. 15
head 7 15 20 17

12. What about removing several nodes?
Deleting consecutive nodes in a list…
Delete(15 AND 20)
Mark 15, then mark 20?
What can go wrong…
Or performing rotations in trees by replacing nodes…
head 7 15 20 27 D D A A B C B C

13. Or changing two pointers at once?
Doubly-linked list
Insert(17)
If the two pointer changes are not atomic
Insertions and deletions could happen between them
Example: after 15.next := 17, but before 20.pred := 17, someone inserts between 17 and 20 7 15 20 17

14. Easy Lock-based solution
Doubly-linked list
Insert(17)
Simplest locking discipline
Never access anything without locking it first
Correct, but at what cost?
To respect the locking discipline, we have to lock while searching!
pred
succ 7 15 20 17

15. Can we search without locks?
Insert(k):
Search without locking until we reach nodes pred & succ where pred.key < k <= succ.key
If we found k, return false
Lock pred, lock succ
If pred.next != succ, unlock and retry
Create new node n
pred.next = n
succ.prev = n
Unlock all
Insert(17)
pred
succ 7 15 20 17
Contains(k):
pred = head
succ = head
Loop
If succ == NULL or succ.key > k then return false
If succ.key == k then return true
succ = succ.next
Where do we linearize?

16. What if we have different types of searches?
Could imagine an application that wants a doubly linked list so:
Some threads can search left-to-right (containsLR)
Some threads can search right-to-left (containsRL)
When do we linearize insertions in such an algorithm?

17. Bi-directional searches really complicate linearization…
Insert(17)
Insert(k):
Search without locking until we reach nodes pred & succ where pred.key < k <= succ.key
If we found k, return false
Lock pred, lock succ
If pred.next != succ, unlock and retry
Create new node n
pred.next = n
succ.prev = n
Unlock all
pred
succ 7 15 20 17
Where should we linearize?
When we set pred.next?
What if p is searching for 17 from left to right and q is searching for 17 from right to left?
p sees 17 and returns true, q does not
q must see 17 if we’ve linearized the insert
When we set succ.prev?
p shouldn’t see 17 until succ.prev has been set…

18. Summary Implementation of hash table Insert with expansion
Making expansion more efficient (scaled index function)
Singly- and doubly-linked lists
Main challenges in node-based data structures
Preventing changes to deleted nodes
Atomically modifying two or more variables
(Hard if you have searches that do not lock!)