Multicore programming Unordered sets: concurrent hash tables Week 3 - Monday Trevor Brown http://tbrown.pro

Last time Fixed size, probing hashtables With deletion (using tombstones)

How tombstones fix our problem Insert(7) hashes to 2 Insert(3) hashes to 5 7 9 3 Insert(9) hashes to 2 Delete(7) hashes to 2 Delete(9) hashes to 2

Downsides of tombstones? TOMBSTONEs are never cleaned up Cleaning them up seems hard Cannot remove a TOMBSTONE until there are no keys in the table that probed past the TOMBSTONE when they were inserted Thought: if we do table expansion, there’s no need to copy over TOMBSTONEs… 9 3 Probes when 9 was inserted

Efficiently Expanding a hash table Concurrent Hash Tables: Fast and General(?)! [MSD2016] Issues to solve When to expand? Too much probing? Estimate of table size vs capacity? Which threads will perform the expansion? Enslaving user threads that access the table? Custom thread pool? How to expand? Block updates when expansion is happening? Always allow updates? Hard to always allow updates “we are fine with small amounts of (b)locking, as long as it improves the overall performance” (a healthy attitude)

High level idea When estimated size > 50% of table size, expand by 2x & migrate contents into the new table Naïve idea: global lock on table, copy with one thread --- inefficient! Partition old table into chunks (default size 4096) Numbered 0, 1, 2, …, floor(capacity / 4096) Threads use fetch & add to claim chunks they will migrate Migrate a chunk to the new table by performing insert() on each key

What if expansion is Concurrent with updates? 1 2 Thread 1: F&A to claim a chunk chunksClaimed Thread 2: F&A to claim a chunk Thread 1 Thread 2 Thread 1: see bucket 0 is empty Thread 1: copy bucket 1 (val 6) Old table 9 6 3 4 2 Thread 3: insert 9 at index 0 Threads 1&2: finish expansion Lost key 9! Must prevent updates to cells we’ve already migrated! New table 6 3 4 2

Solution: marking Steal a bit from each key field Use it to store a mark Invariant: a bucket with a marked key cannot be modified In expansion, mark each bucket before migrating it currKey = bucket->key CAS(bucket->key, currKey, currKey | MARKED_MASK) In insert/delete, before performing CAS(bucket->key, currKey, newKey), must verify that old is not marked (otherwise, help with expansion) Could we use a write to mark the bucket, instead of CAS? unmarked marked 7 7 X

How marking helps 1 2 Thread 1: F&A to claim a chunk chunksClaimed Thread 1: F&A to claim a chunk chunksClaimed Thread 2: F&A to claim a chunk Thread 1 Thread 2 Thread 1: mark bucket 0 Thread 1: mark & copy bucket 1 Old table 6 3 4 2 X X Thread 3: try insert(9) @ idx0 CAS(&bucket[0]->key, <NULL, unmarked>, <9, unmarked>) CAS FAILS! New table 6 Thread 3: help expansion (if there are any unclaimed chunks) After expansion, retry insert(9)

Detecting table > 50% full Approximate counter: Global data: volatile int size Per-thread data: int numInserts When a thread inserts, it increments its own private numInserts After a thread does Ω(𝑛𝑢𝑚𝑇ℎ𝑟𝑒𝑎𝑑s) increments, it does Fetch&Add(&size, numInserts) then sets its private numInserts to 0 How far off can size be from the true value? 𝐸𝑟𝑟𝑜𝑟=𝑐⋅𝑛𝑢𝑚𝑇ℎ𝑟𝑒𝑎𝑑 𝑠 2 Insignificant for large tables (for c=1, 100 threads, error is 10000: 1% of 1M keys) Could we do better? Reduces contention on size vs F&A every time

Implementation sketch With expansion struct hashmap 1 volatile char padding0[64]; 2 table * volatile currentTable; 3 volatile char padding1[64]; /* code for operations ... */ Without expansion struct hashmap 1 volatile char padding0[64]; 2 volatile uint64_t * data; 3 const int capacity; 4 volatile char padding1[64]; /* code for operations ... */ struct table 1 volatile char padding0[64]; 2 volatile uint64_t * data; 3 volatile uint64_t * old; 4 volatile int chunksClaimed; 5 volatile int size; 6 const int capacity; 7 volatile char padding1[64];

Implementation sketch Check if expansion is in progress, and help if so. Otherwise, create new struct table and CAS it into currentTable. int hashmap::insert(int key) table * t = currentTable; int h = hash(key); for (int i=0;i<capacity;++i) { if (isTooFull(t)) { tryExpand(); return insert(key); } int index = (h+i) % t->capacity; int found = t->data[index]; if (found & MARKED_MASK) { helpExpand(); return insert(key); } else if (found == key) return false; else if (found == NULL) { if (CAS(&t->data[index], NULL, key)) return true; else { found = t->data[index]; } } } assert(false); Check if expansion is in progress, and help if so. Check if expansion is in progress, and help if so. Could fail CAS become someone inserted, or because someone marked

Summary Deletion in concurrent hash tables Expansion of concurrent hash tables Padding to avoid false sharing in the hash table At the start & end of the hash table struct/class At the start & end of the data array Not padding each bucket!