Hazard Pointers C++ Memory Ordering Issues Maged Michael Facebook NY Dagstuhl, 21-25 November 2016

Maged Michael - Hazard Pointers References Maged Michael, Hazard Pointers: Safe Memory Reclamation for Lock-Free Objects. IEEE Transactions on Parallel and Distributed Systems. 15 (8): 491–504, June 2004. [P0233R2] Latest version of the proposal to the C++ Standard Committee http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0233r2.pdf Prototype library under facebook/folly/experimental https://github.com/facebook/folly/tree/master/folly/experimental/hazptr Maged Michael - Hazard Pointers

Problems

Running Example: Wide CAS Wide CAS (compare and set) operates atomically on memory locations wider than the width of standard atomic primitives Wide CAS X atomically if (X == u) X = v return true else return false u A common solution: copy-on-write Place wide data in a dynamic block Updates replace the block P u v Maged Michael - Hazard Pointers

Maged Michael - Hazard Pointers Wide CAS Class class WideCAS { class Node { T val_; ... }; atomic<Node*> s_ = {new Node()}; bool compareAndSet(T& u, T& v) { while (true) { Node* p = s_.load(); if (p->val_ != u) return false; Node* n = new Node(v); if (s_.compare_exchange_weak(p, n)) break; delete n; } delete p; return true; }; ABA Problem Unsafe Memory Reclamation Unsafe Memory Access incorrect Maged Michael - Hazard Pointers

Unsafe Memory Reclamation Example (Wide CAS) 1 Thread i reads pointer value A from s_ Node* p = s_.load(); if (p->val_ != u) ... Node* n = new Node(v); if (!s_.cas(p,n)) ... delete p; return true; 1 2 Thread j sets s_ to B and frees A to OS 3 3 Thread i accesses unmapped memory ACCESS VIOLATION s_ u A w B returned to OS Maged Michael - Hazard Pointers

Maged Michael - Hazard Pointers The ABA Problem Example 1 Thread i reads A from s_ 1 Node* p = s_.load(); if (p->val_ != u) ... Node* n = new Node(v); if (!s_.cas(p,n)) ... delete p; return true; 2 2 Thread i reads u from *A 6 3 Thread j sets s_ to B 7 4 Thread j reuses block A to hold value z s_ 5 Thread j sets s_ to A again u A w B 6 Thread i allocates block C to hold value v v C z A Thread i checks that s_ is equal to A CAS succeeds although s_->val_ == z != u 7 INCORRECT OUTCOME Maged Michael - Hazard Pointers

Maged Michael - Hazard Pointers Non-Blocking Progress Guarantees Three levels of non-blocking progress: An operation is wait-free, if whenever a thread executing the operation takes a finite number of steps, the operation must have completed, regardless of the actions/inaction of other threads. An operation is lock-free, if whenever a thread executing the operation takes a finite number of steps, some operation must have completed, regardless of the actions/inaction of other threads. An operation is obstruction-free, if whenever a thread executing the operation takes a finite number of steps alone, the operation must have completed, regardless of where the other threads stopped. Maged Michael - Hazard Pointers

Hazard Pointers

Maged Michael - Hazard Pointers Features Lock-free progress end-to-end Bounded to-be-reclaimed objects No contention among readers Can reclaim cycles Unrestricted reclamation Maged Michael - Hazard Pointers

Maged Michael - Hazard Pointers Protecting Objects A hazard pointer is a single-writer multi-reader pointer Each hazard pointer has one owner (that can write to it) By setting a hazard pointer to the address of an object, the owner is telling all threads: “if you remove this object after the last time I set this hazard pointer to this object don’t reclaim this object until the hazard pointer changes” Maged Michael - Hazard Pointers

Maged Michael - Hazard Pointers Reclaiming Objects 1. Read active hazard pointers. Keep a private copy of non-null values Private copy can be arranged in an efficient search structure e.g., O(1) expected lookup time 2. For each removed object, do a lookup in the private structure Found? Keep the object for a future scan of hazard pointers Not found? Reclaim the object Maged Michael - Hazard Pointers

Maged Michael - Hazard Pointers Wide CAS with Hazard Pointers 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 class WideCAS { class Node : hazptr_obj_base <Node> { T val_: ... }; atomic<Node*> s_ = {new Node()}; ... bool compareAndSet(T& u, T& v) { hazptr_owner<Node> hptr; do { Node* p = p_.load(); hptr.set(p); if (s_.load() != p)) continue; if (p->val_ != u) return false; // access hazard Node* n = new Node(v); if (s_.compare_exchange_weak(p, n)) break; // aba hazard delete n; } while (true); hptr.clear(); delete p; p.retire(); // reclaim when safe return true; } }; Maged Michael - Hazard Pointers

Maged Michael - Hazard Pointers Thread Roles User Owns hazard pointers Protects objects E.g., Traverses linked structures Remover Makes objects unreachable Prohibits the creation of new references to objects E.g., Removes objects from data structures Reclaimer Checks hazard pointers Reclaims objects that are not protected by hazard pointers Maged Michael - Hazard Pointers

Maged Michael - Hazard Pointers Simplest Fully Concurrent Form of Hazard Pointers User Write hp := obj // no need for atomicity Read src == obj <safe use of obj> Write hp := !obj // no need for atomicity Remover / Reclaimer Write src := !obj Read hp != obj //no need for atomicity <reclaim obj> Maged Michael - Hazard Pointers

C++ Memory Ordering

Maged Michael - Hazard Pointers Three-Thread Pattern User hp.store(obj) src.load() == obj <unsafe use of obj> (prohibited) // application hp.store(nullptr) Remover src.store(other,maybe_weak) // application retired.insert(obj) Reclaimer retired.contains(obj) // bulk hp.load() != obj // bulk <reclaim obj> // application Maged Michael - Hazard Pointers

Maged Michael - Hazard Pointers Three-Thread Pattern User hp.store(1) src.load() == 0 obj.load() == 1 (prohibited) // application hp.store(0) Remover src.store(1,maybe_weak) // application retired.store(1) Reclaimer retired.load() == 1 // bulk hp.load() == 0 // bulk obj.store(1,maybe_weak) // application Maged Michael - Hazard Pointers

is this all the needed ordering? Three-Thread Pattern Memory Order Using Herd User hp.store(1,release) fence(seq_cst) src.load(relaxed) == 0 obj.load() == 1 (prohibited) // application hp.store(0,release) Remover src.store(1, maybe_weak) // application fence(seq_cst) retired.store(1,relaxed) Reclaimer retired.load(relaxed) == 1 // bulk fence(seq_cst) hp.load(relaxed) == 0 // bulk fence(acquire) obj.store(1,maybe_weak) // application is this all the needed ordering? Maged Michael - Hazard Pointers

Maged Michael - Hazard Pointers Four-Thread Pattern User hp.store(obj) src.load() == obj <unsafe use of obj> (prohibited) // application hp.store(nullptr) Remover src.store(other,maybe_weak) // application retired.insert(obj) Reclaimer retired.contains(obj) // bulk hp.load() != obj // bulk <reclaim obj> // application Reuser <reallocate obj> // application src.store(obj, release) // application Maged Michael - Hazard Pointers

Maged Michael - Hazard Pointers Four-Thread Pattern User hp.store(1) src.load() == 0 obj.load() == 1 (prohibited) // application hp.store(0) Remover src.store(1,maybe_weak) // application retired.store(1) Reclaimer retired.load() == 1 // bulk hp.load() == 0 // bulk obj.store(1,maybe_weak) // application Reuser obj.load(maybe_weak) == 1 // application obj.store(0,maybe_weak) // application src.store(0, release) // application Maged Michael - Hazard Pointers

Maged Michael - Hazard Pointers Four-Thread Pattern Memory Order Using Herd User hp.store(1,release) fence(seq_cst) src.load(acquire) == 0 obj.load() == 1 (prohibited) // application hp.store(0,release) Remover src.store(1,maybe_weak) // application fence(seq_cst) retired.store(1,relaxed) Reclaimer retired.load(,relaxed) == 1 // bulk fence(seq_cst) hp.load(relaxed) == 0 // bulk fence(acquire) obj.store(1,maybe_weak) // application Reuser obj.load(maybe_weak) == 1 // application obj.store(0,maybe_weak) // application src.store(0, release) // application Maged Michael - Hazard Pointers

Maged Michael - Hazard Pointers Hazard Pointers Functions with Memory Order void set(T* ptr) { hp.store(release); fence(seq_cst); } bool try_protect(T* ptr,atomic<T*>& src) { set(ptr); return src.load(acquire) == ptr; void clear() { hp.store(nullptr,release); User Remover void retire() { fence(seq_cst); retired.insert(this); } Void bulkReclaim() { List objs = retired.extractAll(); // bulk fence(seq_cst); Set h = getHPVals(); // bulk fence(acquire) reclaimUnmatched(objs,hps); // bulk } Reclaimer Maged Michael - Hazard Pointers

Maged Michael - Hazard Pointers Summary of Experience Support for RMW operations is important Automatic generation of valid memory order combinations would be convenient Maged Michael - Hazard Pointers

Thank You

Split Reference Counting atomic_shared_ptr RCU (Read-Copy-Update) Safe Reclamation Solutions Reference Counting shared_ptr Split Reference Counting atomic_shared_ptr Hazard Pointers RCU (Read-Copy-Update) Unreclaimed objects Bounded (+chains) Bounded Unbounded Non-blocking traversal Blocking (or lock-free w/ restrictions) lock-free Lock-free Wait-free Non-blocking reclamation Wait-free (lock-free with reclamation) Blocking Contention among readers Can be very high No contention Traversal speed Atomic updates (~2) Several Atomic updates (~6) Store-load fence No or low overhead Automatic reclamation Yes (restricted if lock-free) Yes No Cycle Reclamation Maged Michael - Hazard Pointers

C++ Interface

Maged Michael - Hazard Pointers Template Library Interface class hazptr_domain; template <typename T, template Deleter = std::default_delete<T>> hazptr_obj_base; template <typename T> hazptr_owner; Maged Michael - Hazard Pointers

Maged Michael - Hazard Pointers hazptr_domain class hazptr_domain { public: constexpr explicit hazptr_domain( std::pmr::memory_resource* /*C++17*/ = std::pmr::get_default_resource()) noexcept; ~hazptr_domain(); }; hazptr_domain& default_hazptr_domain(); Maged Michael - Hazard Pointers

Maged Michael - Hazard Pointers hazptr_obj_base template <typename T, template Deleter = std::default_delete<T>> hazptr_obj_base { public: void retire(hazptr_domain& domain = default_hazptr_domain(), Deleter reclaim = {}); }; Maged Michael - Hazard Pointers

Maged Michael - Hazard Pointers hazptr_owner template <typename T> class hazptr_owner { public: /* Automatically acquire a hazard pointer */ explicit hazptr_owner( hazptr_domain& domain = default_hazptr_domain()); /* Automatically clear and release the owned hazard pointer */ ~hazptr_owner(); /** Hazard pointer operations */ /* Return true if successful in protecting the object. * Otherwise set ptr to src */ bool try_protect(T*& ptr, const std::atomic<T*>& src) noexcept; /* Get a protected reference from a source */ T* get_protected(const std::atomic<T*>& src) noexcept; /* Set the hazard pointer to ptr */ void set(const T* ptr) noexcept; /* Clear the hazard pointer */ void clear() noexcept; Maged Michael - Hazard Pointers

Maged Michael - Hazard Pointers hazptr_owner continued /* Swap ownership of hazard pointers with another hazptr_owner. * The owned hazard pointers remain unmodified during the swap and * continue to protect the respective objects that they were * protecting before the swap, if any. */ void swap(hazptr_owner<T>&) noexcept; }; Template <typename T> void swap(hazptr_owner<T>&, hazptr_owner<T>&) noexcept; Maged Michael - Hazard Pointers

Maged Michael - Hazard Pointers Wide CAS with Hazard Pointers Template Interface class WideCAS { class Node : hazptr_obj_base <Node> { T val_: ... }; atomic<Node*> s_ = {new Node()}; ... bool compareAndSet(T& u, T& v) { do { Node* p = s_.load(); hazptr_owner<Node> hptr; if (!hptr.try_protect(p, s_)) continue; if (p->val_ != u) return false; // access hazard Node* n = new Node(v); if (s_.compare_exchange_weak(p, n)) break; // aba hazard delete n; // Automatically clear and release the owned hazard pointer. } while (true); p.retire(); return true; } }; Maged Michael - Hazard Pointers

Maged Michael - Hazard Pointers Search Ordered Singly Linked List bool contains(T val) { // Acquire two hazard pointers for hand-over-hand traversal. hazptr_owner<Node> hptr_prev, hptr_curr; T elem; bool done = false; while (!done) { std::atomic<Node*>* prev = &head_; Node* curr = prev->load(); while (true) { if (!curr) { return false; } if (!hptr_curr.try_protect(curr, *prev)) break; Node* next = curr->next_.load(); // access hazard elem = curr->elem_; // access hazard if (prev->load() != curr) break; // aba hazard if (elem >= val) { done = true; break; } prev = &(curr->next_); curr = next; // hand-over-hand swap(hptr_curr, hptr_prev); } return elem == val; // The hazard pointers are released automatically. Maged Michael - Hazard Pointers