Scalable Flat-Combining Based Synchronous Queues Danny Hendler, Itai Incze, Nir Shavit and Moran Tzafrir Presentation by Uri Golani

Overview Synchronous queue Synchronous queue using single combiner flat combining Synchronous queue using Parallel flat combining Benchmarks

Overview Synchronous queue Synchronous queue using single combiner flat combining Synchronous queue using Parallel flat combining Benchmarks

Synchronous queue Suited for Handoff designs Each put must wait for a get and vice verca. No capacity Does not permit null elements Does not impose order (unfair)

Overview Synchronous queue Synchronous queue using single combiner flat combining Synchronous queue using Parallel flat combining Benchmarks

So what is flat combining? It means that all requests are laid on a sequential data structure and are combined by traversing it.

F.C Algorithms Attributes: Publication record – per thread Publication list Global lock Private stack Count

A Threads Publication Record : Requ est Item Publication Record Is linked

Publication list A list of containing a static head and publication records A thread has at most one P.R in the list Adding a P.R to the list involves a CAS to the head of the list. Removing P.R doesnt include CAS therefore head.next will never be removed.

Global lock Enables one thread only to traverse the publication list and act as a combiner Publication records can still be added to the publication list even when the lock is taken.

Counter When grabbing the lock and becoming a combiner a thread increments this field. Every predefined number of increments a clean up of old requests from the publication list takes place

Private stack A construct member. Not issued per combiner Stores Push/Pop operations during the traversal of the combiner Keeps the overflow of requests for the next combining rounds

FC Synchronous queue – get/put methods 1.Allocate a publication Record if it is null 2.while(true) 2.1 check that P.R is still active and in the publication list. CAS it to the head of the publication list otherwise. 2.2 try to acquire the lock (CAS) to become combiner and Combine() if succeeded 2.3 check If the publication records item is not null, if so, break.

Combine() 1. increment count 2. for (COMBINING_ROUND) 2.1 traverse the publication list combining complimentary requests. Pushing overflows into the stack 2.2 if (count % CLEAN_UP = 0) remove old P.R

Single Combiner Overlay : Requ est Item Requ est Item Requ est Item Requ est Item 3. Combiner traverses list collecting requests into stack and matching them to other requests along the list Hea d Cou nt Thread C Thread A Thread B Thread D 4. infrequently, new records are CASed by threads to head of list, and old ones are removed by combiner 2. thread acquires lock, becomes combiner, updates count Combiners private stack publicatio n list 1. thread writes push or pop request and spins on local record Push Po p Pu sh

Overview Synchronous queue Synchronous queue using single combiner flat combining Synchronous queue using Parallel flat combining Benchmarks

Parallel flat combining Uses two levels of combining: 1.Dynamic level – Multiple combiners working in parallel 2.Exchange level – Combining overflows of the dynamic level in a single combiner form

Dynamic level Publication list is divided to sub lists of limited size Each sub list is issued a combiner node head, has its own lock, and private stack Multiple combiners can work on the sub lists, and thus the work on the publication list is done in parallel

Exchange level Has a publication list, private stack and lock Each publication record represents one sub list in the dynamic level Publication records item consists of a list of overflow requests (gets/puts) from the dynamic level Combining is done using a single combiner

Parallel Combiner Overlay: Reques t Item Reques t Item Reques t Item Reques t Item Reques t Item Reques t Item 1 st combiner node Reques t Item Threa d C Combiner Node Threa d A Reques t Item Thread G Reques t Item Threa d B Reques t Item Thread D Reques t Item Threa d E Combiner Node 2 nd combiner node 3 rd combiner node Combiner Node Coun t privat e stack Coun t privat e stack Head of the dynamic FC publication list 2 nd combiner Reques t Item 3 rd combiner Reques t Item 1 st combiner Reques t Item Head of the exchange FC publication list Cou nt privat e stack Reques t overflo ws

Is parllel flat combining linearizable? Operations can be linearized at the point of release of the first of two combined requests. It can be viewed as an object whose history is made of a sequence of pairs consisting of a push followed by a pop (i.e. push, pop, push, pop...) Well, it is.

Overview Synchronous queue Synchronous queue using single combiner flat combining Synchronous queue using Parallel flat combining Benchmarks

Pitfalls of parallel flat combining: Performance is highly based on the balance of requests type on the various sub lists