Skiplist-based Concurrent Priority Queues Itay Lotan Stanford University Nir Shavit Sun Microsystems Laboratories

2 Priority Queues For Large-Scale MultiProcessor Machines Priority Queue – A data structure that allows n asynchronous processes to insert an item, or delete the item with highest priority Large Scale Machine – Hundreds of Processors –Usually different architecture than small scale machines

3 Fixed Set Priority Queue A priority queue with a fixed and predetermined set of priorities Used by operating systems to distribute CPU time. Funnel Tree [Shavit, Zemach]There is a scalable solution - Funnel Tree [Shavit, Zemach]

4 General Priority Queue Supports an unlimited range of priorities Between any two priorities may be another Useful for –Heuristic searches –Graph searches HeapHunt et al Best solution – Heap algorithm of Hunt et al binary search treesWorks better than binary search trees

5 Hunt et al Heaps [Hunt et al]

6 Scalability Problem of Heaps

7 New Approach - SkipQueues Skip ListPughBased on Skip List of Pugh All locking is distributed and local Balancing of structure is probabilistic Deletions evenly distributed, minimal contention No need to pre-allocate memory All operations in expected logarithmic time

8 Coming up Next … Skip Lists Pugh 1.The inside story on concurrent Skip Lists [Pugh] SkipQueue 2.How to make a SkipQueue 3.Experimental Results

9 Skip List Structure [Pugh]

10 Inserting an Element InsertInsert (key_t k): –Randomly choose a level l –Find at each level of the list up to l the largest key smaller than k –For each level from 1 to l Acquire a lock on the item found before Insert the new key after it Release lock

11 Skip List - Insertion New Item

12 Deleting an Element DeleteDelete (key_t k) –Find k at all levels at which it appears. –For each level from l to 1 Acquire locks on k and the item before it. Remove k Release locks

13 Skip List - Deletion

14 SkipQueues Delete_MinProblem: How to allow Delete_Min operations without creating a bottle neck around the first element?

15 Our Solution Skip ListObservation - The lowest level of the Skip List contains all the elements in ascending order Processes can advance down this list and “logically” delete the first available key Each process can then delete the key that it previously deleted “logically”

16 Delete_Min Operation Skip ListTraverse the bottom level of the Skip List structure For each traversed Item –Try to SWAP its Deleted flag –If it was not already deleted, return the key of the item –Else, go on to next item If at the end of the list, return EMPTY No locking at all

17 From Skip List to SkipQueue

18 Performance Benchmarks We compared the performance of 3 structures: HeapHunt et al 1.The Heap structure of Hunt et al 2.FunnelList combining-funnel Shavit, Zemach 2.FunnelList – a simple linked list, access to which is governed by a combining-funnel [Shavit, Zemach] structure 3.SkipQueue 3.SkipQueue structure as described before

19 Benchmark Methodology Proteus Brewer et alWe used the Proteus multiprocessor simulator by Brewer et al We simulated a 256 processors machine similar to the MIT Alewife Processors alternate between performing small amount of work and accessing the queue.

20 Benchmark Methodology Cont’d Processors randomly choose whether to insert or delete Priorities of inserted items are chosen uniformly at random Insert Delete_MinWe measured the average latency of Insert and Delete_Min operations

21 Small Structure - Deletions

22 Small Structure - Insertions

23 Large Structure - Deletions

24 Large Structure - Insertions

25 Large Structure Benchmark With 70% Deletions - Deletions

26 Large Structure Benchmark With 70% Deletions - Insertions

27 Conclusions SkipQueues HeapsSkipQueues scale significantly better than Heaps SkipQueuesSkipQueues are highly distributed – no hot spots or bottle necks Deletes are 3 times faster and Inserts are 10 times faster when concurrency reaches 256 processors Future Directions –Implementation without locks –Experimenting with other data structures

28 Delete_Min Cont’d In order to assure a stronger ordering property we added the following: –Each item is time–stamped when its insertion is completed Delete_Min –Each Delete_Min operation notes the time at which it begins Delete_Min –The Delete_Min operation will only try to “logically” delete items that were inserted before it started.

29 Serialization Order Delete_MinEach successful Delete_Min is ordered at the time its “logical” deletion was completed Delete_Min EMPTYEach unsuccessful Delete_Min that returned EMPTY is ordered at the time of its return instruction Delete_MinEach uncompleted Delete_Min is ordered at the time of its invocation

30 SkipQueue Specification Delete_MinFor every Delete_Min operation –Let I be the set of keys inserted by Insert operations serialized before it Delete_Min –Let D be the set of keys removed by Delete_Min operations serialized before it EMPTYThe returned value is the smallest in the set I – D, or EMPTY if I – D is empty.