1. NOBLE: A Non-Blocking Inter-Process Communication Library
Håkan Sundell
Philippas Tsigas
Computing Science
Chalmers University of Technology

2. Systems Multi-processor systems: cache-coherent shared memory
UMA
NUMA
Desktop computers

3. Synchronization
A significant part of the work performed by today’s parallel applications is spent on synchronization
Mutual exclusion (Locks)
Blocking
Convoy effects
Deadlocks

4. Convoy effects
The slowdown of one process may cause the whole system to slowdown

5. Research
Non-blocking synchronization has been researched since the 70’s
Lock-free
Wait-free
Non-blocking are based on usage of
atomic synchronization primitives
shared memory

6. Non-blocking Synchronization
Lock-Free Synchronization
Retries until not interfered by other operations
Usually detecting interference by using some kind of shared variable indicating busy-state or similar.
Guarantees live-ness but not starvation-free.
One alternative to using lock is the lock-free method, which is quite simple to use.
Some things that characterize the lock-free method are ...
Change flag to unique value, or remember current state
... do the operation while preserving the active structure ...
Check for same value or state and then validate changes , otherwise retry

7. Non-blocking Synchronization
Wait-free synchronization
All concurrent operations can proceed independently of the others.
Every process always finishes the protocol in a bounded number of steps, regardless of interleaving
No starvation

8. Practice
Non-blocking synchronization is still not used in many practical applications
Non-blocking solutions are often
complex
having non-standard or un-clear interfaces
non-practical
Many results show that non-blocking improves the performance of parallel applications significantly… ? ?

9. Non-blocking Synchronization – Practice
P. Tsigas, Y. Zhang “Evaluating the Performance of Non-Blocking Synchronization on Modern Shared Memory Multiprocessors”, ACM Sigmetrics 2001
The wait-free method has a lot of positive aspects...

10. NOBLE: Brings Non-blocking closer to Practice
Schedule
Goals
Design
Examples
Experiments
Status
Conclusions and Future work

11. Goals
Create a non-blocking inter-process communication interface that have these properties:
Attractive functionality
Programmer friendly
Easy to adapt existing solutions
Efficient
Portable
Adaptable for different programming languages

12. Design: Attractive functionality
Data structures for multi-threaded usage
Queues.
Stacks.
Singly linked lists.
Snapshots.
Data structures for multi-process usage
Shared Register.
Clear specifications
enqueue and dequeue
push and pop
first, next, insert, delete and read
update and scan
read and write

13. Design: Programmer friendly
Hide the complexity as much as possible!
Just one include file
Simple naming convention: Every function is beginning with the NBL characters
#include <Noble.h>
NBLQueueEnqueue() NBLQueueDequeue() …

14. Design: Easy to adapt solutions
Support lock-based as well as non-blocking solutions.
Several different create functions
Unified functions for the operations, independent of the synchronization method 
NBLQueue *NBLQueueCreateLF();
NBLQueue *NBLQueueCreateLB();
NBLQueueFree(handle); NBLQueueEnqueue(handle,item); NBLQueueDequeue(handle);

15. Design: Efficient To minimize overhead, usage of function pointers
In-line redirection
typedef struct NBLQueue { void *data; void (*free)(void *data); void (*enqueue)(void *data,void *item); void *(*dequeue)(void *data); } NBLQueue;
#define NBLQueueFree(handle) (handle->free(handle->data)) #define NBLQueueEnqueue(handle,item) (handle-> enqueue(handle->data,item)) #define NBLQueueDequeue(handle) (handle->dequeue(handle->data))

16. Design: Portable Exported definitions Identical on all platforms
#define NBL...
Noble.h
Identical on all platforms
Platform in-dependent
#include “Platform/Primitives.h” …
QueueLF.c
#include “Platform/Primitives.h” …
StackLF.c
. . .
Platform dependent
SunHardware.asm
IntelHardware.asm
. . .
CAS, TAS, Spin-Locks …
CAS, TAS, Spin-Locks
...

17. Design: Adaptable for different programming languages
Implemented in C, all compiled into a library file.
C++ compatible include files and easy to make C++ wrappers
class NOBLEQueue { private: NBLQueue* queue; public: NOBLEQueue(int type) {if(type==NBL_LOCKFREE) queue=NBLQueueCreateLF(); else … } ~NOBLEQueue() {NBLQueueFree(queue);} inline void Enqueue(void *item) {NBLQueueEnqueue(queue,item);}

18. Examples
First create a global variable handling the shared data object, for example a stack:
#include <noble.h>
...
NBLStack* stack;
Globals
Create the stack with the appropriate implementation:
stack=NBLStackCreateLF(10000);
...
NBLStackFree(stack);
Main
When the data structure is not in use anymore:
NBLStackPush(stack, item); or item=NBLStackPop(stack);
Threads
When some thread wants to do some operation:

19. Examples
#include <noble.h>
...
NBLStack* stack;
Globals
To change the synchronization mechanism, only one line of code has to be changed!
stack=NBLStackCreateLB();
...
NBLStackFree(stack);
Main
NBLStackPush(stack, item); or item=NBLStackPop(stack);
Threads

20. Experiment
Set of random operations performed multithreaded on each data structure, with either low or high contention
Comparing the different synchronization mechanisms and implementations available
Varying number of threads from 1 – 30
Performed on multiprocessors:
Sun Enterprise with 64 CPUs, Solaris
Compaq PC with 2 CPUs, Win32

21. Experiments: Linked List
Lock-Free nr.1 – J. Valois “Lock-Free Data Structures” Ph.D-thesis 1995.
Lock-Free nr.2 - T. Harris “A Pragmatic Implementation of Non-Blocking Linked Lists.” 2001 Symposium on Distributed Computing.
Lock-Based – Spin-locks (Test-And-Set).

22. Experiments: Linked List (high)

23. Experiments: Linked List (low)

24. Experiments: Linked List (high) - Threads

25. Experiments: Queues
Lock-Free nr.1 – J. Valois “Lock-Free Data Structures” Ph.D-thesis 1995.
Lock-Free nr.2 - P. Tsigas, Y. Zhang “A Simple, Fast and Scalable Non-Blocking Concurrent FIFO queue for Shared Memory Multiprocessor Systems”, ACM SPAA’01, 2001.
Lock-Based – Spin-locks (Test-And-Set).

26. Experiments: Queues (high)

27. Experiments: Queues (low)

28. Experiments: Queues (high) - Threads

29. Status Multiprocessor support Extensive Manual
Sun Solaris (Sparc)
Win32 (Intel x86)
SGI (Mips) – Testing phase
Linux (Intel x86) – Testing phase
Extensive Manual
Web site up and running,

30. Conclusions and Future work
NOBLE: Easy to use, efficient and portable
Non-blocking protocols always performs better than or similar to lock-based, especially on multi-processor systems.
To do:
Use in real parallel applications
Extend with more shared data object implementations
Extend to other platforms, especially suitable for real-time systems