1. Techniques and Structures in Concurrent Programming Wilfredo Velazquez

2. Outline Basics of Concurrency Concepts and Terminology Advantages and Disadvantages Amdahl’s Law Synchronization Techniques Concurrent Data Structures Parallel Correctness Treading A.P.I.’s

3. Basics of Concurrency  A concurrent program is any in which two or more of its modules or sections are run either by a separate process, or by another thread  Not much attention given historically  Concurrent programs are much more difficult to reason about and implement  Physical limits of modern processors are being reached, Moore’s Law no longer applies  Instead of faster processors, use more of them

4. Concepts and Terminology  Process  A ‘program’, which has its own memory space, stack, etc.  Difficult to communicate between processes –Message Passing Communication  Thread  A ‘sub-program’  Threads share all program features with that of their parent process. That is to say, same memory space, stack, etc.  Easy to communicate between threads –Shared Memory Communication

5. Concepts and Terminology  Concurrent Program  Processes/threads which execute tasks in an ordering relative to each-other that is not defined  Essentially covers all multi-process/multi-threaded programs  Parallelism  Processes/threads that execute completely simultaneously  Parallelism is more readily applied to sections of a program  Impossible in single-core processors (those still exist?)  Increased parallelism = more processors used  Atomic action  An action (instruction) that either happens, completely without interruption, or not at all  For many purposes, the idea that an action ‘looks’ atomic is enough to classify it as such

6. Advantages and Disadvantages  Advantages:  Concurrent Programs + More Processors = Faster Programs  Some problems more easily described in parallel environments  General Multitasking  Non-Determinism  Disadvantages  Concurrent Programs + Few Processors = Slower Programs  Most problems more difficult to implement in parallel environments  Non-Determinism

7. Amdahl’s Law  Relates the speed-up of a program when more processors are added  Has very limiting implications

8. Outline Basics of Concurrency Synchronization Techniques Mutual Exclusion and Locks The Mighty C.A.S. Lock-free and Wait-free Algorithms Transactional Algorithms Concurrent Data Structures Treading A.P.I.’s

9. Synchronization Techniques  These are techniques that assure program correctness in areas where the non-determinism inherited from a concurrent environment would cause undesirable behavior  Example: Let T1 and T2 be threads, x be a shared variable between them  x = 0;//initially  T1::x++;  T2::x++;  Value of x ?

10. Synchronization Techniques x++ becomes read x; add 1; write x; So T1 and T2’s instructions could occur in the following order: T1::read x //reading 0 T2::read x //reading 0 T1::add 1 //0+1 T2::add 1 //0+1 T1::write x //writing 1 T2::write x //writing 1

11. Mutual Exclusion and Locks  Algorithm that allows only one thread to execute a certain ‘area’ of code at a time  It essentially ‘locks out’ all other threads from accessing the area, thus ‘mutex’ and ‘lock’ are typically used synonymously  Varying algorithms exist for implementation, differing in robustness and performance  Typically easy to reason about their use  High overhead compared to other synchronization techniques  Can cause problems such as Deadlock, Livelock, and Starvation

12. The Mighty C.A.S.  Compare And Swap  Native instruction on many modern multiprocessors  Widely used in synchronizing threads  Cheap, compared to using locking algorithms  Expensive, compared to loading-storing as uses a hardware lock  ABA > CAS boolean CAS(memoryLocation, old, new) { If(*memoryLocation == old) { *memoryLocation = new; return true; } return false; }

13. Lock-Free and Wait-Free Algorithms  Wait-Free Algorithm  An algorithm is defined to be ‘wait-free’ if it guarantees that for any number of threads, all of them will make progress in a finite number of steps  Deadlock-free, Livelock-free, Starvation-free  Lock-Free Algorithm  An algorithm is defined to be ‘lock-free’ if it guarantees that for any number of threads, at least one will make progress in a finite number of steps  Deadlock-free, Livelock-free  All wait-free algorithms are also lock-free, though not vice versa  Note that neither definition actually forbids the use of locks, thus a lock-free algorithm could be implemented with locks

14. Transactional Algorithms  Inspired by database systems 1. Gather data from memory locations (optional) 2. Make local changes to the locations 3. Commit changes to the actual locations as an atomic step 4. If commit fails (another transaction occurred), start again  Essentially a generalization of CAS, except that no prior knowledge of the data is needed (for CAS we needed an ‘expected’ value)

15. Outline Basics of Concurrency Synchronization Techniques Concurrent Data Structures Safety and Liveliness Properties Differing Semantics Treading A.P.I.’s

16. Concurrent Data Structures  In sequential programming, data structures are invaluable as programming abstractions as they:  Provide abstraction of the inner-workings via interfaces  Provide a set of properties and guarantees as per what happens when certain operations are performed  Increase modularity of code  In concurrent programming they provide similar benefits, in addition to:  Allows threads to communicate in a simple and maintainable manner  Can be used as a focal point for the work done by multiple threads

17. Safety and Liveliness Properties  Safety  Assures that ‘nothing bad will happen’, for example, two calls to the ‘push’ function of a stack should result in two elements being added to the stack  Liveliness  Assures that progress continues  Deadlock  Livelock  Starvation  All bad!

18. Differing Semantics  Structures must share properties and guarantees with the sequential versions which they mimic, thus their operations must be deterministic (with a few exceptions)  Semantics of use and implementation differ greatly purely due to the concurrent environment  Example:  The result obtained from popping the stack is non- deterministic, even though the implementation of the interfaces themselves are deterministic

19. Differing Semantics  So how can we write the program in such a way that it is well-behaved for our purposes?  De-Facto standard: Use a lock  Parallelism suffers, as other threads may not operate at all during the entire given section of code  Introduces liveliness problems

20. Constructing Concurrent Data Structures  A concurrent data structure must abide by its sequential counter-part’s properties and guarantees when operations are performed on it  It must be ‘thread-safe’, no matter how many parallel calls are made to it, the data structure will never be corrupted  It should be free from any liveliness issues such as Deadlock  Just as sequential ones are constructed for abstraction, concurrent data structures should be opaque in their implementation

21. Constructing Concurrent Data Structures

22.  The sequential version of this data structure  Not suitable as-is for concurrent programming  Lacks any safety properties, though it has no liveliness issues  How can we resolve the issue?  Lock it

23. Constructing Concurrent Data Structures

24.  Safety is no longer a concern, though liveliness now is  Deadlock possible should a thread die during execution  Starvation in case of an interrupt  Lock overhead will overwhelm applications with many pops/push  Look back to original implementation; What sequential assumptions were made? (push)

25. Constructing Concurrent Data Structures  Correct, but original property lost: pushing on to a stack does not always place the element on the stack  Easy solution: Keep trying

26. Constructing Concurrent Data Structures  Pop implemented using the same logic:

27. Outline Basics of Concurrency Synchronization Techniques Concurrent Data Structures Treading A.P.I.’s pthreads M.C.A.S., W.S.T.M., O.S.T.M.

28. Threading API’s  pthreads  C library for multithreading. Contains utilities such as mutexes, semaphores, and others  Available on *nix platforms, though subset ports exist for windows  MCAS  A C API that allows the use of a software-built MCAS (Multiple-Compare-And-Swap) function  Very powerful, though larger overhead than CAS  WSTM  Word-Based Software Transactional Memory  API for easy use of the Transactional Model  Mixes normal objects with WSTM datatypes  Easy to implement on existing systems  OSTM  Object-Based Software Transactional Memory  Similar to WSTM, except that it is more streamlined in its implementation due to operating exclusively on its own data types  More difficult to implement on existing systems

29. Refferences  Concurrent Programming Without Locks  http://research.microsoft.com/en- us/um/people/tharris/papers/2007-tocs.pdf http://research.microsoft.com/en- us/um/people/tharris/papers/2007-tocs.pdf  MCAS, WSTM, OSTM implemented in paper  The art of Pultiprocessor Programming  By Maurice Herlihy, Nir Shavit  http://books.google.com/books?id=pFSwuqtJgxYC&printsec=fr ontcover#v=onepage&q&f=false http://books.google.com/books?id=pFSwuqtJgxYC&printsec=fr ontcover#v=onepage&q&f=false  DCAS is not a Silver Bullet for Nonblocking Algorithm Design  http://labs.oracle.com/scalable/pubs/SPAA04.pdf http://labs.oracle.com/scalable/pubs/SPAA04.pdf