Advanced Topics in Concurrency and Reactive Programming: Asynchronous Programming Majeed Kassis

Thread Based vs Event Driven Programming Threads Procedure-oriented systems Large number of processes Shared data Events Message-oriented systems Small number of processes No data sharing Thread costs 1MB of overhead on 64bit OS Actor costs 400Byte of overhead

Building Blocks of These Programming Models thread-based event-driven monitor event handler scheduling event loop exported functions event types accepted by event handler returning from a procedure dispatching a reply executing a blocking procedure call dispatching a message, awaiting a reply waiting on condition variables awaiting messages

Threads: Programming Complexity Developing correct concurrent code is extremely difficulty Sharing a single state is the cause of all troubles Programmers must synchronize and coordinate between threads Done by using locking primitives This increases the complexity Composition Problem: Given two different components that are thread-safe Their composition is not always thread-safe! This also increases the programming complexity Given two different components that are thread-safe, a composition of them is not thread-safe per se. For instance, placing circular dependencies between multi-threaded components unknowingly can introduce severe deadlocks.

Threads: Synchronization Problems Liveness problems: Erroneous locking introduces deadlocks or livelocks! Performance Issues: Too coarse locks Slow down concurrent code This leads to degraded sequential execution performance Too fine locks Increase the danger of deadlocks/livelocks This increases locking overhead Overhead is any combination of excess or indirect computation time, memory, bandwidth, or other resources that are required to perform a specific task. These operations are not part of the task itself – but they ensure its success.

Threads: Debugging Difficulty Preemptive scheduling causes nondeterminism Every execution of threaded code progresses differently The use of Arbiters causes nondeterminism A memory arbiter is a device used in a shared memory system to decide, for each memory cycle, which CPU will be allowed to access that shared memory. Multithreading is error-prone, and very difficult to debug! Preemptive scheduling and the use of arbiters The state explosion as a result from all possible interleavings of multiple threads renders a reasonable execution analysis of concurrent code virtually impossible!

Threads Coupling Inherit only if you want to override some behavior! 1 2 3 4 5 6 7 89 public class MyThread extends Thread { public MyThread() { super("MyThread"); } public void run() { //Code //Started with a "new MyThread().start()" call Thread implementation used to extend Thread class! Couples the Runnable with its own Thread. Inherit only if you want to override some behavior! This job is better suited for an interface implementation!

Thread Decoupling 1 2 3 4 5 6 public class MyRunnable implements Runnable { public void run() { //Code } //Started with a "new Thread(new MyRunnable()).start()" call Decouples Thread creation from Runnable implementation Allows “feeding” a single Thread multiple Runnables! What happens when the runnable finishes executing it’s code? Thread is closed What is I need to run a new runnable? New thread is created! What about communication between threads?

Executors: Allows Thread Reuse Resources are limited The number of threads running in parallel is always limited Due to CPU bottleneck, database connection limit, or memory limitations Thread pools improve resource utilization The overhead of creating a new thread is significant! Thread pool is a collection of threads that are created once at start. Executor Queue Number of tasks is normally larger than the number of running threads An executor has a queue containing these tasks

Handling Tasks Thread pool has a fixed number of threads Each task asks for a thread when starting and returns the thread to the pool after finishing. When there are no available threads in the pool the thread that initiates the task waits till the pool is not empty Problem? Not fair Tasks might use threads for a long time. Which leads to starvation This is a scheduling problem!

Pool Size A large pool can cause starvation Each thread consumes resources memory, management overhead, etc. A small pool can cause starvation Incoming tasks wait for a free thread Therefore, you have to tune the thread pool size: The number of expected tasks The required average processing time of the tasks

Ideal Pool Size The main goal: 𝑊𝑇 = 𝑒𝑠𝑡𝑖𝑚𝑎𝑡𝑒𝑑 𝑎𝑣𝑒𝑟𝑎𝑔𝑒 𝑤𝑎𝑖𝑡𝑖𝑛𝑔 𝑡𝑖𝑚𝑒 Processing should continue while waiting for slow operations such as I/O 𝑊𝑇 = 𝑒𝑠𝑡𝑖𝑚𝑎𝑡𝑒𝑑 𝑎𝑣𝑒𝑟𝑎𝑔𝑒 𝑤𝑎𝑖𝑡𝑖𝑛𝑔 𝑡𝑖𝑚𝑒 𝑆𝑇 = 𝑒𝑠𝑡𝑖𝑚𝑎𝑡𝑒𝑑 𝑎𝑣𝑒𝑟𝑎𝑔𝑒 𝑝𝑟𝑜𝑐𝑒𝑠𝑠𝑖𝑛𝑔 𝑡𝑖𝑚𝑒 𝑓𝑜𝑟 𝑎 𝑡𝑎𝑠𝑘 About 𝑊𝑇 𝑆𝑇 +1 threads will keep the processor fully utilized For example, if 𝑊𝑇 is 20𝑚𝑠 and 𝑆𝑇 is 5𝑚𝑠, we will need 5 threads to keep the processor busy.

Workers vs Event Queue Workers: Event Queue: Synchronous call Worker blocks, executes code Call returns Event Queue: Events occur, stored in queue Process next event, executes callback as call “return”

Event Loop – ties everything together!

Tools used in conjunction with event-driven programming First Class Function Support The ability to pass functions as arguments to other functions, The ability to return functions as the values from other functions The ability to assign function to variables or storing them in data structures Callback Functions They are passed to the function that acts asynchronously as a parameter. When the asynchronous operation will finish, the callback will be executed. Closures The ability for a first class function to access variables outside its scope when created. Asynchronous IO A form of input/output processing that permits other processing to continue before the transmission has finished.

Java 8: Lambda Expression & Closure 1 2 3 4 5 6 7 8 9 10 11 12 13 public Function<Integer, Integer> make_func() { int n = 0; return arg -> { System.out.print(n + " " + arg + ": "); arg += 1; return n + arg; }; } public void main() { Function<Integer, Integer> x = make_fun(); for(int i = 0; i < 5; i++) System.out.println(x.apply(i)); Function is generic Java8 interface to implement lambdas Closure: Accessing “n” from inside the lambda in line 4, 6 is possible for read only. You can’t modify “n”! If “n” is a reference, you can modify the object referenced by it, but can’t change “n” itself! This is Java8 limitation by design. Callback: “x” is generated function – callback is done on line 12 using apply() (Function interface)

Reactor Pattern Channels Selector ThreadPool Allows Non-blocking IO from and to sockets. Contains event queue: 3 types of events: ACCEPT, READ, WRITE Selector Used to extract events from Channels Implements event loop: Handles events in sequential manner Executes ACCEPT/READ/WRITE operations Delegates response generation to workers ThreadPool Contains workers (threads) Used to handle response generation Handles: Decoding, tokenization, generating response, encoding response

Reactor Pattern – Important Bits Callbacks are used to change event types the channel is interested in: When channel is first created: READ When responses are ready to be sent: READ/WRITE When there are no responses left to send to client: READ When “close connection” flag is received: WRITE Event loop: Main server loop is an event loop Loops over events generated in every active channel sequentially With every client connecting a new channel is especially made for the client And it joins the even loop. Selector: Used to extract events from each active channel Once a new channel is created it is registered with the selector

CODE TIME!!!