1. Advanced Topics in Concurrency and Reactive Programming: Asynchronous Programming
Majeed Kassis

2. Thread Based vs Event Driven Programming
Threads
Procedure-oriented systems
Large number of processes
Shared data
Events
Message-oriented systems
Small number of processes
No data sharing
Thread costs 1MB of overhead on 64bit OS
Actor costs 400Byte of overhead

3. Building Blocks of These Programming Models
thread-based
event-driven
monitor
event handler
scheduling
event loop
exported functions
event types accepted by event handler
returning from a procedure
dispatching a reply
executing a blocking procedure call
dispatching a message, awaiting a reply
waiting on condition variables
awaiting messages

4. Threads: Programming Complexity
Developing correct concurrent code is extremely difficulty
Sharing a single state is the cause of all troubles
Programmers must synchronize and coordinate between threads
Done by using locking primitives
This increases the complexity
Composition Problem:
Given two different components that are thread-safe
Their composition is not always thread-safe!
This also increases the programming complexity
Given two different components that are thread-safe, a composition of them is not thread-safe per se. For instance, placing circular dependencies between multi-threaded components unknowingly can introduce severe deadlocks.

5. Threads: Synchronization Problems
Liveness problems:
Erroneous locking introduces deadlocks or livelocks!
Performance Issues:
Too coarse locks
Slow down concurrent code
This leads to degraded sequential execution performance
Too fine locks
Increase the danger of deadlocks/livelocks
This increases locking overhead
Overhead is any combination of excess or indirect computation time, memory, bandwidth, or other resources that are required to perform a specific task.
These operations are not part of the task itself – but they ensure its success.

6. Threads: Debugging Difficulty
Preemptive scheduling causes nondeterminism
Every execution of threaded code progresses differently
The use of Arbiters causes nondeterminism
A memory arbiter is a device used in a shared memory system to decide, for each memory cycle, which CPU will be allowed to access that shared memory.
Multithreading is error-prone, and very difficult to debug!
Preemptive scheduling and the use of arbiters
The state explosion as a result from all possible interleavings of multiple threads renders a reasonable execution analysis of concurrent code virtually impossible!

7. Threads Coupling Inherit only if you want to override some behavior!
public class MyThread extends Thread {
public MyThread() {
super("MyThread"); }
public void run() {
//Code
//Started with a "new MyThread().start()" call
Thread implementation used to extend Thread class!
Couples the Runnable with its own Thread.
Inherit only if you want to override some behavior!
This job is better suited for an interface implementation!

8. Thread Decoupling
public class MyRunnable implements Runnable {
public void run() {
//Code }
//Started with a "new Thread(new MyRunnable()).start()" call
Decouples Thread creation from Runnable implementation
Allows “feeding” a single Thread multiple Runnables!
What happens when the runnable finishes executing it’s code?
Thread is closed
What is I need to run a new runnable?
New thread is created!
What about communication between threads?

9. Executors: Allows Thread Reuse
Resources are limited
The number of threads running in parallel is always limited
Due to CPU bottleneck, database connection limit, or memory limitations
Thread pools improve resource utilization
The overhead of creating a new thread is significant!
Thread pool is a collection of threads that are created once at start.
Executor Queue
Number of tasks is normally larger than the number of running threads
An executor has a queue containing these tasks

10. Handling Tasks Thread pool has a fixed number of threads
Each task asks for a thread when starting and returns the thread to the pool after finishing.
When there are no available threads in the pool the thread that initiates the task waits till the pool is not empty
Problem?
Not fair
Tasks might use threads for a long time.
Which leads to starvation
This is a scheduling problem!

11. Pool Size A large pool can cause starvation
Each thread consumes resources
memory, management overhead, etc.
A small pool can cause starvation
Incoming tasks wait for a free thread
Therefore, you have to tune the thread pool size:
The number of expected tasks
The required average processing time of the tasks

12. Ideal Pool Size The main goal: 𝑊𝑇 = 𝑒𝑠𝑡𝑖𝑚𝑎𝑡𝑒𝑑 𝑎𝑣𝑒𝑟𝑎𝑔𝑒 𝑤𝑎𝑖𝑡𝑖𝑛𝑔 𝑡𝑖𝑚𝑒
Processing should continue while waiting for slow operations such as I/O
𝑊𝑇 = 𝑒𝑠𝑡𝑖𝑚𝑎𝑡𝑒𝑑 𝑎𝑣𝑒𝑟𝑎𝑔𝑒 𝑤𝑎𝑖𝑡𝑖𝑛𝑔 𝑡𝑖𝑚𝑒
𝑆𝑇 = 𝑒𝑠𝑡𝑖𝑚𝑎𝑡𝑒𝑑 𝑎𝑣𝑒𝑟𝑎𝑔𝑒 𝑝𝑟𝑜𝑐𝑒𝑠𝑠𝑖𝑛𝑔 𝑡𝑖𝑚𝑒 𝑓𝑜𝑟 𝑎 𝑡𝑎𝑠𝑘
About 𝑊𝑇 𝑆𝑇 +1 threads will keep the processor fully utilized
For example, if 𝑊𝑇 is 20𝑚𝑠 and 𝑆𝑇 is 5𝑚𝑠, we will need 5 threads to keep the processor busy.

13. Workers vs Event Queue Workers: Event Queue: Synchronous call
Worker blocks, executes code
Call returns
Event Queue:
Events occur, stored in queue
Process next event, executes callback as call “return”

14. Event Loop – ties everything together!

15. Tools used in conjunction with event-driven programming
First Class Function Support
The ability to pass functions as arguments to other functions,
The ability to return functions as the values from other functions
The ability to assign function to variables or storing them in data structures
Callback Functions
They are passed to the function that acts asynchronously as a parameter.
When the asynchronous operation will finish, the callback will be executed.
Closures
The ability for a first class function to access variables outside its scope when created.
Asynchronous IO
A form of input/output processing that permits other processing to continue before the transmission has finished.

16. Java 8: Lambda Expression & Closure
public Function<Integer, Integer> make_func() {
int n = 0;
return arg -> {
System.out.print(n + " " + arg + ": ");
arg += 1;
return n + arg; }; }
public void main() {
Function<Integer, Integer> x = make_fun();
for(int i = 0; i < 5; i++)
System.out.println(x.apply(i));
Function is generic Java8 interface to implement lambdas
Closure: Accessing “n” from inside the lambda in line 4, 6 is possible for read only. You can’t modify “n”! If “n” is a reference, you can modify the object referenced by it, but can’t change “n” itself! This is Java8 limitation by design.
Callback: “x” is generated function – callback is done on line 12 using apply() (Function interface)

17. Reactor Pattern Channels Selector ThreadPool
Allows Non-blocking IO from and to sockets.
Contains event queue:
3 types of events: ACCEPT, READ, WRITE
Selector
Used to extract events from Channels
Implements event loop:
Handles events in sequential manner
Executes ACCEPT/READ/WRITE operations
Delegates response generation to workers
ThreadPool
Contains workers (threads) Used to handle response generation
Handles: Decoding, tokenization, generating response, encoding response

18. Reactor Pattern – Important Bits
Callbacks are used to change event types the channel is interested in:
When channel is first created: READ
When responses are ready to be sent: READ/WRITE
When there are no responses left to send to client: READ
When “close connection” flag is received: WRITE
Event loop:
Main server loop is an event loop
Loops over events generated in every active channel sequentially
With every client connecting a new channel is especially made for the client
And it joins the even loop.
Selector:
Used to extract events from each active channel
Once a new channel is created it is registered with the selector

19. CODE TIME!!!