Introduce Concurrency Threading in Servers

A Recurring Theme Concurrent programming is not for the weak hearted! –Deadlocks, blocked threads, critical regions, … Tactics and Patterns –Ressource pool –Introduce Concurrency Bærbak Christensen2

Thread Pool Thread pool –Keep a fixed size pool of N threads ready –Accept request and Retrieve a thread from pool to process request Return thread to pool after processing has finished –Result N requests can be processed in parallel Bærbak Christensen3

Why Not… Multi-threaded –Accept request and Spawn a new thread Terminate thread after processing has finished Why is this a lesser solution? –Performance (time/space) –Availability Bærbak Christensen4

Java Support ExecutorService Hint: Jenkov has nice tutorial –SocketReactor is modeled over his single-thread srv. Bærbak Christensen5

In Action… Bærbak Christensen6

But… Bærbak Christensen7

Parallel Execution The nasty-ness of shared resources hits us… Let us analyze the flow in SkyCave –Consider that a max peak we have 15 concurrent threads commited to executing requests from 15 clients Can some of these 15 requests represent a request from the same player? Some may include ‘login’s? Bærbak Christensen8

In Reactor s = serverSocket.accept() –readAndDispatch(s) Bærbak Christensen9 Any problems?

In Invoker Bærbak Christensen10

In (Player) Dispatcher Bærbak Christensen11

Ex: digRoom Bærbak Christensen12

MongoDB Phew… –Seems we are safe there… Bærbak Christensen13

Java Concurrency Super fast reiteration Bærbak Christensen14

Monitor Add ‘syncronized’ to a method –Becomes critical region Only one thread allowed to execute at a time –Multiple threads wait in queue for execution If several threads call the method at the same time The ‘queue’ controlling object is the object of the method –i.e. there is a ‘per object’ queue –If the method freeze All queued threads does as well! Bærbak Christensen15

ReentrantLock You can declare a specific ‘queue/lock’ object and synchronize on that –More tedios, but much finer control Locking across a set of objects –Allows timeout!Avoid ‘blocked threads’ Bærbak Christensen16

Idiom Alternative: tryLock(timeout) Bærbak Christensen17 lock.lock(); try { // critical region } finally { lock.unlock(); } lock.lock(); try { // critical region } finally { lock.unlock(); }

Tutorials… Jenkov’s tutorials appear very nice… Bærbak Christensen18

Our Tool Stack Help! Bærbak Christensen19

Blocked Threads! Load Generator, my faulty first version –N players execute M sets of operations Output Bærbak Christensen20

How to diagnose? ‘jvisualvm’ Bærbak Christensen21

Thread Dump Pick VM, Threads Pane, Click Thead Dump Bærbak Christensen22

Review Dump Locate the blocking thread, see stack trace ! Bærbak Christensen23

Monitoring Performance Started ‘load.cave’ Bærbak Christensen24

More Load Started SkyCave app server on small machine Bærbak Christensen25

More Load Started 1 x load.cave of 100 players each Bærbak Christensen26

More Load Started 15 x load.cave of 100 players each Bærbak Christensen27 One server, Approx req/s One server, Approx req/s

The ‘Lyon Airport’ Test Karibu System: Daemon died from exhausted memory Bærbak Christensen28

Going Further Bærbak Christensen29