1. Introduce Concurrency Threading in Servers

2. A Recurring Theme Concurrent programming is not for the weak hearted! –Deadlocks, blocked threads, critical regions, … Tactics and Patterns –Ressource pool –Introduce Concurrency CS@AUHenrik Bærbak Christensen2

3. Thread Pool Thread pool –Keep a fixed size pool of N threads ready –Accept request and Retrieve a thread from pool to process request Return thread to pool after processing has finished –Result N requests can be processed in parallel CS@AUHenrik Bærbak Christensen3

4. Why Not… Multi-threaded –Accept request and Spawn a new thread Terminate thread after processing has finished Why is this a lesser solution? –Performance (time/space) –Availability CS@AUHenrik Bærbak Christensen4

5. Java Support ExecutorService Hint: Jenkov has nice tutorial –SocketReactor is modeled over his single-thread srv. CS@AUHenrik Bærbak Christensen5

6. In Action… CS@AUHenrik Bærbak Christensen6

7. But… CS@AUHenrik Bærbak Christensen7

8. Parallel Execution The nasty-ness of shared resources hits us… Let us analyze the flow in SkyCave –Consider that a max peak we have 15 concurrent threads commited to executing requests from 15 clients Can some of these 15 requests represent a request from the same player? Some may include ‘login’s? CS@AUHenrik Bærbak Christensen8

9. In Reactor s = serverSocket.accept() –readAndDispatch(s) CS@AUHenrik Bærbak Christensen9 Any problems?

10. In Invoker CS@AUHenrik Bærbak Christensen10

11. In (Player) Dispatcher CS@AUHenrik Bærbak Christensen11

12. Ex: digRoom CS@AUHenrik Bærbak Christensen12

13. MongoDB Phew… –Seems we are safe there… CS@AUHenrik Bærbak Christensen13

14. Java Concurrency Super fast reiteration CS@AUHenrik Bærbak Christensen14

15. Monitor Add ‘syncronized’ to a method –Becomes critical region Only one thread allowed to execute at a time –Multiple threads wait in queue for execution If several threads call the method at the same time The ‘queue’ controlling object is the object of the method –i.e. there is a ‘per object’ queue –If the method freeze All queued threads does as well! CS@AUHenrik Bærbak Christensen15

16. ReentrantLock You can declare a specific ‘queue/lock’ object and synchronize on that –More tedios, but much finer control Locking across a set of objects –Allows timeout!Avoid ‘blocked threads’ CS@AUHenrik Bærbak Christensen16

17. Idiom Alternative: tryLock(timeout) CS@AUHenrik Bærbak Christensen17 lock.lock(); try { // critical region } finally { lock.unlock(); } lock.lock(); try { // critical region } finally { lock.unlock(); }

18. Tutorials… Jenkov’s tutorials appear very nice… CS@AUHenrik Bærbak Christensen18

19. Our Tool Stack Help! CS@AUHenrik Bærbak Christensen19

20. Blocked Threads! Load Generator, my faulty first version –N players execute M sets of operations Output CS@AUHenrik Bærbak Christensen20

21. How to diagnose? ‘jvisualvm’ CS@AUHenrik Bærbak Christensen21

22. Thread Dump Pick VM, Threads Pane, Click Thead Dump CS@AUHenrik Bærbak Christensen22

23. Review Dump Locate the blocking thread, see stack trace ! CS@AUHenrik Bærbak Christensen23

24. Monitoring Performance Started ‘load.cave’ CS@AUHenrik Bærbak Christensen24

25. More Load Started SkyCave app server on small machine CS@AUHenrik Bærbak Christensen25

26. More Load Started 1 x load.cave of 100 players each CS@AUHenrik Bærbak Christensen26

27. More Load Started 15 x load.cave of 100 players each CS@AUHenrik Bærbak Christensen27 One server, Approx 13.100 req/s One server, Approx 13.100 req/s

28. The ‘Lyon Airport’ Test Karibu System: Daemon died from exhausted memory CS@AUHenrik Bærbak Christensen28

29. Going Further CS@AUHenrik Bærbak Christensen29