1. Dr. Christian Vecchiola Postdoctoral Research Fellow csve@unimelb.edu.au Cloud Computing and Distributed Systems (CLOUDS) Lab Dept. of Computer Science and Software Engineering The University of Melbourne Thread Programming Most concepts are drawn from Chapter 6 © Pearson Education Most concepts are drawn from Chapter 14 © Mc Graw Hill

2. Thread Programming Distributed Systems Principles and Paradigms Outline  Introduction  Threaded Applications  Defining Threads  Java Threads  Architecture of Multithreaded servers  Threads Synchronization  Summary

3. Thread Programming Distributed Systems Principles and Paradigms Introduction  Building Distributed Systems – Why not using… Distributed Operating Systems? – It is a radical choice! – Application reconversion costs – Tight coupling between machines Network Operating Systems? – Good option, but bare support for DS – Examples: » Linux (Unix, flavors) » Windows

4. Thread Programming Distributed Systems Principles and Paradigms Introduction  Building Distributed Systems – DOS or NOS are not enough to build a DS! – NOS are a good starting point but …. – … we need an additional layer “gluing” all together NOS Middleware

5. Thread Programming Distributed Systems Principles and Paradigms Introduction  Building Distributed Systems – Middleware High-level features for DS – Communication – Management – Application specific Uniform layer where to build DS services Runtime environment of applications – Operating System Low / medium level (core) features – Process / threads management – Local hardware (CPU, disk, memory) – Security (users, groups, domain, ACLs) – Basic networking

6. Thread Programming Distributed Systems Principles and Paradigms Introduction  Operating System Layer and Middleware Application Services Middleware OS2 Processes / Threads Communication Computer & Network Hardware OS1 Processes / Threads Communication Computer & Network Hardware Node 1 Node 2

7. Thread Programming Distributed Systems Principles and Paradigms Introduction  Inside the Operating System – (High Level view) – Communication Manager – Thread & Process Manager – Memory Manager – Device Drivers – Supervisor Threads? What are they?

8. Thread Programming Distributed Systems Principles and Paradigms Threaded Applications  Modern Systems – Multiple applications run concurrently! – This means that… there are multiple processes on your computer web & email multimedsa office automation games pictures Multitasking

9. Thread Programming Distributed Systems Principles and Paradigms Threaded Applications  Modern Systems – Applications perform many tasks at once! – This means that… there are multiple threads within a single process. Background printing GUI rendering Application core logic

10. Thread Programming Distributed Systems Principles and Paradigms Threaded Applications  Modern Applications – Example: Internet Browser + Youtube Video Streaming Favorities, Share, Comments Posting

11. Thread Programming Distributed Systems Principles and Paradigms Threaded Applications  Modern Applications – Example: Multithreaded Web Server Web/FTP server Client 1 Client 2 Client N Process Request Client 1Process Request Client 2Process Request Client N

12. Thread Programming Distributed Systems Principles and Paradigms Threaded Applications  Modern Applications & Systems – Operating System Level Multitasking: multiple applications running at once – Application Level Multithreading: multiple operations performed at the same time – Bottom Line: Illusion of concurrency Today’s focus

13. Thread Programming Distributed Systems Principles and Paradigms Defining Threads  Threads – Thread: set of instructions that are executed sequentially within an program. public class SingleThread { public static void main(String[] args) { …… … for (int i=0; i<args.Length; i++) { if (args[i].equals("-r") == true) { SingleThread.executeOptionR(args); } private static void executeOptionR(String[] args) { …… … } public class SingleThread { public static void main(String[] args) { …… … for (int i=0; i<args.Length; i++) { if (args[i].equals("-r") == true) { SingleThread.executeOptionR(args); } private static void executeOptionR(String[] args) { …… … } One Thread

14. Thread Programming Distributed Systems Principles and Paradigms Defining Threads  Multithreading – Multithreading: execution of multiple threads within the same process space. public class MultiThread { public static void main(String[] args) { …… … for (int i=0; i<args.Length; i++) { if (args[i].equals("-op1“) == true) { Thread t1 = new Thread(Op1); t1.start(); } if (args[i].equals("-op2“) == true) { Thread t2 = new Thread(Op2); t2.start(); } public class MultiThread { public static void main(String[] args) { …… … for (int i=0; i<args.Length; i++) { if (args[i].equals("-op1“) == true) { Thread t1 = new Thread(Op1); t1.start(); } if (args[i].equals("-op2“) == true) { Thread t2 = new Thread(Op2); t2.start(); } public class Op1 implements Runnable { public void run() { …… … } public class Op1 implements Runnable { public void run() { …… … } public class Op2 implements Runnable { public void run() { …… … } public class Op2 implements Runnable { public void run() { …… … } Thread

15. Thread Programming Distributed Systems Principles and Paradigms Defining Threads  Threads and Processes – Each application that is executed on modern operating systems contains at least one thread. When a process starts a default (main) thread is created. This threads lasts for the entire execution of the process. It executes the code that is loaded in the process space sequentially. When a process starts a default (main) thread is created. This threads lasts for the entire execution of the process. It executes the code that is loaded in the process space sequentially. Single-threaded Process Address space Main Thread Execution Timeline

16. Thread Programming Distributed Systems Principles and Paradigms Defining Threads  Threads and Processes Common address space Thread Execution Timeline Thread Main Thread Multiple Execution Streams Thread When a process starts a default (main) thread is created. From this thread other threads can be created. Each of these threads can run for different periods Each of these threads can create new threads All the threads have access to a shared memory space. When a process starts a default (main) thread is created. From this thread other threads can be created. Each of these threads can run for different periods Each of these threads can create new threads All the threads have access to a shared memory space. Multithreaded Process

17. Thread Programming Distributed Systems Principles and Paradigms Defining Threads  Why to use threads? – Originally conceived to simplify and improve algorithms programming  Major applications: – GUI programming – Throughput computing algorithm independent components that can be performed concurrently process threads

18. Thread Programming Distributed Systems Principles and Paradigms Defining Threads  Applications – Threads are mostly used to perform: – Parallelism and concurrent execution of independent tasks / operations. – Implementation of reactive user interfaces. – Non blocking I/O operations. – Asynchronous behavior. – Timer and alarms implementation.

19. Thread Programming Distributed Systems Principles and Paradigms Defining Threads  Example: Web/FTP Server Web/FTP server while { } while { } Main Thread Worker Thread Worker Thread Worker Thread Worker Thread Worker Thread Worker Thread Execution Timeline

20. Thread Programming Distributed Systems Principles and Paradigms Defining Threads  Summing Up – A Thread is a piece of code that run in concurrent with other threads. – Each thread is a statically ordered sequence of instructions. – Threads are being extensively used express concurrency on both single and multiprocessors machines. – Programming a task having multiple threads of control – Multithreading or Multithreaded Programming.

21. Thread Programming Distributed Systems Principles and Paradigms Java Threads  Overview – Objects and classes for thread management java.lang.Thread java.lang.Runnable java.lang.ThreadGroup – Language constructs for synchronization synchronized keyword Object.[wait(), notify(), notifyAll()] – Advanced features: thread-safe collections

22. Thread Programming Distributed Systems Principles and Paradigms Java Threads  Thread Creation – Extends the java.lang.Thread class – Implement the java.lang.Runnable interface import java.lang.Thread; public class Worker extends Thread { public Worker() { …… } // main method of the thread public void run() { …… } public static void main(String[] args) { Worker thread = new Worker(); thread.start(); } import java.lang.Thread; public class Worker extends Thread { public Worker() { …… } // main method of the thread public void run() { …… } public static void main(String[] args) { Worker thread = new Worker(); thread.start(); } Method 1 import java.lang.Runnable; public class Worker implements Runnable { public Worker() { …… } // main method of the thread public void run() { …… } public static void main(String[] args) { Worker wk = new Worker(); Thread thread = new Thread(wk); thread.start(); } import java.lang.Runnable; public class Worker implements Runnable { public Worker() { …… } // main method of the thread public void run() { …… } public static void main(String[] args) { Worker wk = new Worker(); Thread thread = new Thread(wk); thread.start(); } Method 2

23. Thread Programming Distributed Systems Principles and Paradigms Java Threads  Thread Creation – Extending java.lang.Thread Objects are threads Define a new class that extends Thread Implement the run method. – Advantages: Easy to create and start… – Disadvantages: The class must inherit from Thread

24. Thread Programming Distributed Systems Principles and Paradigms Java Threads  Thread Creation – Extending java.lang.Thread import java.lang.Thread; public class Worker extends Thread { public Worker() { …… } // main method of the thread public void run() { System.out.println("This class extends Thread"); } public static void main(String[] args) { Worker thread = new Worker(); thread.start(); } import java.lang.Thread; public class Worker extends Thread { public Worker() { …… } // main method of the thread public void run() { System.out.println("This class extends Thread"); } public static void main(String[] args) { Worker thread = new Worker(); thread.start(); }

25. Thread Programming Distributed Systems Principles and Paradigms Java Threads  Thread Creation – Implementing java.lang.Runnable Objects are not threads Define a new class that implements Runnable Implement the run method. – Advantages: The class can inherit from any type… – Disadvantages: The setup requires more work…

26. Thread Programming Distributed Systems Principles and Paradigms Java Threads  Thread Creation – Implementing java.lang.Runnable import java.lang.Runnable; public class Worker implements Runnable { public Worker() { …… } // main method of the thread public void run() { System.out.println("This class implement Runnable"); } public static void main(String[] args) { Worker worker = new Worker(); Thread thread = new Thread(worker); thread.start(); } import java.lang.Runnable; public class Worker implements Runnable { public Worker() { …… } // main method of the thread public void run() { System.out.println("This class implement Runnable"); } public static void main(String[] args) { Worker worker = new Worker(); Thread thread = new Thread(worker); thread.start(); }

27. Thread Programming Distributed Systems Principles and Paradigms Java Threads  Thread Lifecycle Management ready running waiting sleeping new blocked dead notify() start() wait() sleep() stop() suspend() resume() dispatch completion timer expire / interrupted timer expire / interrupted block on I/O I/O completion

28. Thread Programming Distributed Systems Principles and Paradigms Java Threads  Example: Three threads public class A extends Thread { public void run() { for(int i=0; i<5; i++) { System.out.println(“\tFrom Thread A i=“+i); } System.out.println(“Exit from Thread A i=“+i); } public class A extends Thread { public void run() { for(int i=0; i<5; i++) { System.out.println(“\tFrom Thread A i=“+i); } System.out.println(“Exit from Thread A i=“+i); } A.java public class B extends Thread { public void run() { for(int i=0; i<5; i++) { System.out.println(“\tFrom Thread B i=“+i); } System.out.println(“Exit from Thread B i=“+i); } public class B extends Thread { public void run() { for(int i=0; i<5; i++) { System.out.println(“\tFrom Thread B i=“+i); } System.out.println(“Exit from Thread B i=“+i); } B.java public class C extends Thread { public void run() { for(int i=0; i<5; i++) { System.out.println(“\tFrom Thread C i=“+i); } System.out.println(“Exit from Thread C i=“+i); } public class C extends Thread { public void run() { for(int i=0; i<5; i++) { System.out.println(“\tFrom Thread C i=“+i); } System.out.println(“Exit from Thread C i=“+i); } C.java public class ThreadTest { public void main(String[] args) { new A().start(); new B().start(); new C().start(); } public class ThreadTest { public void main(String[] args) { new A().start(); new B().start(); new C().start(); } ThreadTest.java

29. Thread Programming Distributed Systems Principles and Paradigms Java Trhraeds  Example: Three threads [raj@mundroo]raj: java ThreadTest From Thread A: i= 1 From Thread A: i= 2 From Thread A: i= 3 From Thread A: i= 4 From Thread A: i= 5 Exit from Thread A From Thread C: k= 1 From Thread C: k= 2 From Thread C: k= 3 From Thread C: k= 4 From Thread C: k= 5 Exit from Thread C From Thread B: j= 1 From Thread B: j= 2 From Thread B: j= 3 From Thread B: j= 4 From Thread B: j= 5 Exit from Thread B [raj@mundroo]raj: java ThreadTest From Thread A: i= 1 From Thread A: i= 2 From Thread A: i= 3 From Thread A: i= 4 From Thread A: i= 5 Exit from Thread A From Thread C: k= 1 From Thread C: k= 2 From Thread C: k= 3 From Thread C: k= 4 From Thread C: k= 5 Exit from Thread C From Thread B: j= 1 From Thread B: j= 2 From Thread B: j= 3 From Thread B: j= 4 From Thread B: j= 5 Exit from Thread B Run 1 [raj@mundroo]raj: java ThreadTest From Thread A: i= 1 From Thread A: i= 2 From Thread A: i= 3 From Thread A: i= 4 From Thread A: i= 5 From Thread C: k= 1 From Thread C: k= 2 From Thread C: k= 3 From Thread C: k= 4 From Thread C: k= 5 Exit from Thread C From Thread B: j= 1 From Thread B: j= 2 From Thread B: j= 3 From Thread B: j= 4 From Thread B: j= 5 Exit from Thread B Exit from Thread A [raj@mundroo]raj: java ThreadTest From Thread A: i= 1 From Thread A: i= 2 From Thread A: i= 3 From Thread A: i= 4 From Thread A: i= 5 From Thread C: k= 1 From Thread C: k= 2 From Thread C: k= 3 From Thread C: k= 4 From Thread C: k= 5 Exit from Thread C From Thread B: j= 1 From Thread B: j= 2 From Thread B: j= 3 From Thread B: j= 4 From Thread B: j= 5 Exit from Thread B Exit from Thread A Run 2

30. Thread Programming Distributed Systems Principles and Paradigms Java Threads  Threads Priorities – Java allows developer to set the priority with which it will be executed by the JVM. – By default each thread is assigned the NORM_PRIORITY and threads are scheduled in FCFS mode. – There is a range of values for priorities: MIN_PRIORITY = 1 NORM_PRIORITY = 5 MAX_PRIORITY = 10

31. Thread Programming Distributed Systems Principles and Paradigms Java Threads  Thread Priorities – Let’s modify the previous example… public class ThreadPriorities { public void main(String[] args) { A threadA = new A(); B threadB = new B(); C threadC = new C(); threadA.setPriority(Thread.MAX_PRIORITY); threadB.setPriority(threadA.getPriority()+1); threadC.setPriority(Thread.MIN_PRIORITY); System.out.println(“Started Thread A…"); threadA.start(); System.out.println("Started Thread B…"); threadB.start(); System.out.println("Started Thread A…"); threadC.start(); System.out.println("End of main Thread"); } public class ThreadPriorities { public void main(String[] args) { A threadA = new A(); B threadB = new B(); C threadC = new C(); threadA.setPriority(Thread.MAX_PRIORITY); threadB.setPriority(threadA.getPriority()+1); threadC.setPriority(Thread.MIN_PRIORITY); System.out.println(“Started Thread A…"); threadA.start(); System.out.println("Started Thread B…"); threadB.start(); System.out.println("Started Thread A…"); threadC.start(); System.out.println("End of main Thread"); } ThreadPriorities.java

32. Thread Programming Distributed Systems Principles and Paradigms Java Threads  Thread Priorities – Priority values are… – This means.. The number of available priorities depends on The implementation of the JVM The hosting OS The mapping of java Threads to physical OS threads Only the ordering of priority values is guaranteed INDICATIVE VALUES!

33. Thread Programming Distributed Systems Principles and Paradigms Java Threads  Other operations – Identification get/set Name and Id IsActive / isInterrupted activeCount / currentThread – StackTrace access – Direct operations suspend / resume / interrupt start / stop / sleep – Synchronization join holdsLock – … more

34. Thread Programming Distributed Systems Principles and Paradigms Synchronization  Overview – Required when accessing shared resources – Prevents...... spurious write/reads... races on data... inconsistency of data – Ensures that...... only the allowed number of threads (generally one) access the data at the same time. – Requires primitives at language level

35. Thread Programming Distributed Systems Principles and Paradigms  Overview – Examples: Printer (two person jobs cannot be printed at the same time). Simultaneous operations on your bank account. Can the following operations be done at the same time on the same account? - Deposit() - Withdraw() - Enquire()

36. Thread Programming Distributed Systems Principles and Paradigms Synchronization  Scenario: Online banking Internet Bank Server Internet Bank Server Bank Local Area Network Client N Client 2 Client 1 Bank Operator 1 Bank Operator M Bank Database

37. Thread Programming Distributed Systems Principles and Paradigms Synchronization  Scenario: online bank – Assumption The internet bank server processes each request with a different thread – Case study A customer operates on account X An operator updates account X (Alternatively: two or more customers/operators update the same account)

38. Thread Programming Distributed Systems Principles and Paradigms Synchronization  Scenario: online bank – Problem! If two threads access the same bank account the final value of the account could be unpredictable! In particular the value could be: – Not valid – Or an amount that does not map to any operation performed This happens because there is no exclusive access to the bank account data from each thread.

39. Thread Programming Distributed Systems Principles and Paradigms Synchronization  Scenario: online bank – Synchronization allows to solve the problem: It allow each thread to have exclusive access It makes the accesses of threads sequential It preserves the consistency of data – But… It does not enforce an ordering for accessing the data from each thread

40. Thread Programming Distributed Systems Principles and Paradigms Synchronization  Java Synchronization APIs – synchronized keyword (method modifier) Transforms the entire scope of the method a protected region that is executed atomically used for simple monitors and binary semaphores – Object.[wait, notify, notifyAll] “special methods” implement the primitives wait and signal (Dijkstra) used for integer semaphore, and more complex synchronization patterns

41. Thread Programming Distributed Systems Principles and Paradigms Synchronization  Scenario: online bank – Solution (naïve) The threads operating on the bank account share the same Account instance The account class is “synchronized” – deposit() – enquire() – withdraw() are “modified” with the synchronized keyword.

42. Thread Programming Distributed Systems Principles and Paradigms  Scenarion: online bank op: account.withdraw(50); op: account.withdraw(50); Thread Client Request 1 op: account.enquire(); op: account.enquire(); Thread Client Request 2 op: account.deposit(1000) op: account.deposit(1000) Thread Client Request N + deposit(int amount) + withdraw(int amount) + enquire() + deposit(int amount) + withdraw(int amount) + enquire() Account Shared Instance

43. Thread Programming Distributed Systems Principles and Paradigms Synchronization  Scenario: online bank – Account class public class Account { int balance; public synchronized void deposit(int amount) { this.balance += amount; } public synchronized void withdraw(int amount) this.balance -= amount; } public synchronized void enquire( ) { return this.balance; } public class Account { int balance; public synchronized void deposit(int amount) { this.balance += amount; } public synchronized void withdraw(int amount) this.balance -= amount; } public synchronized void enquire( ) { return this.balance; } Account.java

44. Thread Programming Distributed Systems Principles and Paradigms Synchronization  Scenario: online bank – Putting all together.. … public class InternetBankingSystem { public void main(String[] args) { String id = args[0]; Account account = getAccountFromDB(id); // perform three different operations on the same account // asynchronously WithdrawThread w = new WithdrawThread(account, 50); DepositThread d = new DepositThread(account, 10000); EnquireThread e = new EnquireThread(account); w.start(); d.start(); e.start(); } … public class InternetBankingSystem { public void main(String[] args) { String id = args[0]; Account account = getAccountFromDB(id); // perform three different operations on the same account // asynchronously WithdrawThread w = new WithdrawThread(account, 50); DepositThread d = new DepositThread(account, 10000); EnquireThread e = new EnquireThread(account); w.start(); d.start(); e.start(); }

45. Thread Programming Distributed Systems Principles and Paradigms Synchronization  Scenario: online bank – Threads… import Account; public class WithdrawThread extends Thread { private Account account; public withdrawThread(Account acc, int amnt) { this.account = acc; this.amount = amnt; } public void run() { this.account.withdraw(this.amount); } import Account; public class WithdrawThread extends Thread { private Account account; public withdrawThread(Account acc, int amnt) { this.account = acc; this.amount = amnt; } public void run() { this.account.withdraw(this.amount); } WithdrawThread.java

46. Thread Programming Distributed Systems Principles and Paradigms Synchronization  Object.[wait, notify, notifyAll] – These methods are “special”: The JVM ensure that they are executed atomically They have a specific implementation – When to use them? They can be used to emulate synchronized They can implement different synchronization patterns They require more work and a better understanding

47. Thread Programming Distributed Systems Principles and Paradigms Architecture of Multithreaded Servers  Scenario – Multithreading enables servers to maximize their throughput, measured as the number of requests processed per second. – Threads may need to treat requests with varying priorities: A corporate server could prioritize request processing according to class of customers. – Architectures: Worker pool Thread-per-request Thread-per-connection Thread-per-object

48. Thread Programming Distributed Systems Principles and Paradigms Architecture of Multithreaded Servers  Worker Pool - Architecture Master Thread Thread Thread Thread Worker Pool Input/ Request Server Client 1 Can I have one thread to serve the request?

49. Thread Programming Distributed Systems Principles and Paradigms Architecture of Multithreaded Servers  Thread per Connection Main Server Thread Thread Active Conn 1 Server Active Conn 2Active Conn N Resources Databases File System Thread Thread Thread

50. Thread Programming Distributed Systems Principles and Paradigms Architecture of Multithreaded Servers  Thread per Object Main Server Thread Thread Thread Thread Active Conn 1 Server Active Conn 2Active Conn N Resources Databases File System

51. Thread Programming Distributed Systems Principles and Paradigms Architecture of Multithreaded Servers  Thread per Request Main Server Thread Thread Active Conn 1 Server Active Conn N Resources Databases File System Thread Thread Thread

52. Thread Programming Distributed Systems Principles and Paradigms Final Observation  Distributed Systems – NOS + Middleware NOS (processes, threads, basic networking) Middleware (services, gluing, applications) – Multi-threading Definition of thread Java API Synchronizations – Multithreaded Server Architectures Worker Pool Thread per (request | connection | object) – Ready for assignment 1?