1. CS Parallelism Dr. Vicki H. Allan

2. 2 Concurrency Concurrency can occur at four levels: 1.Machine instruction level – may have both an adder and a multiplier that are used at the same time. 2. High-level language statement level – might have a loop (a,b,c) where c from one iteration and a from the next are executed at the same time. 3. Unit level: several methods execute together 4. Program level: several program execute together

3. 3 Suppose we have two methods Populate marsh{ for(i=0;i< 1000; i++){ create frog // high level language statement create carp create mosquitos} } Populate prehistoric world{ for (i=0;i<10,i++) create dinosaur(i) }

4. 4 Concurrency can occur at four levels (termed granularity) 1.Machine instruction level – Create frog is decomposed into basic parts. If one basic instruction is to fold both sides into center, perhaps one “processor” folds the left side and one folds the right. 2. High-level language statement level - different parts of “make frog” happen together 3. Unit level: populate marsh occurs with populate prehistoric world 4. Program level: several programs (to do other things not shown here) execute together

5. 5 What would be the advantages/disadvantages of each type of parallelism?

6. 6 1. Late 1950s - One general-purpose processor and one or more special-purpose processors for input and output operations 2. Early 1960s - Multiple complete processors, used for program-level concurrency 3. Mid-1960s - Multiple partial processors, used for instruction-level concurrency 4. Single-Instruction Multiple-Data (SIMD) machines. The same instruction goes to all processors, each with different data - e.g., vector processors 5. Multiple-Instruction Multiple-Data (MIMD) machines Independent processors that can be synchronized (unit-level concurrency) The Evolution of Multiprocessor Architectures

7. 7 Making a Frog Fold in sides

8. 8 Take lower corner and fold up to top. Repeat with other side. Fold into middle Repeat

9. 9 Examples SIMD - all do the same things at the same time. All fold; All Open; All fold again Pipelined – one person does fold, and then passes. Problems? MIMD – all do different things

10. 10

11. 11 Def: A thread of control in a program is the sequence of program points reached as control flows through the program Categories of Concurrency: 1. Physical concurrency - Multiple independent processors (multiple threads of control) 2. Logical concurrency - The appearance of physical concurrency is presented by time- sharing one processor (software can be designed as if there were multiple threads of control)

12. What would be the advantage of logical concurrency? Consider the TV remote as performing “context switch”. Why does one switch between multiple programs? What is downside to switch?

13. 13 Example: Smart Remote Ads play when you are not watching, assume “program” doesn’t continue when you aren’t watching it You might be an E-mail Junkie.. You might be a computer science major Attraction to computer scientists

14. 14 Concerns? Is switching between tasks confusing? What would need to be retained? Is switching between tasks expensive? Would there be a minimal size at which you spawn more tasks? What is the gain?

15. 15 What is the gain? –Models actual situation better –response time –Use delays in processing

16. 16 Why do we want parallelism? Price-performance curves Used to be – paid more for computer - got more (linear relationship between price and performance). Now, for little money, get a lot of power. As you add more money, performance curve levels off. Not an efficient way to get more performance Parallelism is the answer – string cheap computers together to do more work.

17. 17 What is a Thread ? Just as multitasking OS’s can run more than one process “concurrently”, a process can do the same by running more than a single thread. Each Thread is a different stream of control that can execute its instructions independently. Compared to a process, a thread is inexpensive to create, terminate, schedule or synchronize.

18. 18 What is a Thread ? A process is a HEAVY-WEIGHT kernel- level entity. (process struct) A thread is a LIGHT_WEIGHT entity comprising the registers, stack and some other data. The rest of the process struct is shared by all threads. (address space, file desc, etc.) Most of the thread structure is at the user space allowing very fast access.

19. 19 So for our example If we had two processes to populate the marsh and to populate the prehistoric world, each process would be able to stand alone. If we had two threads to populate the marsh and to populate the prehistoric world, they would have some shared resources (like the table or paper supply)

20. 20 Concurrency Vs. Parallelism Concurrency means that two or more threads can be in the “middle” of executing code. Only one can be on the CPU though at any given time. Parallelism actually involves multiple CPUs running threads at the same time. Concurrency is the illusion of Parallelism

21. 21 What can threads do that can’t be done by processes sharing memory ? Answer: Nothing !... If you have –plenty of time to kill programming, –more time to kill processing, –willing to burn money by buying RAM Debugging cross-process programs are tough. In Solaris creating a thread is 30 TIMES FASTER than forking a process. Synchronization is 10 time faster with threads. Context Switching - 5 times faster

22. 22 What Applications to Thread? Multiplexing (communicate two or more signals over a common channel) –Servers Synchronous Waiting (definition?) –clients –I/O Event Notification Simulations Parallelizable Algorithms –Shared memory multiprocessing –Distributed Multiprocessing

23. 23 Which Programs NOT to thread? Compute bounds threads on a uniprocessor. Very small threads (threads are not free) Old Code Parallel execution of threads can interfere with each other. WARNING: Multithreaded applications are more difficult to design and debug than single threaded apps. Threaded programming design requires careful preparation !

24. 24 Synchronization The problem - –Data Race - occurs when more than one thread is trying to update the same piece of data. –Critical Section - Any piece of code to which access needs to be controlled. The Solution - –Mutex –Condition Variables –Operations - init, lock, unlock

25. 25 MUTEX A MUTual EXclusion allows exactly one thread access to a variable or critical section of code. Access attempts by other threads are blocked until the lock is released.

26. 26 Kinds of synchronization: 1. Cooperation –Task A must wait for task B to complete some specific activity before task A can continue its execution e.g., You cut the paper and then I fold it. 2. Competition –When two or more tasks must use some resource that cannot be simultaneously used e.g., we both want the scissors.

27. 27 Liveness means the unit will eventually complete its execution. I’m currently blocked from finishing my frog, but I will eventually get to finish. In a concurrent environment, a task can easily lose its liveness. You were supposed to wake me up when the scissors became available, but you forgot. If all tasks in a concurrent environment lose their liveness, it is called deadlock. I take the paper and wait for the scissors. You take the scissors and wait for the paper. Circular wait is deadlock.

28. 28 Livelock: theoretically can finish, but never get the resources to finish. How do you prevent deadlock? How do you prevent livelock?

29. 29 Questions?