1. Basic of Parallel Programming Library Subroutine : library functions that support parallelism New Constructs : language is extended to support parallelism Compiler Directives : formatted comment identified parallel code

2. Process and Thread Process is an entity with 3 components: –Program –Status –State Control state Data state Thread is a light weight process

3. Process Status ReadyRunning NonexistentSuspended schedule Preempt, Timeout Wake up Create Terminate Suspend

4. Address Space A new address space is created with a new process Threads share the same address space Code Segment Data Segment Stack Process Descriptor User spaceKernel space

5. Process Scheduling Mode Dedicated Mode : Not share Batch : Run to completion Time-Sharing : Interleaving Preemptive : Taking over

6. Component Processes MPMD –Multiple-program-multiple-data SPMD –Single-Program-multiple-data Run A on node 1 Run B on node 2 Run C on node 3 pid = MyProcessID() if (pid ==1) Run A if (pid == 2) Run B if (pid == 3) Run C

7. Dynamic vs. Static Dynamic parallelism –More flexible –Processes are created during run-time –Ex While(c>0) { fork(foo(c)); c = boo(c); } Static parallelism –Number of process is pre-determined. –Less run-time overhead –More efficient binary code

8. fork and join Process A begin fork(B) x = foo(z) join(B) Output(x+y) end Process B begin y = boo(z) end

9. Interaction between Processes Communication –Shared variable –Parameter passing –Message passing Synchronization –Atomicity –Control Synchronization –Data Synchronization Aggregation

10. Interaction Mode Interaction Code P1P2Pn Asynchronous Vs. Synchronous Blocking Vs. Non-Blocking

11. Problems in Concurrency Lost Update Problem Process 1Process 2 Read(A) A = A+mRead(B) Write(A)A=A+B Write(A)

12. Problems in Concurrency Temporary Update Problem –When a process update data and then fail, another process should not see the updated value

13. Problems in Concurrency Incorrect Summary Problem Process1Process2 Data A, B, C in x Count = the number of items in x Write(new item D in x) Read(x) Sum(x) // (A+B+C+D) Ave = sum/count ‘Ave’ value will be incorrect from this example.

14. Concurrency Control Locking Timestamp Optimistic concurrency control (OCC)