Basic of Parallel Programming Library Subroutine : library functions that support parallelism New Constructs : language is extended to support parallelism Compiler Directives : formatted comment identified parallel code

Process and Thread Process is an entity with 3 components: –Program –Status –State Control state Data state Thread is a light weight process

Process Status ReadyRunning NonexistentSuspended schedule Preempt, Timeout Wake up Create Terminate Suspend

Address Space A new address space is created with a new process Threads share the same address space Code Segment Data Segment Stack Process Descriptor User spaceKernel space

Process Scheduling Mode Dedicated Mode : Not share Batch : Run to completion Time-Sharing : Interleaving Preemptive : Taking over

Component Processes MPMD –Multiple-program-multiple-data SPMD –Single-Program-multiple-data Run A on node 1 Run B on node 2 Run C on node 3 pid = MyProcessID() if (pid ==1) Run A if (pid == 2) Run B if (pid == 3) Run C

Dynamic vs. Static Dynamic parallelism –More flexible –Processes are created during run-time –Ex While(c>0) { fork(foo(c)); c = boo(c); } Static parallelism –Number of process is pre-determined. –Less run-time overhead –More efficient binary code

fork and join Process A begin fork(B) x = foo(z) join(B) Output(x+y) end Process B begin y = boo(z) end

Interaction between Processes Communication –Shared variable –Parameter passing –Message passing Synchronization –Atomicity –Control Synchronization –Data Synchronization Aggregation

Interaction Mode Interaction Code P1P2Pn Asynchronous Vs. Synchronous Blocking Vs. Non-Blocking

Problems in Concurrency Lost Update Problem Process 1Process 2 Read(A) A = A+mRead(B) Write(A)A=A+B Write(A)

Problems in Concurrency Temporary Update Problem –When a process update data and then fail, another process should not see the updated value

Problems in Concurrency Incorrect Summary Problem Process1Process2 Data A, B, C in x Count = the number of items in x Write(new item D in x) Read(x) Sum(x) // (A+B+C+D) Ave = sum/count ‘Ave’ value will be incorrect from this example.

Concurrency Control Locking Timestamp Optimistic concurrency control (OCC)