1. 3/12/2013Computer Engg, IIT(BHU)1 PRAM ALGORITHMS-1

2. Introduction Shared memory N synchronized processors

3. Introduction ● Constant time access to the memory standard multiplication/addition Communication (implemented via access to shared memory)

4. Shared Memory Access Conflicts Different variations: – Exclusive Read Exclusive Write (EREW) PRAM: no two processors are allowed to read or write the same shared memory cell simultaneously – Concurrent Read Exclusive Write (CREW): simultaneous read allowed, but only one processor can write – Concurrent Read Concurrent Write (CRCW)

5. Shared Memory Access Conflicts Concurrent writes: ● Priority CRCW: processors assigned fixed distinct priorities, highest priority wins ● Arbitrary CRCW: one randomly chosen write wins ● Common CRCW: all processors are allowed to complete write if and only if all the values to be written are equal

6. Boolean OR Of n bits A[1:n] Algorithm – Processor i (in parallel for 1<= i <=n) does – If(A[i]=1) then A[0] := A[i] O(1) on CRCW Omega(log n) on EREW or CREW

7. Prefix Computation Problem Definition: the parallel prefix operation take a binary associative operator, and an array of n elements [x[1], x[2], … x[n]] and produces the array [x[1], (x[1] *x[2]), … (x[1] *x[2]... *x[n])] Example: get prefix sum of [1, 2, 0, 4, 2, 1, 1, 3] is [1, 3, 3, 7, 9, 10, 11, 14] Can be implemented in O(n) time by a serial algorithm – Obvious n-1 applications of operator will work

8. Prefix Computation Step 1: If n=1, output x1 Step 2: – Let first n/2 processors compute prefixes for x[1],x[2]…x[n/2] and store in y[1],y[2],…y[n/2] – At same time let rest n/2 compute prefixes for x[n/2+1]…x[n] in y[n/2+1]..y[n] Step 3: – First half of answer: y[1],y[2],…y[n/2] – Second half: y[n/2+1]*y[n/2]..y[n]*y[n/2]

9. Prefix Computation Analysis – T(n) = T(n/2) + O(1), T(1) = 1 – T(n) = O(log n) – No of processors = n – Work Done = n*log n – Not work optimal

10. Prefix Computation: Work Optimal Algorithm Step 1: Processor i (i=1,2,…n/(log n)) Compute prefix for x[(i-1)(log (n+1))] …..x[i(log n)] in array z Step 2: – n/(log n) processors compute prefix for z[logn], z[2logn]…z[n] using previous algorithm in array w

11. ● Step 3: ● Each Processor i outputs z[(i-1)(log (n+1)]*w [(i-1)(log n)],…..,z[i(log n)] * w [(i- 1)(log n)]

12. Prefix Computation: Work Optimal Algorithm Step 1 takes O(logn) time Step 2 takes O(log(n/logn)) = O(log n) Step 3 takes O(log n) time So time complexity O(log n) Processors n/logn CREW PRAM processors

13. List Ranking Given a single linked list L with n objects, compute, for each object in L, its distance from the end of the list. Let list be A[1:n] in each element stores its neighbor and a value O(n) time serial algorithm

14. List Ranking: Pointer Jumping For( q=1 to [logn]) do Processor i (in parallel for 1<=i<=n) if(Neighbor[i]!=0) then { Rank[i] := Rank[i] + Rank[ Neighbor[i]] Neighbor[i] := Neighbor[Neighbor[i]] }

15. Selection Given a sequence of n keys and an integer i find the ith smallest key from the sequence Serial algorithm : O(n)

16. Maximal Selection in n 2 processors Algorithm – Step 1: If n=1, output the key – Step 2: Processor pij (for each 1<=i,j<=n in parallel) Compute xij = (ki<kj) – Step 3:Each Processor i computes boolean OR of n numbers xi1 to xin O(1) time with n2 CRCW PRAM

17. Maximum using n processors Algorithm – Step 1: If n=1, output the k1 – Step 2:Partition the input keys into n1/2 parts, then for each part Let n1/2 processors find maximum recursively – Step 3:Let n processors compute maximum of n1/2 values of maximum of each part O(log logn) – T(n) = T(n1/2) + O(1) – N CRCW PRAM

18. Maximal Selection Among Integers Let each integer be in [0,nc] Each key is binary number with <= clogn digits So in each step maximum of n numbers with respect to logn/2 MSB’s, is considered. logn/2 bits implies <= n1/2 -1 Find max of n numbers in range [0 to n1/2 -1 ] – n1/2 global memory cells M initialized to –infinity – In each step, if processor i finds key ki in Mki – Compute max in M, ie maximum of n1/2 numbers using n processors

19. Maximal Selection Among Integers Algorithm – For i:=1 to 2c do { Step1: find max of all alive keys with respect to ith parts. Let M be maximum Step2: Delete each alive keys whose ith part is <M } Output one the alive keys O(1) time using n CRCW PRAM