1 PRAM Algorithms Sums Prefix Sums by Doubling List Ranking

2 Definition: Prefix Sums  Given a set of n values a1, a2,…, an and an associative the Prefix Sums problem is to compute the n quantities a1, Example: {2, 7, 9, 4}  {2, 9, 18, 22}

3 Doubling  A processing technique in which accesses or actions are governed by increasing powers or 2  That is, processing proceeds by 1, 2, 4, 8, 16, etc., doubling on each iteration

4 Prefix Sum by Doubling  Overview 1. Each a(i) is added to a(i+1) 2. Each a(i) is added to a(i+2) 3. Each a(i) is added to a(i+4) 4. Each a(i) is added to a(i+8) ETC…..  At any time if an index exceeds n, the operation is supressed

5 Prefix Sums by Doubling Example * *20* *23*30*36* * Operation supressed T1 = O(n) Tp = O(log n)

6 Prefix Sums by Doubling Example 0# * 0#0,1#1,22,33,44,55,6*6,7* 0#0,1#0,1,2 # 0,1,2, 3# 1,2,3, 4 2,3,4, 5 3,4,5, 6 4,5,6, 7 00,10,1,20,1,2, 3 0,1,2, 3,4 0,1,2, 3,4,5 0,1,2 3,4,5, 6 0,1,2, 3,4,5, 6,7 # contains final sum * operation suppressed T1 = O(n) Tp = O(log n)

7 Time Complexity  O(Log N)  At each step, the number of sums that are complete doubles 1, 2, 4, 8,… Thus, the number of steps is log n

8 Total Operations – Work/Cost  For N data values Step 1 = N -1 additions {-2 0 }  1 PC is suppressed Step 2 = N – 2 additions {-2 1 }  2 PCs are suppressed Step 3 = N – 4 additions {-2 2 }  4 PCs are suppressed Etc.

9 Consider case of N = 8  Step 1 = N-1 = 7  Step 2 = N-2 = 6  Step 3 = N-4 = 4 TOTAL = 17  Generalize (N-1) + (N-2) + (N-4) = 3N -7  Sequential = N-1

10 Generalize the Work Sum(i=0 to (log n) -1: N – 2 i = (N-1) + (N-2) + (N-4) +…+ (N-2 log n -1 ) =N*log N – ( log n -1 ) ( log n -1 ) = ???

11 Total Work for Prefix Sums ( log n -1 ) = 2 log n - 1 Size = N * Log N – (2 log n – 1) = N * Log N - 2 log n + 1 T1 = N

12 Prefix Sums - Comparisons AlgorithmWork - CostDepth Sequential N - 1 Doubling N Log N - 2 log n + 1Log N Upper/Lower N/2 Log NLog N Odd/Even 2N – Log N - 22 Log N - 2

13 Parallel Strategies - PRAM  Broadcast, Fan-out, Expand O(log n)  Reduction, Combination, Fan-in O(log n)  These are basically opposites

14 PRAM Algorithm Instructions - Spawn- For all  Step 1 of all PRAM algorithms is to activate P processors (Broadcast) One processor starts activation Activation takes O(Log P) time  Instruction: spawn (processor names) E.G. spawn (P0, P1,.., Pn)  For all do {stmts} endfor E.G. For all Pi, 0<=i<=n-1, do{…}

15 Sum of elements – EREW PRAM Given: n elements in A[0 … n-1] Var: A & j are global, i is local spawn (P0, P1, P2,..P n/2-1 )// P = n/2 For all Pi, 0 <= i <= (n/2 -1) for j = 0 to log n - 1 do if (i mod 2 j = 0) & (2i + 2 j < n) A[2i] = A[2i] + A[2i + 2 j ]

16 Trace for P 0 & P 1 P 0, i = 0 for all operations j = 0 to 2 (i=0 mod 2 0 ) & (2* < n) – yes A[0] = A[0] + A[1] P 1, i = 1 for all operations j = 0 to 2 (i=1 mod 2 0 ) & (2* < n) – yes A[2] = A[2] + A[3]

17 Prefix Sum - Doubling CREW PRAM Given: n elements in A[0 … n-1] Var: A & j are global, i is local spawn (P1, P2,..P n-1 ) // note # of PC For all Pi, 1 <= i <= n -1) for j = 0 to log n - 1 do if (i - 2 j >= 0) A[i] = A[i] + A[i - 2 j ]

18 List Packing An Application of Prefix Sums  Consider an array of upper and lower case letters. Delete the lower case letters and compact the upper case to the low-order end of the array.

19 List Packing - Demonstration aGHinWbN GHWN

20 List Packing Implementation via Prefix Sums  Assign 1 to items to be packed and 0 to items to be deleted.  Perform the prefix sums on the 1/0  If upper case, store in location = sum, otherwise do nothing  Last sum provides number of packed items

21 List Packing Implementation via Prefix Sums aGHinWbN GHWN S’pose: don’t know how 0/1 assigned. Can I still pack?

22  Given a linked list, stored in an array, compute the distance of each element from the end (either end) of the list.  Problem is similar to prefix sums, using all 1’s to sum.  Called Pointer Jumping (not doubling) when using pointers.  Don’t destroy original list! Linked List Ranking

23 Linked List Ranking - demo AQFCDTBP

24 ADQBCFPT Nil rank next data * List Ranking - Demonstration

25 List Ranking Pointer Jumping - similar to Doubling init nil nil nil nil ► ► ► ► ► ►► ►► ► ► ► ► ► ► ► ► ► ► Note: Must copy pointers to preserve the original linked list.

26 List Ranking Code // Copy next and set 1’s for all elements For all P ί, 0 ≤ ί ≤ n-1, pardo N( ί ) = next ( ί ) Rank ( ί ) = 1 // Doubling on the linked list For j = 1 to log n do if N( ί ) ≠ N (N( ί )) then Rank ( ί ) = Rank( ί ) + Rank (N( ί )) N( ί ) = N (N( ί ))

27 What are we really doing???  If your location and your next do not point to the same location then Add rank of next to yourself Change your next to next(next)  Each step doubles distance of your next pointer  Progression of solution Initially – 1 rank correct Step 1 – 2 ranks correct Step 2 – 4 ranks correct Step 3 – 8 ranks correct O(log n) steps

Work Analysis  Number of Steps: Tp = O(Log N)  Number of Processors: N  Work = O(N log N)  T1 = O(N)  Work Optimal?? 28

29 Applications of List Ranking  Expression Tree Evaluation  Parentheses Matching  Tree Traversals  Ear–Decomposition of Graphs  Euler tour of trees  many others