1. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science pH: A Parallel Dialect of Haskell Jim Cipar & Jacob Sorber University of Massachusetts Amherst * Some material adapted from slides from Arvind(MIT) and Jan-Willem Maesson(Sun)

2. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 2 outline Historical Background pH Language Threading in pH Scheduling Where is pH today?

3. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 3 History Dataflow Languages Describe programs as data flows Inherently parallel Examples: Id and Sisal Id Introduced I-structures and M-structures to avoid copying arrays during construction.

4. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 4 History Id worked great on the Monsoon dataflow architecture. 1x2x4x8x Mat. Multiply1.001.993.907.74 Paraffins n=221.001.993.927.25 Particles(40K)1.001.953.817.35 Simple 100 iters1.001.853.456.27 Speed up Data flow architecture effectively supports the execution model.

5. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 5 History Haskell Purely functional programming language Lazy execution model (expressions only evaluated when needed) “Sexy” new type system [Jan-Willem Maesson] People actually use Haskell (?)

6. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 6 pH: parallel Haskell pH = Haskell (syntax, type system) + Id (evaluation order, side-effect ops) Goal: Unite two communities Bring the dataflow and functional communities under a single language Facilitate code sharing

7. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 7 Language Structure pH has 3 layers pH(F) Purely functional Haskell + loops pH(I) pH(F) + I-structures (Id) pH(M) pH(I) + M-structures (Id)

8. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 8 pH(F) Adds for and while loops to Haskell Syntactic sugar (== tail recursion) for sum=0 in for i <- [1..n] do next sum = sum + i finally sum

9. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 9 pH(I) I-structures I-structure = write-once array Reads are delayed until written All reads return a single consistent value No data races

10. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 10 State? Programs without side-effects/state are not usually very useful. Add state using assignment (ML, Scheme) Results in a sequential evaluation order Goal: evaluate sequentially only when necessary.

11. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 11 pH(M) == Full pH M-structures (state) Allows multiple synchronized “takes” and “puts” Take: block until data is written, then remove it Potential for race conditions (use barriers) Useful for classic mark-based graph algorithms Higher performance than pure functional (sometimes) Program Version Executed Instructions Critical PathAverage Parallelism Heap used (words) HA1 HA2 HA3 HA4 HA5 M-structures 199,757 72,847 325,472 266,223 253,370 60,496 1,452 7,324 238 195 28,833 589 138 10 1368 1365 9 103 9,250 2,100 18,423 9 11,301 9

12. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 12 M-Structure Syntax Allocate := M_array(1,n) Allocate := M_array((1,n),(1,m)) Put := A![i] = 5 Take := (A![i] + B![i]) def replace a i v = { x = a![i]; a![i] = v; in x};

13. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 13 Barriers (1) Divide a control region into two subregions that must be evaluated sequentially Syntax = “---” def fib n = if n < 2 then n else { x = fib(n-1); --- y = fib(n-2); in x + y};

14. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 14 Barriers (2) Control parallelism Reduce exponential resource usage Sacrifice parallelism def fib n = if n < 2 then n else { x = fib(n-1); --- y = fib(n-2); in x + y};

15. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 15 Barriers (3) Force sequential evaluation def replace a i v = { x = a![i]; --- a![i] = v; in x};

16. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 16 Implicit Parallelism Program execution == expression reduction Every reduction is evaluated in parallel Exception: lambda expressions and conditionals def fib n = if n < 2 then n else { x = fib(n-1); y = fib(n-2); in x + y};

17. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 17 Implicit Parallelism def fib n = if n < 2 then n else { x = fib(n-1); y = fib(n-2); in x + y}; fib() (-) fib() (+) n12

18. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 18 Parallel Execution Eager evaluation All expressions are reduced in parallel Limited by data dependences and barriers f (4*x) (g 25 (5+6)) Implications Not all Haskell programs will terminate using pH semantics! All pH programs will terminate using Haskell lazy semantics.

19. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 19 Threading in pH

20. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 20 Threading in pH Spawn a new thread only when: There are multiple dependent blocks One of them actually suspends Use strictness analysis to determine what needs to be evaluated.

21. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 21 Scheduling Work stealing (Cilk-style) Follows usual call/return pattern Good temporal locality in practice Low overhead in the common case I-structures/M-structures? Add yourself to the defer list Run the defer list on a write Messes up temporal locality

22. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 22 Where is pH now? Hmmm…..

23. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 23 Implementation Status Limited at best Currently compiles to the Monsoon dataflow machine Coming soon to the UltraSPARC No documentation A few papers, mostly about semantics No support Support for real parallel architectures might encourage involvement from the Haskell community

24. U NIVERSITY OF M ASSACHUSETTS, A MHERST Department of Computer Science 24 Where is pH now? Implementation There might be hope Haskell is actually used Mitre: Speech Recognition System LOLITA: Natural Language Processing System Monadius: a Haskell shoot ‘em up Legacy might live on through Fortress