1. 10/26/20151 GC16/3011 Functional Programming Lecture 21 Parallel Graph Reduction

2. 10/26/20152 Contents  Sequential evaluator  Parallel graph reduction  Lazy/Strict/Non-strict evaluation  Strictness analysis  Elements of parallel graph reduction  Issues

3. 10/26/20153 Sequential evaluator  A sequential evaluator  Normal order (lazy)  Algebraic type = tree  Tree reduction  Can make tree into a graph  Graph reduction  Parallel GR: many sequential evaluators all working on one graph  With additional support for synchronisation and communication

4. 10/26/20154 Making things easier  Turn all functions into combinators  No free variables!  “Lift” all lambdas to top level  Named functions have no embedded lambdas  So no name clashes, no free variable capture  Easy beta reduction!  “supercombinator reduction”

5. 10/26/20155 Lazy/Strict/Non-strict evaluation  Normal order reduction (“lazy evaluation”) is SEQUENTIAL  Alternative: Applicative order (“strict evaluation”)  Parallel GR CANNOT be sequential!  Does PGR therefore lose benefits of lazy evaluation?  No! As long as it only evaluates those bits of the graph that normal order reduction would have evaluated  “non-strict” evaluation  Use “Strictness Analysis”

6. 10/26/20156 Strictness analysis  Done at compile time  For (f x y z) which of x, y and/or z does f always evaluate using normal order reduction?  If (f endless y z) = endless, then assume f MUST evaluate x  Halting problem?  Use approximation technique  Abstract Interpretation

7. 10/26/20157 Elements of parallel GR  Many processors running sequential evaluators  Graph exists in shared heap(s)

8. 10/26/20158 Processor Shared heap Processor Processor Processor Processor@ x5 @ @ + 3 x

9. 10/26/20159 Elements of parallel GR  Processors  Each evaluates a subgraph to (weak head) normal form  Slightly modified for Inter-Process Communication and synchronisation  Values communicated via graph  Two processors might evaluate the same sub-graph  just wasted effort, cannot affect correctness  But prefer not to waste the effort!

10. 10/26/201510 Elements of parallel GR  A processor evaluates one or more tasks  Tasks/threads “to be done” in shared task pool(s)  Synchronisation required whenever a task requires the value of a subgraph that is currently being evaluated by another task  First task blocks, waiting for second task to finish  Processor need not block (can start another task)

11. 10/26/201511 Processor Processor Processor Processor Processor Shared heap @ x5 @ @ + 3 x Shared task pool

12. 10/26/201512 Sparking  Strictness analysis tells us all of the subgraphs that definitely need to be evaluated  Graph is annotated  At start, just one task descriptor is placed in task pool  One evaluator will start evaluating it  Evaluators check annotations and place task descriptors (sparks) into the task pool  Other evaluators will start work on the sparked tasks

13. 10/26/201513 Issues  How things are represented in the heap  How the evaluators work  Shared memory (including Virtual SM) or distributed memory  Synchronising tasks (e.g. block and resume)  Scheduling, task distribution and load balancing

14. 10/26/201514 Summary  Sequential evaluator  Lazy/Strict/Non-strict evaluation  Strictness analysis  Elements of parallel graph reduction  Issues