1. A Functional Language for Departmental Metacomputing Frederic Gava & Frederic Loulergue Universite Paris Val de Marne Laboratory of Algorithms, Complexity and Logic

2. Concurrent Programming Automatic Parallelisation Structured Parallelism High Level Programming Deadlock free, deterministic Parallel Algorithms (BSP)

3. CMPPs CMPP 98: BS -calculus (Loulergue, Hains, Foisy) A confluent calculus of functional Bulk Synchronous Parallel programs CMPP 98: BS -calculus (Loulergue, Hains, Foisy) A confluent calculus of functional Bulk Synchronous Parallel programs CMPP 2000: Bulk Synchronous Parallel ML (Hains, Loulergue) A library for Objective Caml based on BSl CMPP 2000: Bulk Synchronous Parallel ML (Hains, Loulergue) A library for Objective Caml based on BSl CMPP 2004: Departmental Metacomputing ML CMPP 2004: Departmental Metacomputing ML

4. The Caraml Project 2002-2004 French National Grid Program 3 Phases: Improvement of the safety of the existing parallel programming libraries Improvement of the safety of the existing parallel programming libraries Multiprogramming on one cluster: Parallel Compositions for BSML ( Iccs, Europar ) Multiprogramming on one cluster: Parallel Compositions for BSML ( Iccs, Europar ) Programming clusters (cluster of clusters) Programming clusters (cluster of clusters)http://www.caraml.org

5. Outline Bulk Synchronous Parallel ML The Bulk Synchronous Parallel model The Bulk Synchronous Parallel model BSML primitives BSML primitives + Minimally Synchronous Parallel ML The Message Passing Machine model The Message Passing Machine model MSPML primitives MSPML primitives = Departemental Metacomputing ML DMML primitives DMML primitives Semantics Semantics Implementation Implementation

6. Bulk Synchronous Parallelism BSP machine: p processor-memory pairs + network+ global synchronisation unit BSP machine: p processor-memory pairs + network+ global synchronisation unit BSP execution: sequence of super-steps BSP execution: sequence of super-steps Asynchronous computation phase Asynchronous computation phase Communication phase Communication phase Global synchronisation Global synchronisation BSP cost model (one super-step): BSP cost model (one super-step): (max 0<=i<p w i ) + h*g + l

7. The BSMLlib Library For Objective Caml For Objective Caml No SPMD No SPMD Deadlock free, deterministic semantics Deadlock free, deterministic semantics Usual Objective Caml programs + operations on a parallel data structure Usual Objective Caml programs + operations on a parallel data structure Parallel vector of size p:  par (no nesting) Parallel vector of size p:  par (no nesting) Access to the BSP parameters: bsp_p, bsp_g, bsp_l Access to the BSP parameters: bsp_p, bsp_g, bsp_l

8. BSML Primitives (1) mkpar: (int->  ) ->  par mkpar f = mkpar: (int->  ) ->  par mkpar f = apply: (  ) par ->  par ->  par apply = apply: (  ) par ->  par ->  par apply = projection: at projection: at

9. BSML Primitives (2) put: (int->  option) par -> (int->  option) par put = put: (int->  option) par -> (int->  option) par put = If (f 0 2) = None “0 will send no value to 2” (f 0 3) = Some v“0 will send value v to 3” Then (g 2 0)=None “2 received nothing from 0” (g 3 0)=Some v “3 received value v from 0” If (f 0 2) = None “0 will send no value to 2” (f 0 3) = Some v“0 will send value v to 3” Then (g 2 0)=None “2 received nothing from 0” (g 3 0)=Some v “3 received value v from 0” BSMLlib 0.25: http://bsmllib.free.fr BSMLlib 0.25: http://bsmllib.free.fr

10. The Message Passing Machine model machine: p processor-memory pairs + network machine: p processor-memory pairs + network execution: seen as sequence of m-steps execution: seen as sequence of m-steps Asynchronous computations Asynchronous computations Asynchronous communications Asynchronous communications cost model: cost model: p: number of processors p: number of processors g: time required to send one word g: time required to send one word L: network latency L: network latency

11. MSPML Primitives Same primitives than BSML Same primitives than BSML But for communications: get instead of put But for communications: get instead of put get:  par -> int par ->  par get = get:  par -> int par ->  par get = communication environments communication environments http://mspml.free.fr http://mspml.free.fr

12. Departemental Metacomputing ML machine: cluster of BSP clusters within the same organisation machine: cluster of BSP clusters within the same organisation contains the whole BSML language contains the whole BSML language based on departmental vectors:  dep based on departmental vectors:  dep int dep: all proc. of the same BSP unit contain the same value int dep: all proc. of the same BSP unit contain the same value int par dep: possibly different values int par dep: possibly different values

13. Access to Parameters Parameters of the BSP units: Parameters of the BSP units: (dm_bsp_p i): number of proc. of BSP unit i (dm_bsp_p i): number of proc. of BSP unit i (dm_bsp_g i) (dm_bsp_g i) (dm_bsp_l i) (dm_bsp_l i) (dm_bsp_s i): processor speed of BSP unit I (dm_bsp_s i): processor speed of BSP unit I Parameters of the metacomputer: Parameters of the metacomputer: dm_p(): number of BSP units dm_p(): number of BSP units dm_g(): time required to echange 1 word between two units dm_g(): time required to echange 1 word between two units dm_l(): inter-units network latency dm_l(): inter-units network latency

14. Primitives mkdep: (int->  ) ->  dep mkdep: (int->  ) ->  dep dm_p()=3 dm_bsp_p = [3;2;4] mkdep(fun c->mkpar(fun i->c+2*i)) [,, ] apply: (  ) dep ->  dep ->  dep apply: (  ) dep ->  dep ->  dep projection: atdep projection: atdep

15. Communications get: (int-> int-> int option) par dep -> (* f *) (int->  option) par dep -> (* g *) (int-> int->  option) par dep (* h *) get: (int-> int-> int option) par dep -> (* f *) (int->  option) par dep -> (* g *) (int-> int->  option) par dep (* h *) On cluster a at process i: On cluster a at process i: (f b j)=Some n : wants the n th value of process j on cluster b (f b j)=Some n : wants the n th value of process j on cluster b (h b j)= Some v: values sent by process j of cluster b (h b j)= Some v: values sent by process j of cluster b On cluster b at process j: On cluster b at process j: (g n) = Some v (g n) = Some v

16. Formal Semantics Operational semantics: Operational semantics: mini-ML (purely functional)+ DMML operations mini-ML (purely functional)+ DMML operations Confluence Confluence Formal costs: Formal costs: C: expression -> cost formula

17. Implementation & Experiments Library for Objective Caml: Library for Objective Caml: MPI for intra BSP unit communications MPI for intra BSP unit communications TCP/IP for inter BSP units communications TCP/IP for inter BSP units communications Experiments … Experiments … not yet: 2 months ago no cluster dedicated to research Future metacomputer: Future metacomputer: 12 nodes PIII cluster+FastEthernet (classroom) 12 nodes PIII cluster+FastEthernet (classroom) 6 nodes PIV cluster + Gigabit Ethernet 6 nodes PIV cluster + Gigabit Ethernet Virtual Private Network Virtual Private Network

18. Future Work (1) Algorithms & implementation in DMML Algorithms & implementation in DMML Benchmark suite: Benchmark suite: recently done for BSML recently done for BSML Ongoing work for MSPML & DMML Ongoing work for MSPML & DMML Modular implementation (parametric modules) Modular implementation (parametric modules) BSMLlib 0.3 in september (MPI,PVM,PUB,TPCP/IP) BSMLlib 0.3 in september (MPI,PVM,PUB,TPCP/IP) Abstraction done for MSPML Abstraction done for MSPML Ongoing for DMML Ongoing for DMML

19. Future Work (2) Type system: Type system: to avoid nesting of par and dep to avoid nesting of par and dep interaction with imperative features interaction with imperative features Certification: Program extraction from Coq proofs Certification: Program extraction from Coq proofs Fault tolerance Fault tolerance