1. 1 CCA Meeting, Januray 25th 2007 Supporting the Master-Worker Paradigm in the Common Component Architecture Hinde Lilia Bouziane, Christian Pérez, Thierry Priol Christian Pérez, Thierry Priol PARIS research-team IRISA / INRIA Rennes – France contact: Christian.Perez@inria.fr

2. 2 CCA Meeting, Januray 25th 2007 Outline of the talk  Context –Master-worker applications and environments –Software component models  Limits with existing component models  Our proposition: a generic model –Objectives –Overview of the proposed model  Master and workers abstract composition  Request scheduling integration  Cases of study: CCA  Conclusion

3. 3 CCA Meeting, Januray 25th 2007 Introduction  Context –Parametric applications –Several kinds of resources  multi-core processors  SMP machines  clusters  grids –Dedicated environments  BOINC, DIET, NetSolve, Nimrod/G, Ninf,...  Challenges –Simplifying application programming –Independence from resource kinds –HPC u

4. 4 CCA Meeting, Januray 25th 2007 Master-worker environments  Many environments: –BOINC, XTremWEB, XTremWEB, DIET, DIET, NetSolve, Nimrod/G,... NetSolve, Nimrod/G,...  Characteristics: –Scalable –Request delivery policy –Transparent request delivery management for the programmer –Dedicated API –Only master-worker paradigm is supported Reactive transport in Hydrogeology u PrecipitationSorption Relargage Convection Dispersion Aqueous Reactions Gaz-liquid exchange Dissolution Biology worker Master worker Workers collection Requests transport Scheduling Fault tolerance

5. 5 CCA Meeting, Januray 25th 2007 Software component models  Black box  Ports –Method invocations, events/messages/streams  Several component models –Common Component Architecture/CCA Forum (CCA) –CORBA Component Model/OMG (CCM) –Fractal/ObjectWeb –Grid.it-ASSIST/UNIPI –Etc. Software component PROVIDED PORTS REQUIRED PORTS (client interfaces) Software component (server interfaces)

6. 6 CCA Meeting, Januray 25th 2007 A simple master-worker application example master interface Compute { long f (in long value); } ; Master: ForAll i= 0 to N { call f (..) on a worker through yellow port; } worker Worker: long f (long value) { // computation return res; }

7. 7 CCA Meeting, Januray 25th 2007 Limits with existing component models (1/2) m wW W w10 m  Resources infrastructure dependence –SMP machine vs Cluster  Request delivery implementation at the burden of the master –No transparency –Complex

8. 8 CCA Meeting, Januray 25th 2007 Limits with existing component models (2/2) W W w35 m interceptor W W w111 m proxy  Request delivery policy outside but fixed –Resources infrastructure dependence! –For grids, complex request delivery implementations –No transparency

9. 9 CCA Meeting, Januray 25th 2007 A generic model to support master-worker paradigm

10. 10 CCA Meeting, Januray 25th 2007 Features of the model  Resources infrastructure independence  Transparency –No dealing with the number of workers –No dealing with request delivery concerns  An initial number of workers depending on the current resources infrastructure  Introduction of a request delivery policy depending on the current resources infrastructure

11. 11 CCA Meeting, Januray 25th 2007 Abstract master and worker composition (1/2): a collection  Collection definition Exposed provided port worker W1 Wn W2 Wi Type1 Type3 Type2 Type1 T1x Type2 T2y Type3 T3z  Collection at execution Instantiation

12. 12 CCA Meeting, Januray 25th 2007 Abstract master and worker composition (2/2): abstract assembly Composing components with collections ≈ Component composition Component composition m Type1Type3 Type2 X

13. 13 CCA Meeting, Januray 25th 2007 Integration of a request delivery policy (1/2): transformation  Abstract ADL ->concrete ADL Round-Robin Deployment environment Transformation process Round-Robin

14. 14 CCA Meeting, Januray 25th 2007 Integration of a request delivery policy (2/2): patterns w w w M Round-Robin / Random LA M w w w MA Simple component based pattern Hierarchical scheduling pattern DIET pattern M Random w w Round-Robin w w

15. 15 CCA Meeting, Januray 25th 2007 Overview of the proposal Programmer/designer view Collection binding master worker Exposed provided port Resources infrastructure Round-Robin Framework implementation view #workers + Pattern selection Set of request transport mechanism patterns 1.Random 2.Round-Robin 3.NetSolve 4.Diet 5.…

16. 16 CCA Meeting, Januray 25th 2007 Extending the Common Component Architecture

17. 17 CCA Meeting, Januray 25th 2007 Overview of the CCA  All is done at runtime  Examples of specified SIDL interfaces –BuilderService  Component creation –createInstance(… )  Composition –connect(… ) –Services  Port declarations –addProvidesPort(…) –registerUsesPort(…)  Getting a port reference –getPort(…) fw A createInstance(A) setServices registerUsesPort(aP) aP connect(aP, bP) B createInstance(B) setServices addProvidesPort( bP ) bP

18. 18 CCA Meeting, Januray 25th 2007 Mapping principle  Create new interfaces for –collection management  Creation, adding element, binding ports  Let the collection be a concrete component –pattern management  Declare, instantiate it

19. 19 CCA Meeting, Januray 25th 2007 Master-worker paradigm in CCA user view   Collection and binding specification – – Services : 1 op – – BuilderService : 3 ops – – others: 2 interfaces (5 ops)   Pattern specification (optional) – –1 interface for pattern representation (1 op implemented by scheduling expert ) – – BuilderService : 1 op explicit choice of a pattern   Dynamic collection management (optional) – – 1 interface for adding/removing elements (2 ops) – – Services : 1 op – – BuilderService : 2 ops interface BuilderService2 {... CollectionID createInstanceCollection( in string instanceCollName, in string className, in TypeMap properties); BindingID bind(in CollectionID collID, in string collPortName, in ComponentID elem, in string elemPortName); array getBindingIDs(...); } interface CollectionManagement : Port { bool addElement(in CollectionID collID); bool subElement(in CollectionID collID); } interface Services3 { void setCollManagementPort(in string portName, in collectionManagement port); } interface BuilderService3 {... bool addElementToColl(in CollectionID collID); bool subElementToColl(in CollectionID collID); }

20. 20 CCA Meeting, Januray 25th 2007 Master-worker paradigm in CCA User View i nterface BuilderService2 : BuilderService { CollectionID createInstanceCollection (in string instanceCollName, in string className, in TypeMap properties); BindingID bind (in CollectionID collID, in string collPortName, in ComponentID elem, in string elemPortName); … } Interface PatternInstantiation {…} Interface CollectionManagement {…} master createInstance(master) fw setServices registerUsesPort(mP) createInstanceCollection (coll) worker coll setServices addProvidesPort(cP) loop : createInstance(wrk) bind(cP, wP) mP cP wP connect(mP, cP) 1 2 3

21. 21 CCA Meeting, Januray 25th 2007 Master-worker paradigm in CCA (2/2)   Request delivery Pattern specification (optional) – – 1 interface for pattern representation (1 op implemented by scheduling expert )   Dynamic collection management (optional) – – 1 interface for adding/removing elements (2 ops) – – Services : 1 op – – BuilderService : 2 ops addElement(collID) subElement(collID) collManager instantiatePattern (collID) fw pattern

22. 22 CCA Meeting, Januray 25th 2007 Impacts on the CCA framework implementation   Re-implementation of the BuilderService interface – –connect operation   Identifying a provides collection port to instantiate an associated pattern – –getPort operation   Implement “new” interfaces   Framework should provide a default pattern

23. 23 CCA Meeting, Januray 25th 2007 Conclusion  Improving the support of master-worker paradigm in CCA –Collections, request delivery policy patterns –Resources infrastructure independent programming –Adaptation of workers and requests delivery at the burden of the framework  Impact on CCA –Additional interfaces and limited runtime modification  Ongoing feasibility study with MOCCA  Discussions –How CCA component placement should be handle? –Multiple/Collection uses port will simplify life –How to generate the specific code for the “proxies”?