Master Worker Paradigm Support in Software Component Models Hinde Bouziane, Christian Pérez PARIS Research Team INRIA/IRISA Rennes ANR CIGC LEGO (ANR-05-CICG-11) Bordeaux, 2006, December 11 th

Master-worker applications Understanding very high energy cosmic rays AUGER Project (LRI/LAL-IN2P3) XtremWeb Neuron Simulation with MCELL CRPC Rice University NetSolve

Problem overview  Simultaneous independent computations (~ForAll loop)  Dedicated API/environments  BOINC, XTremWEB, DIET, NetSolve, Nimrod/G,... worker Master worker Workers collection Requests transport Scheduling Fault tolerance

Limits with current component models  Different infrastructures  Multi-core processors, SMP, clusters, grids, etc.  Resources dependant properties  Number of workers  Request transport and scheduling policy  At the burden of the programmer  Complex  No transparence  Objectives  Transparency  Re-use existing MW environments W W worker master request delivery policy W W worker

Features of the model  Resources infrastructure independence  Transparency  No dealing with the number of workers  No dealing with request delivery concerns  An initial number of workers depending on the current resources infrastructure  Introduction of a request delivery policy depending on the current resources infrastructure

Abstract master and worker composition (1/2): a collection  Collection definition Exposed provided port worker W1 Wn W2 Wi Type1Type3 Type2 Type1 T1x Type2 T2y Type3 T3z  Collection at execution Instantiation

Abstract master and worker composition (2/2): abstract assembly Composing components with collections ≈ Component composition m Type1Type3 Type2 X

Integration of a request delivery policy (1/2): transformation  Abstract ADL ->concrete ADL Round-Robin Deployment environment Transformation process Round-Robin

Integration of a request delivery policy (2/2): patterns w w w M Round-Robin / Random LA M w w w MA Simple component based pattern Hierarchical scheduling pattern DIET pattern M Random w w Round-Robin w w

Sum up deployment environment Transformation Round-Robin Programmer/designer view Components Collections Abstract assembly During deployment phase #workers + Pattern selection Set of patterns

Extending the CORBA Component Model

Case of study: CCM  Master and worker component definition  Interface Description Language (IDL3)  Collection definition in two steps  External view: IDL3 extension  Internal content and bindings: Collection Description Language (CDL) in XML format interface Compute {..} ; component master { uses Compute m_port; }; component worker { provides Compute w_port; }; collection coll { provides Compute c_port; }; <internPort elemType=“worker“ port=“w_port"/> c_port worker masterworker w_port m_port

Experimenting MW in CCM  Synthetic MW application  Benchmark  Several flavor of the master  Sequential or parallel operation invocations  Latency & bandwidth oriented request  Regular or irregular request bench  Work generated by one request  Several transport request policy component  None (direct connection from master to workers)  Round-robin proxy  Random proxy  DIET

MW CCM & DIET  Client & Server-side DIET Component  Wrapper  Conversion between IDL & DIET data representation  Home-made CCM compiler  DIET support  Generate adequate code for client & server side components  Control with special comments in IDL3 LA Client Component Server component w w MA M w w

Discussion  MW support for Component Model  Map to CCM, CCA & Fractal  Easy to port application  All the work is to componentize the applications  Two applications have been ported  Experiments are in progress  What to measure?  Which API?  Data dypes  Bunch of calls (~ForAll loops?)  DIET deployed as CCM executable  CSD supports executable  DIET hierarchy described in CAD files!  Current ADAGE planner is not aware of DIET specificity  Deployment control through associate element of the ADAGE control parameters