GWpilot: a personal pilot system A.J. Rubio-Montero, E. Huedo and R. Mayo-García EGI Technical Forum 2012 Prague – 20 Sep 2012

EGI TF 2012 – Prague, 20 Sep Outline  Common problems in Grid computation  Pilot Jobs  GWpilot  Advantages  Utilisation  Design and improvements  Suitability  DKEsG  Revision  Description of the calculation performed  Performance measurements  Some results  Conclusions

EGI TF 2012 – Prague, 20 Sep Common problems in Grid Computation  Variable overheads: queue waiting times, overload...  Variable error rate: connection cuts, jobs arbitrary aborted…  Diverse configurations: complexity, unexpected types of WN…  To increase performance by means of scheduling, the resources must be completely characterised, but it is impossible with generic middleware:  Design defects: GLUE specification is incomplete: no broadband, latency, queue policy, average waiting time, resource profile…  Miss-configurations and lacks of maintenance  Solutions  Self-scheduling  Models  Heuristics  Statistics  Pilot jobs

EGI TF 2012 – Prague, 20 Sep Pilot Jobs: basics SLOT APPROPIATION pilot Coordinator Server task pilot LRMS queue CE pilot Pilot Factory task monitoring task pilot

EGI TF 2012 – Prague, 20 Sep Pilot jobs: benefits  Reduce the Grid complexity:  direct use and characterization of assigned WNs  direct monitoring user tasks.  Fix task dispatching overheads  remove the waiting time in remote queues  remove middleware overheads and errors (CREAM,GRAM)  Reduce task error rate: middleware, hardware or connectivity  Increase compatibility  creating special configurations  Implementing legacy communication protocols  Allows the implementation of advanced scheduling techniques

EGI TF 2012 – Prague, 20 Sep Pilot Jobs: Implementations  Centralized frameworks: daunting maintenance, deployment and customization.  AliEn and PanDa (suitable for HEP users)  DIRAC  glideinWMS  EDGeS (XtremWeb) and GridBot (BOINC)  Application-oriented: mono-user, mono-application.  DIANE  They are not exploiting all the scheduling advantages provided by pilot jobs or they lack compatibility or adaptability aspects  Alternative  GWpilot

EGI TF 2012 – Prague, 20 Sep GWpilot: features  Easy-to-install and standalone from remote middleware  Highly customizable and tuneable, even by unskilled users  Multi-user with fair-share policies  Compatible with previously ported applications  Interoperable with diverse Grid infrastructures  Lightweight and scalable, achieving nearly optimal performance  Advanced scheduling policies for any kind of tasks

EGI TF 2012 – Prague, 20 Sep GWpilot: simplicity of use and configuration  GWpilot makes the use of pilot jobs automatic and unattended both to users and developers: # cat ls_template.jt EXECUTABLE = /bin/ls STDOUT_FILE = logs/ls.out.${ARCH}.${JOB_ID} STDERR_FILE = logs/ls.err.${ARCH}.${JOB_ID} REQUIREMENTS = LRMS_NAME = "jobmanager-pilot" RANK = CPU_MHZ # gwsubmit -t ls_template.jt  Usual configuration parameters  maximum of submitted pilots  dispatching suspension timeout (maximum time spent at remote LRMS)  pilot pulling interval against GWpilot and number of retries Users have only to fix this requirement in their tasks # cat gwd.conf … IM_MAD = pilot_im:gw_im_mad_pilot::dummy:pilot_em EM_MAD = pilot_em:gw_em_mad_pilot:-m 550 –t i 45 -f 20 :rsl_nsh …

EGI TF 2012 – Prague, 20 Sep GWpilot: integrated into GridWay metascheduler GridWay Core CREAMGRAM GWpilot Server MSD2 GLUE CLIDRMAABESJSDL Applications Scheduler Allows submitting a % more pilots than the estimated free slots More accurate estimation of free slots pilot task CREAM CE site-BDII GLOBUS CE site-BDII pilot HTTPS pull BDII task GWpilot Factory pilots task

EGI TF 2012 – Prague, 20 Sep GWpilot: suitability for distributed applications  Could give a boost to your computational challenges !!!  Legacy applications previously ported to GridWay or to DRMAA/BES/JSDL standards can directly benefit from GWpilot.  Examples:  Truba/MaRaTra  VMEC  ISDEP  FAFNER-2  gGEM  DKEsG : Drift Kinetic Equation solver for Grid

EGI TF 2012 – Prague, 20 Sep DKEsG: calculating NC transport of Fusion devices * 1 A. J. Rubio-Montero et al. “Drift Kinetic Equation Solver for Grid (DKEsG),” IEEE Trans.Plasma Sci., 38( 9) * 2 D. A. Spong, “Generation and damping of neoclassical plasma flows in stellarators,” Phys. Plasmas, 12(5), Fluxes through the surfaces generated by the magnetic field lines: DRMAA-enabled producer-consumer workflow: chunking tasks and polling time for BoT states are customizable The DKEsG Workflow * 1 Updated with Spong’s DKEs code* 2 NC transport coefficients

EGI TF 2012 – Prague, 20 Sep Experiment: DKEsG parameter scan with the TJ-II standard configuration r[2…141] X EFIELD[-250…250:10] X CMUL[(1…9)10(-7…0)] = 514,080 independent tasks (1 to 12 min proportional to radius) 420 tasks 103,236 independent BoTs 5 tasks X BoT 6.58 years on Intel Xeon X5365 3GHz (64bit)

EGI TF 2012 – Prague, 20 Sep Resources used and configuration bounds GISELA infrastructure (prod.vo.eu-eela.eu) Discarded 32bits, duplicate and CERN resources. Used for submitting pilots  DKEsG only submits 500 BoTs  BoT Susp. timeout: 60 secs  Limited to 100 jobs per resource  queues overloaded with 15% more pilots  suspension timeout: 5 hours Max pilots submitted by GWpilot  Other configuration parameters:  Pilot pulling interval : 45 seconds with 20 retries.  DKEsG polling time: 15 secs.  Resources are ranked/prioritised based on CPU speed

EGI TF 2012 – Prague, 20 Sep Experiment: measured computational results Many overloaded sites Pilots die when a BoT is running inside Suspended BoTs because they have been assigned to death pilots Only 0.4% failing BoTs 62% failing grid jobs Makespan: 94 h: 27 m: 31 s. 610 times faster than sequential Total time wasted at remote queues: 1 year and 20 days. Not appreciable by the user.

EGI TF 2012 – Prague, 20 Sep Experiment: turnaround measurements DKEsG cannot supply enough BoTs The number of available pilots are lower than 500 Pilot overhead is always between secs  Scalability of GWpilot Accumulated turnaround overhead is only 6.21%. If only GridWay were used the resultant one* would be % * A.J. Rubio-Montero et al, “Executions of a Drift Kinetic Ecuation solver on Grid,” in PDP 2010, Pisa, Italy.

EGI TF 2012 – Prague, 20 Sep Plasma results: bootstrap current (D 13 ) of the outer radial plasma position in TJ-II (negative polarization) This surface needs: 3672 DKEsG-Mono tasks = 663 CPU hours consumed from the Grid Boostrap current tends to zero  The collisionless asymptotic value (which depends on the configuration) is recovered.  Larger uncertainties appear in the long mean free path regime. As expected, the coefficients are even in the electric field.

EGI TF 2012 – Prague, 20 Sep Plasma results: normalized NC transport coefficients (L 33 ) from the resistivity enhancement (positive polarization)  CMUL and EFIELD parameters decrease monotonically as K increases.  By solving the integration in K, there is a continuous reutilization of the data included in the database

EGI TF 2012 – Prague, 20 Sep Conclusions  Summary  GWpilot is suitable to easily improve the performance of several kinds of fusion codes.  New features have been implemented in GridWay and DKEsG.  DKEsG execution shows impressive improvements in terms of makespan and turnaround.  Future work  Continue the DKEsG calculations in order to build an extensive database for several fusion devices that allows the user to read the monoenergetic coefficients and to obtain the final fluxes without performing again the calculations.  We are evaluating GWpilot with other applications from other scientific areas.  More information at:  

EGI TF 2012 – Prague, 20 Sep Thanks for your attention