1. GWpilot: a personal (or institutional) pilot system
A.J. Rubio-Montero, E. Huedo and R. Mayo-García
EGI Technical Forum 2012
Prague – 20 Sep 2012

2. Outline Common problems in Grid computation Pilot Jobs GWpilot DKEsG
Advantages
Utilisation
Design and improvements
Suitability
DKEsG
Revision
Description of the calculation performed
Performance measurements
Some results
Conclusions
EGI TF 2012 – Prague, 20 Sep 2012

3. Common problems in Grid Computation
Variable overheads: queue waiting times, overload...
Variable error rate: connection cuts, jobs arbitrary aborted…
Diverse configurations: complexity, unexpected types of WN…
To increase performance by means of scheduling, the resources must be completely characterised, but it is impossible with generic middleware:
Design defects: GLUE specification is incomplete: no broadband, latency, queue policy, average waiting time, resource profile…
Miss-configurations and lacks of maintenance
Solutions
Self-scheduling
Models
Heuristics
Statistics
Pilot jobs
EGI TF 2012 – Prague, 20 Sep 2012

4. Pilot Jobs: basics monitoring Pilot Factory Coordinator Server CE task
LRMS
queue CE
pilot
pilot
pilot
task
task
SLOT APPROPIATION
task
task
task
pilot
EGI TF 2012 – Prague, 20 Sep 2012

5. Pilot jobs: benefits Reduce the Grid complexity:
direct use and characterization of assigned WNs
direct monitoring user tasks.
Fix task dispatching overheads
remove the waiting time in remote queues
remove middleware overheads and errors (CREAM,GRAM)
Reduce task error rate: middleware, hardware or connectivity
Increase compatibility
creating special configurations
Implementing legacy communication protocols
Allows the implementation of advanced scheduling techniques
EGI TF 2012 – Prague, 20 Sep 2012

6. Pilot Jobs: Implementations
Centralized frameworks: daunting maintenance, deployment and customization.
AliEn and PanDa (suitable for HEP users)
DIRAC
glideinWMS
EDGeS (XtremWeb) and GridBot (BOINC)
Application-oriented: mono-user, mono-application.
DIANE
They are not exploiting all the scheduling advantages provided by pilot jobs or they lack compatibility or adaptability aspects
Alternative  GWpilot
EGI TF 2012 – Prague, 20 Sep 2012

7. GWpilot: features
Easy-to-install and standalone from remote middleware
Highly customizable and tuneable, even by unskilled users
Multi-user with fair-share policies
Compatible with previously ported applications
Interoperable with diverse Grid infrastructures
Lightweight and scalable, achieving nearly optimal performance
Advanced scheduling policies for any kind of tasks
EGI TF 2012 – Prague, 20 Sep 2012

8. GWpilot: simplicity of use and configuration
GWpilot makes the use of pilot jobs automatic and unattended both to users and developers:
# cat ls_template.jt
EXECUTABLE = /bin/ls
STDOUT_FILE = logs/ls.out.${ARCH}.${JOB_ID}
STDERR_FILE = logs/ls.err.${ARCH}.${JOB_ID}
REQUIREMENTS = LRMS_NAME = "jobmanager-pilot"
RANK = CPU_MHZ
# gwsubmit -t ls_template.jt
Users have only to fix this
requirement in their tasks
Usual configuration parameters
maximum of submitted pilots
dispatching suspension timeout (maximum time spent at remote LRMS)
pilot pulling interval against GWpilot and number of retries
# cat gwd.conf …
IM_MAD = pilot_im:gw_im_mad_pilot::dummy:pilot_em
EM_MAD = pilot_em:gw_em_mad_pilot:-m 550 –t i 45 -f 20 :rsl_nsh
EGI TF 2012 – Prague, 20 Sep 2012

9. GWpilot: integrated into GridWay metascheduler
task
task
task
Applications
pilots
More accurate
estimation of free slots
BES
JSDL
CLI
DRMAA
GWpilot
Factory
Scheduler
GridWay Core
Allows submitting a % more pilots than the estimated free slots
MSD2
GLUE
CREAM
GRAM
GWpilot
Server
HTTPS
pull
BDII
CREAM CE
site-BDII
pilot
site-BDII
GLOBUS CE
task
pilot
EGI TF 2012 – Prague, 20 Sep 2012

10. GWpilot: suitability for distributed applications
Could give a boost to your computational challenges !!!
Legacy applications previously ported to GridWay or to DRMAA/BES/JSDL standards can directly benefit from GWpilot.
Examples:
Truba/MaRaTra
VMEC
ISDEP
FAFNER-2
gGEM
DKEsG : Drift Kinetic Equation solver for Grid
EGI TF 2012 – Prague, 20 Sep 2012

11. DKEsG: calculating NC transport of Fusion devices
Fluxes through the surfaces generated by the magnetic field lines:
NC transport coefficients
The DKEsG Workflow*1
Updated with Spong’s DKEs code*2
DRMAA-enabled producer-consumer workflow:
chunking tasks and polling time for BoT states are customizable
*1 A. J. Rubio-Montero et al. “Drift Kinetic Equation Solver for Grid (DKEsG),” IEEE Trans.Plasma Sci., 38( 9)
*2 D. A. Spong, “Generation and damping of neoclassical plasma flows in stellarators,” Phys. Plasmas, 12(5), 2005.
EGI TF 2012 – Prague, 20 Sep 2012

12. Experiment: DKEsG parameter scan with the TJ-II standard configuration
EFIELD[-250…250:10]
CMUL[(1…9)10(-7…0)] =
514,080 independent tasks
(1 to 12 min proportional to radius)
420 tasks
5 tasks X BoT
103,236 independent BoTs
6.58 years on Intel Xeon X5365 3GHz (64bit)
EGI TF 2012 – Prague, 20 Sep 2012

13. Resources used and configuration bounds
Max pilots submitted
by GWpilot
DKEsG only submits 500 BoTs
BoT Susp. timeout: 60 secs
Used for submitting pilots
GISELA infrastructure
(prod.vo.eu-eela.eu)
Discarded 32bits, duplicate
and CERN resources.
Limited to 100 jobs per resource
queues overloaded with 15% more pilots
suspension timeout: 5 hours
Other configuration parameters:
Pilot pulling interval : 45 seconds with 20 retries.
DKEsG polling time: 15 secs.
Resources are ranked/prioritised based on CPU speed
EGI TF 2012 – Prague, 20 Sep 2012

14. Experiment: measured computational results
Pilots die when
a BoT is
running inside
Suspended BoTs
because they have been
assigned to death pilots
Only 0.4% failing BoTs
Many overloaded sites
62% failing grid jobs
Makespan: 94 h: 27 m: 31 s.
610 times faster than sequential
Total time wasted at remote queues: 1 year and 20 days.
Not appreciable by the user.
EGI TF 2012 – Prague, 20 Sep 2012

15. Experiment: turnaround measurements
DKEsG cannot supply enough BoTs
The number of available pilots are lower than 500
Pilot overhead is always between secs  Scalability of GWpilot
Accumulated turnaround overhead is only 6.21%.
If only GridWay were used the resultant one* would be 61.48%
* A.J. Rubio-Montero et al, “Executions of a Drift Kinetic Ecuation solver on Grid,” in PDP 2010, Pisa, Italy.
EGI TF 2012 – Prague, 20 Sep 2012

16. Plasma results: bootstrap current (D13) of the outer radial plasma position in TJ-II (negative polarization)
This surface needs:
3672 DKEsG-Mono tasks = 663 CPU hours consumed from the Grid
Boostrap current
tends to zero
As expected, the coefficients are
even in the electric field.
The collisionless asymptotic value (which depends on the configuration) is recovered.
Larger uncertainties appear in the long mean free path regime.
EGI TF 2012 – Prague, 20 Sep 2012

17. Plasma results: normalized NC transport coefficients (L33) from the resistivity enhancement (positive polarization)
CMUL and EFIELD parameters decrease monotonically as K increases.
By solving the integration in K, there is a continuous reutilization of the data included in the database
EGI TF 2012 – Prague, 20 Sep 2012

18. Conclusions Summary Future work More information at:
GWpilot is suitable to easily improve the performance of several kinds of fusion codes.
New features have been implemented in GridWay and DKEsG.
DKEsG execution shows impressive improvements in terms of makespan and turnaround.
Future work
Continue the DKEsG calculations in order to build an extensive database for several fusion devices that allows the user to read the monoenergetic coefficients and to obtain the final fluxes without performing again the calculations.
We are evaluating GWpilot with other applications from other scientific areas.
More information at:
EGI TF 2012 – Prague, 20 Sep 2012

19. Thanks for your attention
EGI TF 2012 – Prague, 20 Sep 2012