SICSA student induction day, 2009Slide 1 Social Simulation Tutorial Session 6: Introduction to grids and cloud computing International Symposium on Grid Computing Taipei, Taiwan, 7 th March 2010

SICSA student induction day, 2009Slide 2 e-Research Increased collaboration in Science, across institutional, disciplinary and national boundaries Need for resource sharing: –Data –Computation –Scientific instruments –Remote collaboration & visualisation Development of distributed computing infrastructures to support research

SICSA student induction day, 2009Slide 3 e-Research Drivers Increased computational capacity & capability Increased instrumentation and data capture Increased reuse of data Lowered costs of instrumentation Community data and collaborations New forms of scholarly communications Network capacity & ubiquity Grand challenges in research Societal Needs

SICSA student induction day, 2009Slide 4 HPC vs. HTC High Performance Computing –Tightly coupled distributed systems –Single owner and location –Single machine illusion, largely homogenous –Standards: MPI and OpenMP High Throughput Computing –Loosely coupled distributed systems –Distributed and provided by multiple parties, heterogeneuous –Aimed at High Throughput Computing –Emerging standards driving interoperability

SICSA student induction day, 2009Slide 5 The EGEE Grid Infrastructure EGEE, EGEE-II, EGEE III projects funded to provide a production quality grid infrastructure for Europe and beyond, to support research in all disciplines Driven by the needs of the high energy physics community Certified middleware installation Support mechanisms Virtual organisation management Shared resources, security Largest multi-disciplinary grid infrastructure worldwide

SICSA student induction day, 2009Slide 6 612/3/06 EGEE Expertise & Resources More than 90 partners 32 countries 12 federations  Major and national Grid projects in Europe, USA, Asia + related projects: –BalticGrid, SEE-GRID, EUMedGrid, EUChinaGrid, EUIndiaGrid, EELA, EUAsiaGrid

SICSA student induction day, 2009Slide 7 Towards EGI EGEE is evolving to become the European Grid Initiative (EGI) Move towards a sustainable model beyond project based funding through federated structure based on national grid initiatives (NGIs) Central coordinating organisation EGI.eu Each country represented in EGI Council via its NGI Specialist Support Centres to provide support for scientific communities Links with other regional initiatives such as the Latin-American Grid Initiative and EUAsiaGrd

SICSA student induction day, 2009Slide 8 Grids and Social Simulation Grids provide primarily an HTC service, so are best suited to run ‘parameter sweeps’, where each simulation run executes on a single worker node. Using MPI, it is also possible to develop simulation models that make use of multiple worker nodes at the same time. Parallelising simulation code like this, however, is not easy and there is a question whether it would not be better to first utilise larger machines (more Cores, more memory) A crucial feature of grids is control over where data gets stored and code executed as well as security.

SICSA student induction day, 2009Slide 9 USING EGEE

SICSA student induction day, 2009Slide 10 EGEE Components gLite Middleware: the ‘glue’ that binds diverse resources together Public key infrastructure – securely identify people VO management system – authorise access Information system – what resources are available? Workload management system – submitting jobs Compute elements – contain the worker nodes that run the jobs (Storage Elements – manage data)

SICSA student induction day, 2009Slide 11 Job Submission Overview Authenticate List available resources Define job Submit job – stage in data Monitor execution Stage out outputs

SICSA student induction day, 2009Slide 12 g-Eclipse Eclipse-based environment (cf. RePast!) Provides access to different grid resources Using graphical user interface rather than command line We will use it to access resources in the Gilda training infrastructure

SICSA student induction day, 2009Slide 13 Clouds Many definitions Here: Infrastructure as a Service (IaaS) Specifically: Amazon EC2 Provides virtual machines on demand Better demonstrated than explained…

SICSA student induction day, 2009Slide 14 Jobs on Clouds How does a job get to get executed on a cloud? Could define AMI image to execute job on startup –Disadvantage: have to modify image for each change Could ssh / remote desktop into machine and start job –Disadvantage: does not scale Could deploy grid middleware (gLite, etc.) –Disadvantage: complex Or use a very simple job submission mechanism…

SICSA student induction day, 2009Slide 15 svn2cloud Very basic job submisson mechanism based on subversion source control system