Grid Remote Execution of Large Climate Models (NERC Cluster Grid) Dan Bretherton, Jon Blower and Keith Haines Reading e-Science Centre Environmental Systems Science Centre University of Reading, UK

Main themes of presentation Sharing HPC clusters used for running climate models  Why share clusters  Grid approach to cluster sharing (NERC Cluster Grid: UK Environmental Res. Council)‏ G-Rex Grid middleware Large climate models as grid services Please also see demonstration and poster

Background Many NERC institutes now have HPC clusters  Beowulf clusters with commodity hardware  Common applications are ocean, atmosphere and climate models Pressure to justify spending and increase utilisation  Sharing clusters helps increase utilisation  Sharing clusters facilitates collaborations Running climate models on remote clusters in traditional way is not easy

Using remote clusters the traditional way Input data Output data Local Remote 100 GB SCP SSH Model input and output Model setup, including source code, work-flow scripts, model input and output

Computational challenges of Climate models Typical requirements  Parallel processing (MPI) with large number of processors (usually )‏  Each cluster needs high speed interconnection (e.g. Myrinet or Infiniband)‏  Long runs lasting several days  Large volumes of output  Large number of separate output files

NEMO Ocean Model (eg. European Operational Oceanogr.) Main parameters of a typical 1° Global Assimilation run for a one year:  Run with 40 processors  2-3 hours per year on Cluster Outputs 300 MB in 700 separate files as diagnostics every 5-10 minutes Output for a one year is roughly 20 GB, a total of separate files 50-year `Reanalysis` = 1Tb. Model automatically re-submitted as a new job each year

NERC Cluster Grid Includes 3 clusters so far... (plans for 11 clusters)  Reading (64 procs.), Proudman, (360 pr.), British Antarctic Survey (160 pr.)‏ Main aim  Make it easier to use remote clusters for running large models Key features  Minimal data footprint on remote clusters  Easy job submission and control  Light-weight grid middleware (G-Rex)  Load and performance monitoring (Ganglia)  Security

Grid Remote EXecution G-Rex is light-weight grid middleware Implemented in Java using Spring framework G-Rex server is a Web application  Allows applications to be exposed as services  Runs inside a servlet container G-Rex client program, grexrun, behaves as if the remote service were actually running on the user's own computer Security based on HTTP digest authentication

NEMO G-Rex service: Deployment scenario 1 Client Server NEMO launch scripts and forcing data (same every run)‏ Input and output via HTTP Port 9092 G-Rex server Tomcat port open to client Apache Tomcat G- Rex client NEMO model setup, including source code, work-flow scripts, input data and output from all runs

NEMO G-Rex service: Deployment scenario 2 Client Server NEMO launch scripts and forcing data (same every run)‏ Input and output via HTTP Port 9092 G-Rex server Apache Tomcat G- Rex client NEMO model setup, including source code, work-flow scripts, input data and output from all runs

Advantages of G-Rex Output continuously transferred back to user  Job can be monitored easily  No data transfer delay at end of run Files deleted from server when no longer needed  Prevents unnecessary accumulation of data  Reduces data footprint of services Work-flows can be created using shell scripts Very easy to install and use See Poster; Demonstration also available