Web: OMII-UK Campus Grid Toolkit NW-GRID Campus Grids Workshop 31 st October 2007 University of Liverpool Tim Parkinson OMII-UK Southampton Operations Manager

Web: OMII-UK What is OMII-UK? o Collaboration between Open Source Software developers at Southampton, Manchester, Edinburgh. Mission: o OMII-UK aims to provide software and support to enable a sustained future for the UK e-Science community and its international collaborators. Software Solutions for e-Research Led by user / community requirements

Web: Characteristics of a Campus Grid Generally Speaking.... One or more o HPC Clusters o Condor Pools o Specialised Computation Devices or Data Resource o Owned and managed by a single institution o Subject to single AuthN regime and internal AuthZ policies? o Shared by users belonging to the institution o Applications / Services useful to users o Maintenance & support of that environment

Web: Uses of a Campus Grid Typically o To run large scale computations or simulations o That can be partitioned into scenarios or parameter sweeps. o That therefore benefit by running many scenarios in parallel or on larger machines. Reduce Time to Publication Obtain results that would be unobtainable in available time if done sequentially or on a smaller desktop.

Web: Requirements of Campus Grid Users Determined from a survey of the UK Campus Grid SIG membership o Chaired by David Wallom, OeRC Backed up by the SUPER report o Study of User Priorities for e-Infrastructure for e-Research (NeSC 2007) o pdf

Web: SIG Requirements Five areas: Applications How to transparently execute applications across heterogeneous resources to maximise throughput and minimise execution time. Identify common applications for shrink-wrapping. Security Users: Just want to login once and use what they are allowed to use transparently or to be refused gracefully. Admins: Want to be able to apply suitable authentication method and suitable access control policy. Want to support attribute based virtual organisations (VOMS / Shibboleth) to support collaborations.

Web: SIG Requirements Five areas (ctd) Accounting: o Admins: Driven by FEC. Associate usage with user. Ability to log, price, and ultimately to bill. o Users: Any such billing should be fair and trustworthy. Monitoring o Admins: Track resource utilisation and availability o Users: See what is happening to their jobs.

Web: SIG Requirements Five areas (ctd) Storage o Users: ability to seamlessly transfer data to and from a variety of distributed storage systems (such as SRB) or databases. o Admins: ability to configure and such storage mechanisms into job services and to monitor them.

Web: SUPER Requirements Similar to CG SIG Spectrum of User Interaction Styles o Web PortalsWrapped Applications o Desktop GUI o CLI o Scripting Languages o Programmatic APICustom Applications

Web: Quality Requirements Installability o Ease of installation and configuration. Reliability o Should stay up and running or at least fail gracefully Portability / Availability o Should work on a range of different architectures and operating systems and back end job managers. Scalability o Should work for personal installations up to Campus Wide and beyond.

Web: What is the Campus Grid Toolkit? An enhanced packaging of existing and future OMII-UK components, principally GridSAM, that provides o Consistent job submission across heterogeneous resources for the scientist (via OGF standard JSDL) over a Web Services interface. o Ease of installation and configuration for the administrator.

Web: What is the Campus Grid Toolkit Provides (ctd) o a range of interaction styles for the scientist (Desktop, CLI, Portal / Portlets) o a range of configurable security policies for the administrator (OMII-AuthZ, SPAM-GP, VOMS / Shibboleth integration). o a way to wrap legacy, unmodified applications (AHE, OGRSH) o a way to create new applications that access grid resources directly (SAGA and its scripting bindings)

Web: What is the Campus Grid Toolkit? A vehicle to deliver an integrated set of OMII-UK components that work together to enhance the scientist’s ability to submit large numbers of computational jobs to the resources available on campus and beyond in a seamless fashion. Should become the installation of choice for Campus Grid providers.

Web: First Release (Nov 2007) Address the Installability of GridSAM onto a Condor pool. GridSAM on Condor in a Box. Autoconfiguration Better example programs Address reliablity issues Prototype demonstrated at OGF21

Web: The Old Way – Component Based Follow an installation and configuration process (1-2 hours): 1. Install the OMII-Server bundle and select GridSAM as an option 2. Use temporary server certificate 3. Configure the Condor DRM yourself 4. Download and install the client 5. Use temporary client certificate 6. Test with uname application (too trivial) 7. Make client available to users 8. Replace temporary CA and cert with real ones

Web: The New Way Move towards one-step installation and configuration (10 mins): 1. Install CGT which offers the option to trust UK CA and configures server accordingly 2. Attempts to install trust for UK CA and your own ‘real’ certificate 3. Attempts to detect and auto-configure Condor DRM 4. Automatically installs and configures client to match the server that was just installed 5. Test with Mandelbrot set – a real application requiring significant processing time 6. Make pre-configured client available

Web: Future Plans (no particular order) Add one-step installation and auto-configure for AHE (Demo SC2007) Extend one-step installation to other DRMs (Globus, PBS/Torque, Platform LSF etc) Extend GridSAM to propagate user identity and to collect resource usage for jobs. Investigate ways to add grid monitoring, perhaps with OGM Investigate job service brokering perhaps with Knoogle or Grid-BS.

Web: Future Development Enhance all major OMII-UK components to interact with the main storage management solutions.

Web: Questions?