The National Grid Service Guy Warner NeSC Training Team

2 Policy for re-use This presentation can be re-used for academic purposes. However if you do so then please let training- know. We need to gather statistics of re-use: no. of events, number of people trained. Thank you!!training-

3 Acknowledgements This talk was written by Mike Mineter of the NeSC Training Team. Some NGS and GOSC slides are taken from a talk by Stephen Pickles, Technical Director of GOSC Also slides from Malcolm Atkinson on e-Science programme

4 Overview The UK e-science programme Grid Operations Support Centre The NGS

5 Staff costs only - Grid Resources Computers & Network funded separately EPSRC Breakdown UK e-Science Budget ( ) Source: Science Budget 2003/4 – 2005/6, DTI(OST) Total: £213M + Industrial Contributions £25M + £100M via JISC

6 Globus Alliance CeSC (Cambridge) Digital Curation Centre e-Science Institute Open Middleware Infrastructure Institute The e-Science Centres EGEE Grid Operations Support Centre National Centre for e-Social Science National Institute for Environmental e-Science

7 Grid Operations Support Centre

8 GOSC and the National Grid Service The NGS is the core UK grid, intended for the production use of computational and data grid resources. NGS is the core service resulting from the UK's e-Science programme. NGS is supported by JISC, and is run by the Grid Operations Support Centre (GOSC).

9 GOSC The Grid Operations Support Centre is a distributed “virtual centre” providing deployment and operations support for the UK e-Science programme.

10 GOSC Roles  UK Grid Services  National Services  Authentication, authorization, certificate management, VO management, security, network monitoring, help desk + support centre.   NGS Services  Job submission, data transfer, data access and integration, resource brokering, monitoring, grid management services, operations centre,…  NGS core-node Services  CPU, (meta-) data storage, key software  Services coordinated with others (eg OMII, NeSC, LCG):  Integration testing, compatibility & Validation Tests, User Management, training  Administration:  Policies and acceptable use conditions  Resource providers: service level agreements,…  Coordinate deployment and Operations

11 The National Grid Service

12 Scales of collaboration The pattern that is emerging: –Inside an institute: computation (e.g. Condor) and data –Multi-disciplinary in a university: across admin domains (“Campus grids”) –Nation-wide collaboration: National Grid Service (core resources + resources for a VO) –International collaboration: EGEE Campus Grid Meeting on today at NeSC, Edinburgh –

13 Core NGS resources Nodes providing compute services –White Rose Grid, Leeds: grid-compute.leeds.ac.uk –Oxford e-Science Centre: grid-compute.oesc.ox.ac.uk For list of compilers, software,… e.g. Nodes providing data services –Manchester: grid-data.man.ac.uk –Rutherford Appleton Lab. (RAL): grid-data.rl.ac.uk When you have joined the NGS you can access each of the core nodes.

14 NGS software Computation services based on GT2 –Use compute nodes for sequential or parallel jobs, primarily from batch queues –Can run multiple jobs concurrently (be reasonable!) Data services: –Storage Resource Broker: Primarily for file storage and access Virtual filesystem with replicated files –“OGSA-DAI”: Data Access and Integration Primarily for grid-enabling databases (relational, XML) –NGS Oracle service

15

16 Gaining Access NGS core nodes data nodes at RAL and Manchester compute nodes at Oxford and Leeds all access is through digital X.509 certificates –from UK e-Science CA –or recognized peer National HPC services HPCx CSAR Must apply separately to research councils Digital certificate and conventional (username/ password) access supported

17 U of A HPCxHPCx UofD GOSC NGS Core Nodes: Host core services, coordinate integration, deployment and support +free to access resources for all VOs. Monitored interfaces + services NGS Partner Sites: Integrated with NGS, some services/resources available for all VOs Monitored interfaces + services NGS Affiliated Sites: Integrated with NGS, support for some VO’s Monitored interfaces (+security etc.) RAL Oxford Leeds Man. CSARCSAR U of B U of C Commercial Provider PSRE BRISTOL CARDIFF

18 Managing middleware evolution Important to coordinate and integrate this with deployment and operations work in EGEE, LCG and similar projects. Focus on deployment and operations, NOT development. ETF NGS Other software sources Software with proven capability & realistic deployment experience ‘Gold’ services Prototypes & specifications Feedback & future requirements EGEE… Deployment/testing/advice Operations Engineering Task Force UK, Campus and other grids

19 Key facts Production: deploying middleware after selection and testing – major developments via Engineering Task Force. Evolving: –Middleware –Number of sites –Organisation: VO management Policy negotiation: sites, VOs International commitment Gathering users’ requirements – National Grid Service

20 Joining the NGS See talks given at NGS Induction events at NeSC, Edinburgh –

21 Joining the NGS As –Users –Projects (VO’s) –Resource providers

22 How Do I Get A Certificate? You need a valid UK certificate before applying for an NGS account: –Contact the UK Certificate Authority. –You will probably need to provide non-electronic proof of identity to your local representative of the CA. For example: a photocopy of your passport. See –Ensure that you use the same browser (on the same computer) that you sent the request from to download your certificate. –Always keep this certificate secure.

23 How Do I Get An NGS Account? Once you have your certificate: –If your browser has access to your UK Certificate go to (Easiest option.) –Otherwise you will need the Distinguished Name (DN) from your UK Certificate, then go to –You will need to provide a description of your reason for needing access, plus your expected CPU/Disk usage. –Make sure you read and agree to the Terms and Conditions of Usage – see

24 End-users: what to do You need a current UK e-Science Certificate Complete the application form on the NGS web site, and read the conditions of use: – Wait 1-2 weeks for peer review You gain access to all core nodes automatically (including SRB) Contact GOSC to gain access to OGSA-DAI and Oracle Use the NGS and GSC web sites and help-desk Happy computing!

25 Projects and VOs Just need access to compute and data resources for users in your project? –Currently, mainly applications from individuals –project-based applications being dealt with case-by-case, as procedures are established –if in doubt, talk to us! Want to host your data on NGS? –consider GridFTP, SRB, Oracle, or OGSA-DAI –NGS maintains infrastructure –you populate and manage data Want to use NGS resources to provision services, portals for a community of users? –next slide

26 Providing services for a VO NGS resources can be used to provide a portal (or portlets) or other service for your community Deployment and security scenarios are negotiable NGS policies (core nodes): –your portal can present its own, or a delegated user’s credential to NGS, but tasks should be traceable to initiating end-user –you should not run your own services in user space without prior agreement of NGS and hosting site we need to know that services are secure, will not jeopardise operation of other NGS services, or consume too much precious resource on head nodes Talk to GOSC!

27 Resource providers Resource providers join the NGS by Defining level of service commitments Adopting NGS acceptable use and security policies Run compatible middleware –as defined by NGS Minimum Software Stack –and verified by compliance test suite Support monitoring and accounting Two levels of membership 1.Affiliates run compatible stack, integrated support 2.Partners also contribute “significant resources”

28 Web Sites NGS – –To see what’s happening: GOSC – CSAR – HPCx –

29 Summary NGS is a production service –Therefore cannot include latest research prototypes! –ETF recommends what should be deployed Core sites provide computation and also data services NGS is evolving –OMII, EGEE, Globus Alliance all have m/w under assessment by the ETF for the NGS Selected, deployed middleware currently provides “low-level” tools –New deployments will follow soon –New sites and resources being added !