6 th CIC on Duty meeting Lyon 27-29/03/2006 Enabling Grids for E-sciencE Grid INTER-Operations Hélène Cordier EGEE/WLCG Operations IN2P3 Computing Centre.

Slides:



Advertisements
Similar presentations
INFSO-RI Enabling Grids for E-sciencE Update on LCG/EGEE Security Policy and Procedures David Kelsey, CCLRC/RAL, UK
Advertisements

EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks EGEE Operations Ian Bird, CERN IT/GD LHCC.
OSG Operations and Interoperations Rob Quick Open Science Grid Operations Center - Indiana University EGEE Operations Meeting Stockholm, Sweden - 14 June.
The National Grid Service User Accounting System Katie Weeks Science and Technology Facilities Council.
INFSO-RI Enabling Grids for E-sciencE GLOBAL GRID USER SUPPORT THE MODEL AND EXPERIENCE IN LCG/EGEE Gilles Mathieu(1), Torsten Antoni(2),
EGEE ARM-2 – 5 Oct LCG Security Coordination Ian Neilson LCG Security Officer Grid Deployment Group CERN.
INFSO-RI Enabling Grids for E-sciencE SA1: Cookbook (DSA1.7) Ian Bird CERN 18 January 2006.
GGF12 – 20 Sept LCG Incident Response Ian Neilson LCG Security Officer Grid Deployment Group CERN.
SEE-GRID-SCI Regional Grid Infrastructure: Resource for e-Science Regional eInfrastructure development and results IT’10, Zabljak,
Monitoring in EGEE EGEE/SEEGRID Summer School 2006, Budapest Judit Novak, CERN Piotr Nyczyk, CERN Valentin Vidic, CERN/RBI.
EGEE is a project funded by the European Union under contract IST Plan for ROC verification Hélène Cordier - Alistair Mills IN2P3, CRNS, France.
INFSO-RI Enabling Grids for E-sciencE EGEE 1 st EU Review – 9 th to 11 th February 2005 CERN.
GridPP Deployment & Operations GridPP has built a Computing Grid of more than 5,000 CPUs, with equipment based at many of the particle physics centres.
Enabling Grids for E-sciencE System Analysis Working Group and Experiment Dashboard Julia Andreeva CERN Grid Operations Workshop – June, Stockholm.
EGEE-III INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks David Kelsey RAL/STFC,
EGEE is a project funded by the European Union under contract IST User support in EGEE Alistair Mills Torsten Antoni EGEE-3 Conference 20 April.
EGEE-III INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Operations Automation Team James Casey EGEE’08.
GGUS at PEB – –- page 1 LCG Klaus-Peter Mickel, GridKa Karlsruhe LCG-PEB-Meeting ( ) The Global Grid User Support Model (Report of GDB.
EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE Security Coordination Group Linda Cornwall CCLRC (RAL) FP6 Security workshop.
LCG/EGEE Security Operations HEPiX, Fall 2004 BNL, 22 October 2004 David Kelsey CCLRC/RAL, UK
EGEE-III INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks SA1: Grid Operations Maite Barroso (CERN)
Summary of AAAA Information David Kelsey Infrastructure Policy Group, Singapore, 15 Sep 2008.
EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE Security Coordination Group Dr Linda Cornwall CCLRC (RAL) FP6 Security workshop.
EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks The EGEE User Support Infrastructure Torsten.
INFSO-RI Enabling Grids for E-sciencE ARDA Experiment Dashboard Ricardo Rocha (ARDA – CERN) on behalf of the Dashboard Team.
Site Manageability & Monitoring Issues for LCG Ian Bird IT Department, CERN LCG MB 24 th October 2006.
DTI Mission – 29 June LCG Security Ian Neilson LCG Security Officer Grid Deployment Group CERN.
INFSO-RI Enabling Grids for E-sciencE An overview of EGEE operations & support procedures Jules Wolfrat SARA.
Security Policy: From EGEE to EGI David Kelsey (STFC-RAL) 21 Sep 2009 EGEE’09, Barcelona david.kelsey at stfc.ac.uk.
EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Communication tools between Grid Virtual.
PIC port d’informació científica EGEE – EGI Transition for WLCG in Spain M. Delfino, G. Merino, PIC Spanish Tier-1 WLCG CB 13-Nov-2009.
EGEE-II INFSO-RI Enabling Grids for E-sciencE Operations procedures: summary for round table Maite Barroso OCC, CERN
EGEE ARM-2 – 5 Oct LCG/EGEE Security Coordination Ian Neilson Grid Deployment Group CERN.
JSPG Update David Kelsey MWSG, Zurich 31 Mar 2009.
Kati Lassila-Perini EGEE User Support Workshop Outline: – CMS collaboration – User Support clients – User Support task definition – passive support:
INFSO-RI SA2 ETICS2 first Review Valerio Venturi INFN Bruxelles, 3 April 2009 Infrastructure Support.
INFSO-RI Enabling Grids for E-sciencE User and Virtual Organisation Support in EGEE Flavia Donno, CERN Torsten Antoni, FZK Alistair.
EGEE-III INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Operations Automation Team Kickoff Meeting.
Mardi 8 mars 2016 Status of new features in CIC Portal Latest Release of 22/08/07 Osman Aidel, Hélène Cordier, Cyril L’Orphelin, Gilles Mathieu IN2P3/CNRS.
Mercredi 9 mars 2016 CIC Portal/COD Activities Hélène Cordier IN2P3/CNRS Computing Centre, Lyon, France.
Operations model Maite Barroso, CERN On behalf of EGEE operations WLCG Service Workshop 11/02/2006.
Open Science Grid OSG Resource and Service Validation and WLCG SAM Interoperability Rob Quick With Content from Arvind Gopu, James Casey, Ian Neilson,
Opensciencegrid.org Operations Interfaces and Interactions Rob Quick, Indiana University July 21, 2005.
EGEE-III INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks What all NGIs need to do: Helpdesk / User.
EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks EGEE Operational Procedures (Contacts, procedures,
EGI-Engage is co-funded by the Horizon 2020 Framework Programme of the European Union under grant number GGUS Service Provider GGUS –
EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Security aspects (based on Romain Wartel’s.
EGI Process Assessment and Improvement Plan – EGI core services – Tiziana Ferrari FedSM project 1EGI Process Assessment and Improvement Plan (Core Services)
1 Grid Service Monitoring James Casey, CERN IT-GD WLCG/OSG Operations Meeting 14th June 2007.
EGEE-III INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks The Dashboard for Operations Cyril L’Orphelin.
 Daily Operations for EGEE/LCG infrastructure Hélène Cordier EGEE/LCG Operations IN2P3 Computing Centre Lyon (France) -
NAREGI Lyon 04/07/2007 Enabling Grids for E-sciencE Global Grid Operations and their Tools Hélène Cordier EGEE/WLCG Operations IN2P3 Computing Centre Lyon.
INFSO-RI Enabling Grids for E-sciencE EGEE general project update Fotis Karayannis EGEE South East Europe Project Management Board.
EGEE-III INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks COD-16 (Transition to EGEE-III) Report to.
EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks Operations automation team presentazione.
Scuola Grid - Martina Franca, Thursday 08 November Il Sistema di Supporto INFNGrid & GGUS ( Global Grid User.
EGI-InSPIRE RI EGI-InSPIRE EGI-InSPIRE RI Operations Portal OTAG September, 21th 2011 Cyril L’Orphelin – CCIN2P3/CNRS.
Enabling Grids for E-sciencE EGEE-II INFSO-RI ROC managers meeting at EGEE 2007 conference, Budapest, October 1, 2007 Admin Matters Vera Hanser.
CERN WLCG Grid Storage Systems Deployment Flavia Donno, CERN 6 November 2007 Organization of Storage Support through GGUS Flavia Donno CERN/IT-GD CERN.
James Casey, CERN IT-GD WLCG Workshop 1st September, 2007
Operations Interfaces and Interactions
User Support Workflow in EGEE
LCG Security Status and Issues
Ian Bird GDB Meeting CERN 9 September 2003
LCG/EGEE Incident Response Planning
Grid Service Monitoring Working Group
The CCIN2P3 and its role in EGEE/LCG
Maite Barroso, SA1 activity leader CERN 27th January 2009
EGEE: Grid Operations & Management
EGEE Operation Tools and Procedures
Presentation transcript:

6 th CIC on Duty meeting Lyon 27-29/03/2006 Enabling Grids for E-sciencE Grid INTER-Operations Hélène Cordier EGEE/WLCG Operations IN2P3 Computing Centre Lyon (France) -

2 Enabling Grids for E-sciencE Contents Existing Common Interests in solving mainly 2 issues so far: –Security and accounting issues, monitoring workflow efforts are diverse. Existing efforts at inter-project level involving: –Grid Interoperability Now (GIN, as a workgroup from OGF) Existing efforts at project level involving: –EGEE, WLCG and OSG –NDGF, PRAGMA, TERAGRID and NAREGI Existing efforts at IN2P3-CC: –IGTMD Concerns and Updates

3 Enabling Grids for E-sciencE Security & Policy Joint Security Policy Group Certification Authorities – EUGridPMA  IGTF and so one. Grid Acceptable Use Policy (AUP) – common, general and simple AUP – for all VO members using many Grid infrastructures e.g. EGEE, OSG, SEE-GRID, DEISA, national Grids… Incident Handling and Response – defines basic communications paths – defines requirements ( must s) for IR – not to replace or interfere with local response plans Security & Availability Policy Usage Rules Certification Authorities Audit Requirements Incident Response User Registration & VO Management Application Development & Network Admin Guide VO Security Grid Security Policy (v5.7) : Grid Site Operations Policy (v1.4): Virtual Organisation Operations Policy (v1.0):

4 Enabling Grids for E-sciencE Usage record working group Mandate : In order for resources to be shared, sites must be able to exchange basic accounting and usage data in a common format. This working group proposes to define a common usage record based on those in current practice. The record format will be specific enough to facilitate information sharing among grid sites, yet general enough that the usage data can be used for a variety of purposes - traditional usage accounting, service usage monitoring, perfomance tuning, etc. This group will therefore be concentrating on collecting and disseminating resource consumption data. We will not be addressing how that data is to be collected by the resource sites, nor how it will be used by its recipients.

5 Enabling Grids for E-sciencE Accounting Tools needed to collect and report information on resource utilization – Intended audience: site managers, virtual organization managers, grid operators, funding agencies,… – Need to define common ways of measuring resource consumption  Including usage of same units LCG/EGEE – CPU usage information (per user or per VO) provided by each site and stored in a central repository : Reports (charts and numeric data) available through a web interface – Next step: collect information on storage utilization. – Developed and operated by Grid Operations Centre (UK) and CESGA (SWE).

6 Enabling Grids for E-sciencE Accounting – Cont’d

7 Enabling Grids for E-sciencE Accounting

8 Enabling Grids for E-sciencE Accounting

9 Enabling Grids for E-sciencE High-Level Model Site monitoring

10 Enabling Grids for E-sciencE Site monitoring (cont’d) We can’t/won’t impose a solution on sites, as they might/should have something Already. Specification based approach allows our probes fit into any fabric monitoring system : Data Exchange format allows higher-level services consume the data regardless of fabric monitoring system WLCG Monitoring Working Groups since January 23 rd 2007: System Management Working Group – SMWG /J. Casey, I. Neilson Grid Service Monitoring Working Group – GSMWG / A. Forti, M. Jouvin System Analysis Working Group – SAWG / J. Andreeva, P. Saiz [Rob Quick, Workshop on Grid services Monitoring HPDC’07 – June 27th 2007]

11 Enabling Grids for E-sciencE CMS Dashboard 1/2

12 Enabling Grids for E-sciencE CMS Dashboard 2/2

13 Enabling Grids for E-sciencE CIC Operations Portal Web portal for integrating all the tools and sources of operations-related information into one single place Developed and operated by CC-IN2P3, failover instance at CNAF – – Provides and maintains an integrated operations dashboard for grid on duty operator – Provides mechanisms for keeping information needed for appropriate hand over between operators on duty – Easy access to appropriate contact information on every actor involved in the operations of the grid – Provides communication tools

14 Enabling Grids for E-sciencE Alarms Dashboard

15 Enabling Grids for E-sciencE Opening tickets

16 Enabling Grids for E-sciencE Tracking incidents via GGUS Incident tracking model –Unique channel for opening tickets  End-users : e.g job submission failures, data transfer failed  Operators : e.g job submission failures –Classification and 1rst assignment done by the ticket process manager –Tickets are assigned to support units - one per domain of expertise  Grid operators, applications, federations, m/w experts,..  OSG : Automatic helpdesk/ XML Format Exchange  4 tickets created by cms users from June 27th WLCG/EGEE –Central incident tracking tool : –Same tool used by grid operators and end users via and web interface –Sites failing the tests receive are assigned a ticket  Escalation procedure for solving site-related problems  Involves the regional operator and the site operator Interface with ticket handling tools used by sites/federations (if needed) Tools for collecting metrics on the responsiveness of support units

17 Enabling Grids for E-sciencE The ENOC The EGEE Network Operations Centre (ENOC): –Single point of contact between EGEE and the NRENs –Where EGEE and the network can exchange operational information –Network support unit in GGUS ENOC

18 Enabling Grids for E-sciencE IGTMD Grid Interoperability and Massive Data Transfer 3 years, started in Feb 2006 Renater, ENS, CC-IN2P3, FNAL-unfunded Goals 1.Disk to disk Bulk data transfer 2.Replication and referring mechanisms 3.Information Sytem and job management interoperability 4.Grid control and monitoring 5.Usage of statistics and accounting data

19 Enabling Grids for E-sciencE IGTMD Roadmap Network: items 1and 2 –2* 1 Gb/s CC-IN2P3/FNAL on October 16th 2006 – LCG/EGEE –Tests on Massive Data transfer – CC-IN2P3/FNAL Interoperability: item 3 –Access to grid resources through standard APIs – LCG/EGEE –State-of-the art cf. JTR – October17th; –RoadMap on the IGTMD face-to face meeting May 4th Inter-operations: items 4 to 5 –Tests suite relevancy to US sites – EGEE –Operations and Daily Monitoring of services – EGEE –Usage Records and accounting – OGF

20 Enabling Grids for E-sciencE Concerns and updates Achieve a real 24x7 production quality-like service : Failover mechanisms Increase automation of daily monitoring tools and alarms treatment. OGF20—GIN JOBS - EGEE/TERAGRID/OSG/NORDUGRID/DEISA nvironmentOGF20https://forge.ogf.org/sf/wiki/do/viewPage/projects.gin/wiki/WorkerNodeE nvironmentOGF20 29/08/ /03/2007 mail from Laurence Field on GIN-JOB GIN-OPS : Savannah and Ninf-G GIN-IS :EGEE-NDGF and EGEE-OSG not updated since 17 Août 2006 GIN-data :idem GIN-auth : AUP for the gin.gg.org VO since 12/06.

21 Enabling Grids for E-sciencE Credits and References Gstat – GGUS – GOC-DB – SAM – – CMS Dashboard – GridIce – Lavoisier – CIC Operations Portal EGEE WLCG Slides from : Ian Bird - OGF/EGEE User Forum - May 9th 2007 Rob Quick, Workshop on Grid services Monitoring HPDC’07 – June 27th 2007

22 Enabling Grids for E-sciencE Links SAM/GridView Monitoring Portal: TWiki: SAM OSG Probe Dev Homepage: (Service Availability Monitor) Test Page: TWiki: GridICE Monitoring Portal: Documentation: Experiment Dashboard Portal: TWiki: GridPP Real Time Monitor Homepage: (2D map and 3D globe visualizations) GStat Portal: TWiki: Lemon Portal (CERN Compute Center): Documentation: